Component Architecture
WorkPackage Report - Luca De Feo
1st reporting period
## Component architecture
### Components
- **User interfaces**: Jupyter, SageMathCloud;
- **Systems**: GAP, PARI/GP, SageMath, Singular;
- **Software libraries**: LinBox, MPIR, ...;
- **Modules**, **packages**, ...;
- **Databases**: LMFDB, MathHub, ....
How do we want them? **Flexible**, **Portable**, **Modular**,
**Open**.
--
### Goals
- **Flexible.** Enable components to be *flexibly combined*;
- **Portable.** Enable components *run smoothly* across a wide range
of environments (e.g., cloud, local, server, ...);
- **Modular.** Develop, demonstrate, standardize APIs;
- **Open.** Streamline user/developer workflows.
--
### Examples
- *SageMath's elliptic curves* module calls *C functions in PARI/GP*
library.
- Import *data from LMFDB* at the *GAP command line*.
- Run commands on *remote Singular server* from a *GAP client* through
*MMT mediator*.
- Deploy *SageMath virtual machines* on Windows machines in *Microsoft
Azure cloud*.
- Deploy a computational mathematics software stack on a *HPC
cluster*, controlled through *Jupyter notebooks*.
--
### Inputs
- **WP4 →** User interface components;
- **WP5 →** HPC-enabled components;
- **WP6 →** Interface languages, ontologies;
- **WP7 →** User behaviour.
### Outputs
- Network protocol implementations **→ WP4, WP6**;
- Tools, infrastructure (for testing, bug reporting, packaging, distributing) **→ WP7**;
- Micromagnetic components **→ WP4**.
---
## Tasks
### T3.1, Portability
- (mostly) Windows;
- Testing, continuous integration.
### T3.2, Interfaces between systems
- Application level interfaces;
- Network protocols.
**Delivered:** D3.3, SCSCP support.
--
### T3.3, Modularisation and packaging
- Virtual machines, containers;
- Packaging and distributing user code;
- Packaging for popular repositories (Debian, Anaconda, ...).
**Delivered:** D3.1, Virtual images and containers.
### T3.4, Simulagora integration
- Commercial VRE, showcasing OpenDreamKit components.
- Interest for the project: measure effort needed to produce a
Debian-based VM.
*Note:* Internal task, no public output.
--
### T3.6, Document and modularise SageMathCloud's codebase
- Collaboration on the *open source software* powering the commercial
service SageMathCloud.
- Share technologies, know-how.
**Delivered:** D3.2, Understand and docuement SageMathCloud backend
### T3.7, Improving the development workflow in mathematical software
- Experiments with improving user/developer experience in VREs;
- Integration into SageMathCloud.
--
### T3.8, Python interface for OOMMF micromagnetic simulation library
- See H. Fanghor's talk.
- Completed!
*Note:* no deliverables in this WP.
### T3.5, Component architecture for High Performance Computing and Parallelism
- Experiment: integrate outputs of WP5 into SageMath;
*Note:* Only starts in month 36.
---
## Deliverables
11 deliverables were submitted.
- D3.10: Packaging components and user-contributed code for major Linux distributions
- D3.11: HPC enabled Sage distribution
- D3.1: Virtual images and containers
- D3.2: Understand and document SageMathCloud backend code.
- D3.3: Support for the SCSCP interface protocol in all relevant components ( Sage, GAP, etc.) distribution
- D3.4: Personal SageMathCloud : single user version of SageMathCloud distributed with Sage.
- D3.5: Integration between SageMathCloud and Sage's TRAC server
- D3.6: Open package repository for Sage
- D3.7: One-click install Sage distribution for Windows with Cygwin 32bits and 64bits
- D3.8: Continuous integration platform for multi-platform build/test.
- D3.9: Semantic-aware Sage interface to GAP.
--
### D3.10: Packaging components and user-contributed code for major Linux distributions
**Output:** *Virtual images and containers for most of the relevant components of
OpenDreamKit.*
- MathHub, <https://hub.docker.com/r/kwarc/>;
- GAP, <https://hub.docker.com/u/gapsystem/>;
- OOMMF <https://github.com/fangohr/virtualmicromagnetics>;
- PARI/GP, SageMath, Singular, ...
<https://hub.docker.com/u/sagemath/>.
Based on modern and popular technologies: Ansible, Docker, Vagrant,
VirtualBox.
**Partners involved:** JacobsUni, UPSud, USouthampton, USTAN, UVersailles.
--
### D3.11: HPC enabled Sage distribution
**Output:** *Published
[documentation on the internals of SageMathCloud](https://github.com/sagemathinc/smc/tree/master/src/doc/design_overview)*
- Part of a planned interaction with SageMathCloud (SMC).
- Development cycles in SMC are very short, architecture, technologies
change quickly.
- ODK benefits from keeping track of SMC internal details.
- SMC benefits from publishing better documentation.
**Partners involved:** UPSud.
--
### D3.1: Virtual images and containers
**Output:** *Implementations of SCSCP in GAP, Python, MMT/MathHub*
- SCSCP = *Symbolic Computation Software Composability Protocol*.
- Network protocol (XML-based) for *RPC* on *semantically annotated*
mathematical data.
- Developed in EU FP6 project 026133 *SCIEnce*, standardized by the
*OpenMath consortium*.
- Implementations already available in Java, C, C++. **This
deliverable:** extend spectrum of available implementations.
**Partners involved:** CNRS, JacobsUni, USTAN, UVSQ.
--
### GAP
- OpenMath language implementation: <https://gap-packages.github.io/openmath/>;
- SCSCP implementation: <https://gap-packages.github.io/scscp/>.
Adds SCSCP support to: *GAP, Singular*
### MathHub
- OpenMath language implementation;
- SCSCP implementation: <https://mathhub.info/>.
--
### Python
- OpenMath language implementation: <https://pypi.python.org/pypi/openmath>;
- SCSCP implementation: <https://pypi.python.org/pypi/scscp>.
Adds SCSCP support to: *Python, LMFDB, SageMath, PARI/GP* (via D4.10)
---
## Achievements
- Two workshops on *modularization and packaging*, great occasion to
interact with the community:
- Sage Days 77, March 2016, Cernay (France);
- Sage Days 85, March 2017, Cernay (France).
- Interactions with other work packages: WP4, WP6.
- OOMMF task (Python interface to OOMMF) completed. See H. Fanghor
talk.
---
## Upcoming work
Four deliverables due on month 24:
- D3.4: *Single user* SageMathCloud.
- D3.5: Integration between SageMathCloud and Trac (SageMath bug
tracking system).
- D3.6: Open package repository for SageMath.
- D3.7: One-click install SageMath distribution for Windows.
--
### D3.4: Single user SageMathCloud
**Good news!**
Already done by SMC developers, will re-allocate efforts.
### D3.5: Integration between SageMathCloud and Trac
Ongoing discussions with SMC developers.
### D3.6: Open package repository for SageMath
Current plan is to provide *cookiecutter* templates for hosting
packages on PyPI (most popular Python package repository).
--
### D3.7: One-click install SageMath distribution for Windows.
This deliverable is mostly done:
- Erik Bray (UPsud) has already completed a one-click install
distribution for Windows based on Docker (not original plan, but
important milestone).
- The latest version of SageMath compiles on Cygwin!
---
## Longer term
- **Packaging:** With help from the community, SageMath will be in the
next Debian (Linux) release. More than 24 months ahead of schedule!
- **Packaging (bis):** experimental *Conda* packages for SageMath and
components are being made, with the help of the community. This is
not a planned deliverable, but it would be a pity to miss this
opportunity.
- **GAP/SageMath Interface:** experimental work started in
January 2016. First prototype complete.
