OpenDreamKit2024-02-23T15:57:09+00:00http://opendreamkit.org/The OpenDreamKit consortiumcontact@opendreamkit.orgLa Science Ouverte dans un Projet Européen de logiciel de recherche?2021-11-18T00:00:00+00:00http://opendreamkit.org/2021/11/18/odk-and-open-science<section data-markdown="" data-separator="^---\n" data-separator-vertical="^----\n">
<script type="text/template">
<!--
## La Science Ouverte dans un Projet Européen de logiciel de recherche?
Secrétariat Permanent de la Science Ouverte, MESRI
18 novembre 2021
<img src="/public/images/rp3-meeting-rehearsal.jpg" align="right" width="55%"/>
Nicolas M. Thiéry<br>
LISN, Université Paris-Saclay
<br><br><br>
!-->
## La Science Ouverte dans un Projet Européen de logiciel de recherche?
1. Contexte: logiciels métier pour la recherche (en mathématiques)
2. OpenDreamKit?
3. Données Ouvertes, Publications Ouvertes, Logiciels Libres
4. Montage et gestion ouvert!
5. Opinions
---
## Contexte: logiciels métier pour les mathématiques (pures)
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
- Pour quoi faire?
- **Calculer**, **Tabuler**
- **Explorer**, **découvrir** et **tester des conjectures**
⇒ *Mathématiques Expérimentales*
- **Enseigner**
- (voire **Formaliser**, **Prouver**, hors champ pour aujourd'hui)
----
### 2000-2020: Évolution du paysage du calcul
- de **logiciels en silo** (Maple, Mathematica, Matlab, GAP, ...)
- vers un **écosystème d'écosystèmes libres**
avec de l'interopérabilité et des fondements communs
dont environnements interactifs Jupyter, langages (Julia, R, Python,
C++, ...), bibliothèques (scipy, pandas, scikit-learn, ...)
- grâce au logiciel libre, à la science ouverte
----
### Exemple: SageMath (2005-)
- combinaison de centaines de bibliothèques libres
plus 1.5 million de lignes de Python/Cython
- modélisation de milliers de types d'objets mathématiques
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
- modèle de développement: **par les utilisateurs pour les utilisateurs**
- des centaines de contributeurs dans le monde
- mérites intrinsèques (co-construction)
- adaptation aux (maigres) ressources disponibles
- limites: travail de fond (maintenance, ingénierie)
---
## OpenDreamKit (2015-2019) #676541
<img src="/public/images/banner.png" align="right" width="55%" style="opacity:0.5;filter:alpha(opacity=40);"/>
<center>
Open Digital Research Environment Toolkit<br>
for the Advancement of Mathematics<br>
[OpenDreamKit.org](OpenDreamKit.org)
</center>
**Consortium**: [15 sites, 50 participants](http://opendreamkit.org/partners)
... avec la communauté internationale!
**Budget**: 7.6 M€
**Objectif**: soutenir l'écosystème des logiciels libres pour les **maths pure et applications**
- Ingénierie logicielle: portabilité, interoperabilité, distribution, HPC, ...
- Environnements interactifs basés sur Jupyter
- Animation de la communauté
**Programme**: [Horizon 2020](https://ec.europa.eu/programmes/horizon2020/), [European Research Infrastructures](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures)
**Appel à projet**: Environnements Virtuels de Recherche
---
## OpenDreamKit et Science Ouvertes
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
### OpenDreamKit et Publications Ouvertes (Trivial)
Longue tradition de publications libres en maths / info:
- Archives ouvertes: arxiv.org, ... (Green Open Access)
- Transition en cours de nombreuses revues (Diamond Open Access)
Question:
- Publication dans des revues en Gold Open Access?
Finalement non: pas besoin
----
## OpenDreamKit et Logiciel Libre (Facile)
- ~~Réaliser un produit?~~
- Contribuer à un écosystème
Bénéfices:
- Construire sur les épaules des géants
- Travail collaboratif avec la communauté
- Pérennité
À noter:
- Existence de modèles économiques éprouvés
- Collaboration avec des PME: Logilab, QuantStack, CoCalc, ...
- Groupe Logiciel Libre du Pôle de Compétitivité Systematic
----
## OpenDreamKit et Données Libres
### Données en mathématiques?
- Résultats de calculs (parfois très lourds)
- Résultats de recherche:
Atlas des groupes simples: plusieurs décennies de travail par toute
une communauté de chercheurs
- Données très riches, très structurées, très diverses
- Frontière données / calcul floue
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
<p style="margin-bottom:15%"></p>
### «Facile»
- Pas de problèmes éthiques
- Peu de données gigantesques
- Peu de problèmes de reproductibilité
- Tradition de partage dans la communauté
- Gestion de la propriété Intellectuelle
- Plan de Gestion des Données (Data Management Plan): Simple liste de jeux de données
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
<p style="margin-bottom:15%"></p>
### Difficile
- FAIR en pratique:
Findable, Accessible, Interoperable, Reusable
- Formation
----
## Une innovation: montage et gestion de projet ouverts
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
### Mobiliser la communauté et favoriser l'intelligence collective
- Invitation large à la participation à la définition du projet
- Écriture publique du projet
- Construction du bas vers le haut:
- Analyser les besoins de la communauté
quelles tâches requièrent un gros investissement?
- Chercher un appel à projet adéquat
(Cellule Europe!!!)
- Trouver une histoire sincère qui les lie tous
«a smart and creative interpretation of the call»
----
### Impliquer les participants
#### Développer une **vision partagée** du project
- Qui sommes nous?
- Quels sont nos objectifs?
- De quoi partons nous?
- Quelle est notre stratégie?
- Comment notre projet se positionne dans le paysage?
- Pour quoi sommes nous l'équipe rêvée pour la tâche?
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
<p style="margin-bottom:15%"></p>
#### Encourager
- À définir ensemble la vision du project
- À agir selon leurs convictions: c'est eux les experts!
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
<p style="margin-bottom:15%"></p>
#### Faciliter
- À quel stade en est la rédaction du projet?
- Que reste-t'il à faire?
- Comment peuvent-ils aider?
- **Outils collaboratifs**
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
<p style="margin-bottom:15%"></p>
#### Bienveillance, exigence et joie
<center>
**Sourire, courir devant, espérer ... et remercier**
</center>
<center>
[Quand cela fonctionne](https://www.youtube.com/watch?v=kM9zcfRtOqo)
</center>
----
## Montage et gestion ouverte du projet: une clé du succès
- Éthique et qualité
Expression sincère des besoins (propres / de la science)
- Esprit d'équipe et confiance réciproque
- Soutien de la communauté
- Simplicité, agilité, cohérence
- Fraîcheur et sincérité
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
**Distinction**: [Étoiles de l'Europe 2020](https://opendreamkit.org/2020/12/16/etoiles-de-l-europe/)
---
## Conclusions
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
### Science Ouverte dans un Projet Européen
- Choix éthique
- Choix pragmatique:
Ingrédient essentiel d'une collaboration large échelle
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
### Innovation: montage et gestion ouverte de projet
Un succès indubitable, dans un contexte particulier
- projet d'infrastructure de recherche
- des besoins à résoudre et non résoudre des problèmes
Généralisation?
- À essayer dans d'autres contextes!
- À encourager?
Critères d'évaluation des projets?
- Un objet de recherche en science ouverte?
Ref: [On OpenDreamKit's open and collaborative proposal writing](http://opendreamkit.org/2015/01/31/open-proposal-writing/)
----
## Infrastructure numérique pour la recherche
### Ressources matérielles
Mésocentres, super-calculateurs, grilles, cloud, ...
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
### Services numériques
Dont environnement virtuels (JupyterHub, Candyce, ...)
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
### Logiciels métiers
La couche la plus précieuse (opinion)
<p style="margin-bottom:15%"></p>
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
- **Préserver**:
- Software Heritage, ...
- **Soutenir**:
- Reconnaissance de l'implication des utilisateurs-dévelopeurs
- Ingénieurs de Recherche
- **Financer**
- La maintenance des communs et pas que le neuf et disruptif
<!-- +++ {"slideshow": {"slide_type": "fragment"}} !-->
<p style="margin-bottom:15%"></p>
#### Difficultés
- Limites du financement par projet
- alternance d'orages et de traversées du désert
- tension avec les carrières (notamment IR!)
- épuisement des porteurs
- Verrouillage par des lobbies
- plus d'ouverture dans le montage des projets et des appels à projets?
</script></section>
Retour d'expérience: OpenDreamKit et Science Ouverte2021-06-01T00:00:00+00:00http://opendreamkit.org/2021/06/01/odk-and-open-science<section data-markdown="" data-separator="^---\n" data-separator-vertical="^----\n">
<script type="text/template">
## La Science Ouverte dans un Projet Européen?
1. OpenDreamKit?
2. Données Ouvertes
3. Publications Ouvertes
4. Logiciels Libres
5. Montage et gestion ouvert!
---
## OpenDreamKit (2015-2019) #676541
<center>
Open Digital Research Environment Toolkit<br>
for the Advancement of Mathematics<br>
[OpenDreamKit.org](OpenDreamKit.org)
</center>
**Objectif**: soutenir l'écosystème des logiciels libres pour les **maths pure et applications**
- Ingénierie logicielle: portabilité, interoperabilité, distribution, HPC, ...
- Environnements interactifs basés sur Jupyter
- Animation de la communauté
**Consortium**: [15 sites, 50 participants](http://opendreamkit.org/partners), ... together with the international community!
**Programme**: [Horizon 2020](https://ec.europa.eu/programmes/horizon2020/), [European Research Infrastructures](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures)
**Appel à projet**: Environnements Virtuels de Recherche
**Budget**: 7.6 M€
---
## OpenDreamKit et Données Ouvertes
### Données en mathématiques?
- Résultats de calculs (parfois très lourds)
- Résultats de recherche:
Atlas des groupes simples: plusieurs décennies de travail par toute
une communauté de chercheurs
- Données très riches, très structurées, très diverses
- Frontière donnée / calcul floue
<br>
### «Facile»
- Pas de problèmes éthiques
- Peu de données gigantesques
- Peu de problèmes de reproductibilité
- Tradition de partage dans la communauté
- Gestion de la propriété Intellectuelle
- Plan de Gestion des Données (Data Management Plan): Simple liste de jeux de données
<br>
### Difficile
- FAIR en pratique:
Findable, Accessible, Interoperable, Reusable
- Formation
---
## OpenDreamKit et Publications Ouvertes
Longue tradition de publications libres en maths / info:
- Archives ouvertes: arxiv.org, ... (Green Open Access)
- Transition en cours de nombreuses revues (Diamond Open Access)
Question:
- Publication dans des revues en Gold Open Access? (Finalement non)
---
## OpenDreamKit et Logiciel Libre
- ~~Réaliser un produit?~~
- Contribuer à un écosystème
Bénéfices:
- Construire sur les épaules des géants
- Travail collaboratif avec la communauté
- Pérennité
À noter:
- Existence des modèles économiques éprouvés
- Collaboration avec des PME: Logilab, QuantStack, CoCalç, ...
- Groupe Logiciel Libre du Pôle de Compétitivité Systematic
---
## Montage et gestion de projet ouverte
<center><img src="/public/images/haut2.jpg" alt="just a nice picture" width="60%"/></center>
Note:
- Depuis 25 ans, je suis engagé dans le développement de logiciels de
calcul pour les maths -- parce que j'en ai besoin -- et dans la
promotion de la mutualisation, du partage, du logiciel libre. Bref,
de la science ouverte.
Aussi longtemps que j'ai pu, mon cœur de stratégie c'était comment
fonctionner sans financement spécifique. Pour préserver mon temps
pour le développement et l'animation de communauté. Parce que je
pense que le financement par projet n'est souvent pas un bon système
de financement de la recherche. Et pour préserver ma liberté.
Chercher des sous, c'est risquer de vendre son âme.
----
## L’étincelle initiale
<p style="margin-bottom:15%"></p>
**Une question de Bruce Westbury at [FPSAC 2013](http://fpsac.org) in Paris:**
<center>
*Given unlimited funding, what would you do with it for Sage?*
</center>
----
## Le montage du projet: approche
### Suivre mes rêves
- Logiciel libre, données ouvertes, publications ouvertes, **projet ouvert**
- Construction du bas vers le haut:
- Analyser les besoins de la communauté:<br>
quelles tâches requièrent un gros investissement?
- Chercher un appel à projet adéquat (Cellule Europe!!!)
- Trouver une histoire sincère qui les lie tous<br>
«a smart and creative interpretation of the call»
----
### Mobiliser la communauté et favoriser l'intelligence collective
<p style="margin-bottom:2ex;"></p>
#### Projet ouvert
- Invitation large à la participation à la définition du projet
- Écriture publique du projet
<p style="margin-bottom:2ex;"></p>
#### Développer une **vision partagée** du project
- Qui sommes nous?
- Quels sont nos objectifs?
- De quoi partons nous?
- Quelle est notre stratégie?
- Comment notre projet se positionne dans le paysage?
- Pour quoi sommes nous l'équipe rêvée pour la tâche?
----
### Impliquer les participants
#### Encourager
- À définir ensemble la vision du project
- À agir selon leurs convictions: c'est eux les experts!
<p style="margin-bottom:2ex;"></p>
#### Faciliter
- À quel stade en est la rédaction du projet?
- Que reste-t'il à faire?
- Comment peuvent-ils aider?
<p style="margin-bottom:2ex;"></p>
#### Bienveillance, exigence et joie
#### Sourire, courir devant, espérer ... et remercier
<p style="margin-bottom:2ex;"></p>
<center>[Quand cela fonctionne](https://www.youtube.com/watch?v=kM9zcfRtOqo)</center>
----
### Les outils collaboratifs
#### Échelle:
- 100 pages
- 20 coauteurs
- 3000 e-mails; 400 dans les deux derniers jours
Ne même pas penser à Word, Dropbox, ...
----
### Collaborative tools: some recommendations
#### Proposal writing
- Version control and forge: e.g. [GitHub](github.com) or [GitLab](gitlab.com)
- Automate whatever you can: tables, graphics, computation of the budget, ...
- Source files in text: for example LateX + [proposal](http://www.ctan.org/tex-archive/macros/latex/contrib/proposal) style file
<p style="margin-bottom:2ex;"></p>
#### Communication
- General discussions and progress tracking: mailing list, e.g. on sympa
- Discussions on specific topics: e.g. GitHub issues
- Videoconferences: jitsi, BBB, ...
- Chat: gitter / slack / mattermost
- Live text editing and note taking: e.g. [hackmd.io](hackmd.io) / codimd
<p style="margin-bottom:2ex;"></p>
<center>Train, train, train your team; learn, learn, learn</center>
---
## Montage et gestion ouverte du projet: une clé du succès
- Éthique et qualité
- Esprit d'équipe et confiance réciproque
- Soutien de la communauté
- Fraîcheur et sincérité
**Distinction**: Étoiles de l'Europe 2020
<br>
<br>
<br>
<br>
## Science Ouverte
- Choix éthique
- Choix pragmatique:
Ingrédient essentiel des collaborations large échelle
----
## More reading
- [On OpenDreamKit's open and collaborative proposal writing](http://opendreamkit.org/2015/01/31/open-proposal-writing/)
- The [developer's perspective](/about-developers) on OpenDreamKit
- [About OpenDreamKit](/about)
</script>
</section>
Retour d'expérience: OpenDreamKit2021-03-11T00:00:00+00:00http://opendreamkit.org/2021/03/11/odk-feedback-from-experience<section data-markdown="" data-separator="^---\n" data-separator-vertical="^----\n">
<script type="text/template">
## Plan
1. OpenDreamKit?
1. Montage et gestion ouverte
1. Lancer, ou ne pas lancer, un projet Européen?
1. Mieux soutenir les chercheurs
---
## OpenDreamKit (2015-2019) #676541
<center>
Open Digital Research Environment Toolkit<br>
for the Advancement of Mathematics<br>
[OpenDreamKit.org](OpenDreamKit.org)
</center>
**Objectif**: soutenir l'écosystème des logiciels libres pour les **maths pure et applications**
- Ingénierie logicielle: portabilité, interoperabilité, distribution, HPC, ...
- Environnements interactifs basés sur Jupyter
- Animation de la communauté
**Consortium**: [15 sites, 50 participants](http://opendreamkit.org/partners), ... together with the international community!
**Programme**: [Horizon 2020](https://ec.europa.eu/programmes/horizon2020/), [European Research Infrastructures](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures)
**Appel à projet**: Environnements Virtuels de Recherche
**Budget**: 7.6 M€
---
## Le montage du projet
<center><img src="/public/images/haut2.jpg" alt="just a nice picture" width="60%"/></center>
Note:
- Depuis 25 ans, je suis engagé dans le développement de logiciels de
calcul pour les maths -- parce que j'en ai besoin -- et dans la
promotion de la mutualisation, du partage, du logiciel libre. Bref,
de la science ouverte.
Aussi longtemps que j'ai pu, mon cœur de stratégie c'était comment
fonctionner sans financement spécifique. Pour préserver mon temps
pour le développement et l'animation de communauté. Parce que je
pense que ce n'est pas un bon système de financement de la
recherche. Et pour préserver ma liberté.
Chercher des sous, c'est risquer de vendre son âme.
----
## L’étincelle initiale
<p style="margin-bottom:15%"></p>
**Une question de Bruce Westbury at [FPSAC 2013](http://fpsac.org) in Paris:**
<center>
*Given unlimited funding, what would you do with it for Sage?*
</center>
----
## Le montage du projet: approche
### Suivre mes rêves
- Logiciel libre, données ouvertes, publications ouvertes, **projet ouvert**
- Construction du bas vers le haut:
- Analyser les besoins de la communauté:<br>
quelles tâches requièrent un gros investissement?
- Chercher un appel à projet adéquat (Cellule Europe!!!)
- Trouver une histoire sincère qui les lie tous<br>
«a smart and creative interpretation of the call»
----
### Mobiliser la communauté et favoriser l'intelligence collective
<p style="margin-bottom:2ex;"></p>
#### Projet ouvert
- Invitation large à la participation à la définition du projet
- Écriture publique du projet
<p style="margin-bottom:2ex;"></p>
#### Développer une **vision partagée** du project
- Qui sommes nous?
- Quels sont nos objectifs?
- De quoi partons nous?
- Quelle est notre stratégie?
- Comment notre projet se positionne dans le paysage?
- Pour quoi sommes nous l'équipe rêvée pour la tâche?
----
### Impliquer les participants
#### Encourager
- à définir ensemble la vision du project
- à agir selon leurs convictions: c'est eux les experts!
<p style="margin-bottom:2ex;"></p>
#### Faciliter
- À quel stade en est la rédaction du projet?
- Que reste-t'il à faire?
- Comment peuvent-ils aider?
<p style="margin-bottom:2ex;"></p>
#### Bienveillance, exigence et joie
#### Sourire, courir devant, espérer ... et remercier
<p style="margin-bottom:2ex;"></p>
<center>[Quand cela fonctionne](https://www.youtube.com/watch?v=kM9zcfRtOqo)</center>
----
### Les outils collaboratifs
#### Échelle:
- 100 pages
- 20 coauteurs
- 3000 e-mails; 400 dans les deux derniers jours
Ne même pas penser à Word, Dropbox, ...
----
### Collaborative tools: some recommendations
#### Proposal writing
- Version control and forge: e.g. [GitHub](github.com) or [GitLab](gitlab.com)
- Automate whatever you can: tables, graphics, computation of the budget, ...
- Source files in text: for example LateX + [proposal](http://www.ctan.org/tex-archive/macros/latex/contrib/proposal) style file
<p style="margin-bottom:2ex;"></p>
#### Communication
- General discussions and progress tracking: mailing list, e.g. on sympa
- Discussions on specific topics: e.g. GitHub issues
- Videoconferences: jitsi, BBB, ...
- Chat: gitter / slack / mattermost
- Live text editing and note taking: e.g. [hackmd.io](hackmd.io) / codimd
<p style="margin-bottom:2ex;"></p>
<center>Train, train, train your team; learn, learn, learn</center>
----
### Obtenir de l'aide
Écoutez attentivement tous les conseils ...
Et faites à votre sauce! C'est **votre** projet.
#### Déléguer à une compagnie externe?
- Coût? Liberté? Contrôle?
- Pour nous: de bons outils collaboratifs ont fait l'affaire
<p style="margin-bottom:2ex;"></p>
<p style="margin-bottom:2ex;"></p>
#### Quelques bons conseillers!
- Votre service Europe
- Des collègues ayant de l'expérience
<p style="margin-bottom:2ex;"></p>
<p style="margin-bottom:2ex;"></p>
#### Un administrateur (par ex. à temps partiel)
---
## Montage et gestion ouverte du projet: une clé du succès
- Éthique et qualité
- Esprit d'équipe et confiance réciproque
- Soutien de la communauté
- Fraîcheur et sincérité
**Distinction**: Étoiles de l'Europe 2020
---
## Lancer ou ne pas lancer un projet européen?
### Coût
#### Préparation
- Trois mois à temps plein pour le coordinateur
- Quelques mois réparti entre les autres participants
- 3000 € (déplacements + une rencontre)
#### Lancement
- Deux mois à temps plein pour le coordinateur
- 1000 € (site web, déplacements, ...)
#### Coordination
- Un bon quart de mon temps: administration et coordination
- Plus l'implication dans le projet lui-même: ingénierie, recherche, animation
- 15k mails, Coauteur de 21 rapports, éditeur de 40 rapports.
----
### Bénéfices
<p style="margin-bottom:2ex;"></p>
- Des moyens pour ce qui vous tient à cœur: financement et personnel
<p style="margin-bottom:2ex;"></p>
- Opportunité d'étendre ses compétences, son réseau, sa vision
<p style="margin-bottom:2ex;"></p>
- Une expérience humaine exceptionnelle
<p style="margin-bottom:10%"></p>
- Incidemment: pas trop mauvais pour votre carrière
Mais: on vous demandera de prendre plus de responsabilités
----
## Lancer ou ne pas lancer un projet Européen?
- Quels sont les besoins de votre communauté?
- Est-ce que des financements peuvent vraiment y faire quelque chose?
<p style="margin-bottom:2ex;"></p>
- Avez-vous un support administratif solide?
- Avez-vous le temps? L'énergie?
- Avez-vous de l'expérience pour animer une équipe?
<p style="margin-bottom:2ex;"></p>
- Avez-vous un rêve?<br>Qui vous tient suffisamment à cœur pour y
consacrer quelques années de votre vie? de votre recherche?
- Êtes vous prêt pour une aventure?
---
## Bonne chance!
### Et si vous êtes heureux lauréat, n'oubliez pas:
- Vous avez eu beaucoup de chance
- **Les citoyens** vous confient des moyens **pour votre communauté**
- Vous n'en êtes pas propriétaire; simplement responsable du meilleur usage
<center><img src="/public/logos/odk-elected-logo.svg" width="10%" alt="OpenDreamKit"/></center>
---
## Mieux soutenir les chercheurs?
### Les limites des financements par projet (européens)
- Tension entre financement par projet et carrières des personnels
- Surcoût financier et humain de la gestion de projet
- Granularité: on a pas tous les jours besoin de viser la lune
- Aléatoire (des appels à projets, des réussites)
<p style="margin-bottom:2ex;"></p>
### Vraie solution: plus de financements récurrents!
---
## Mesures de mitigation
~~Soutien aux carrières~~ Faire gagner du temps aux chercheurs!!!
### Cellules Europe solides
- Conseil et aide au montage et à la gestion
- Mise à disposition d'ingénieurs projets contre finance
- Soutien financier léger au montage (bonus)
<p style="margin-bottom:2ex;"></p>
- À disposition de tous les chercheurs (sur site!)
- Interlocuteur unique / privilégié
- Personnel stable
- Vrais systèmes d'information (gestion financière, mission, ...)
<p style="margin-bottom:2ex;"></p>
### Outils collaboratifs institutionnels (type Renater, ...), Mentors
---
### Accompagnement au lobbying?
- Financement de la maintenance des logiciels scientifiques libres<br>
(the road and bridges of modern science)
- Projets atypiques / hors poids lourds
- Écosystème Jupyter pour le calcul interactif et la science ouverte
- Écosystème des logiciels de maths
----
## More reading
- [On OpenDreamKit's open and collaborative proposal writing](http://opendreamkit.org/2015/01/31/open-proposal-writing/)
- The [developer's perspective](/about-developers) on OpenDreamKit
- [About OpenDreamKit](/about)
</script>
</section>
Trophy "Les étoiles de l'Europe"2020-12-16T00:00:00+00:00http://opendreamkit.org/2020/12/16/etoiles-de-l-europe<p>OpenDreamKit is one of the twelve H2020 projects that will be awarded
the French trophy “Les étoiles de l’Europe” in 2020 for their success
and impact. There are four categories; guess which one we were elected
in? That’s right, Open Science :-)</p>
<p>The celebration will take place <a href="https://www.horizon2020.gouv.fr/cid155906/les-etoiles-de-l-europe-2020-en-direct.html">online on December 16th of 2020, at 3pm</a>.</p>
<p>We dedicate this trophy to all those who contribute day after day to
the ecosystem of free (as in libre) software powering Open Science.
OpenDreamKit was but a mean to irrigate this ecosystem with resources
it dearly needs: funding of course; even more talented man power; and
in particular <strong><a href="https://researchsoftware.org/">Research Software Engineers</a></strong>.
It is our hope that this institutional recognition will help attract
more resources in the future.</p>
<p>And by this, we don’t just mean more projects. Software is the modern
infrastructure – the road and bridges – on which science thrills.
Occasionally one needs to build one more high bridge; project-based
funding works well for this. But road and bridges also critically need
regular maintenance and development. This too takes brilliant and
dedicated people. We need to attract them and keep them. And they
deserve <strong>true career paths</strong>.</p>
OpenDreamKit: a debriefing2020-02-26T00:00:00+00:00http://opendreamkit.org/2020/02/26/sage-day-109-odk-debriefing<section data-markdown="" data-separator="^---\n" data-separator-vertical="^----\n">
<script type="text/template">
## One of many threads
GAP
Situation:
Open Source
Developped by users for users
MuPAD-Combinat
2008 Sage-Combinat
Categories
Provocative
William's struggling with funding
Funding for research dev, community building: roughly ok
Fund research software engineers to work on long term tasks?
# Building the project
# Some concrete contributions to Sage
- Jupyter: migration & widgets
- Packaging (Docker, Debian, Conda, ...), portability (Windows)
- Build system / continuous integration
- Modularity (Sage packages)
- GAP integration
- Python 3
- Research prototypes: using semantic for high level
A collaborative spirit across projects: GAP, ..., Oscar
# Demo
## Plan
1. OpenDreamKit?
1. Montage et gestion ouverte
1. Lancer, ou ne pas lancer, un projet Européen?
1. Mieux soutenir les chercheurs
Note:
This is a talk I delivered at a meeting organized by platform Wiskunde
Netherlands, NWO, and Yellow Research to encourage Dutch
mathematicians to apply for EU funding.
---
## OpenDreamKit (2015-2019) #676541
<center>
Open Digital Research Environment Toolkit<br>
for the Advancement of Mathematics<br>
[OpenDreamKit.org](OpenDreamKit.org)
</center>
**Objectif**: soutenir l'écosystème des logiciels libres pour les **maths pure et applications**
- Ingénierie logicielle: portabilité, interoperabilité, distribution, HPC, ...
- Environnements interactifs basés sur Jupyter
- Animation de la communauté
**Consortium**: [15 sites, 50 participants](http://opendreamkit.org/partners), ... together with the international community!
**Programme**: [Horizon 2020](https://ec.europa.eu/programmes/horizon2020/), [European Research Infrastructures](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures)
**Appel à projet**: Environnements Virtuels de Recherche
**Budget**: 7.6 M€
---
## Le montage du projet
<center><img src="/public/images/haut2.jpg" alt="just a nice picture" width="60%"/></center>
Note:
- Depuis 25 ans, je suis engagé dans le développement de logiciels de
calcul pour les maths -- parce que j'en ai besoin -- et dans la
promotion de la mutualisation, du partage, du logiciel libre. Bref,
de la science ouverte.
Aussi longtemps que j'ai pu mon cœur de stratégie c'était comment
fonctionner sans financement spécifique. Pour préserver mon temps
pour le développement et l'animation de communauté. Parce que je
pense que ce n'est pas un bon système de financement de la
recherche. Et pour préserver ma liberté.
Chercher des sous, c'est risquer de vendre son âme.
----
## L’étincelle initiale
<p style="margin-bottom:15%"></p>
**Une question de Bruce Westbury at [FPSAC 2013](http://fpsac.org) in Paris:**
<center>
*Given unlimited funding, what would you do with it for Sage?*
</center>
----
## Le montage du projet: approche
### Suivre mes rêves
- Logiciel libre, données ouvertes, publications ouvertes, **projet ouvert**
- Construction du bas vers le haut:
- Analyser les besoins de la communauté:<br>
quelles tâches requièrent un gros investissement?
- Chercher un appel à projet adéquat
- Trouver une histoire sincère qui les lie tous<br>
«a smart and creative interpretation of the call»
----
### Mobiliser la communauté et favoriser l'intelligence collective
<p style="margin-bottom:2ex;"></p>
#### Projet ouvert
- Invitation large à la participation à la définition du projet
- Écriture publique du projet
<p style="margin-bottom:2ex;"></p>
#### Développer une **vision partagée** du project
- Qui sommes nous?
- Quels sont nos objectifs?
- De quoi partons nous?
- Quelle est notre stratégie?
- Comment notre projet se positionne dans le paysage?
- Pour quoi sommes nous l'équipe rêvée pour la tâche?
----
### Impliquer les participants
#### Encourager
- à définir ensemble la vision du project
- à agir selon leurs convictions: c'est eux les experts!
<p style="margin-bottom:2ex;"></p>
#### Faciliter
- À quel stade en est la rédaction du projet?
- Que reste-t'il à faire?
- Comment peuvent-ils aider?
<p style="margin-bottom:2ex;"></p>
#### Bienveillance, exigence et joie
#### Sourire, courir devant, espérer ... et remercier
<p style="margin-bottom:2ex;"></p>
<center>[Quand cela fonctionne](https://www.youtube.com/watch?v=kM9zcfRtOqo)</center>
----
### Les outils collaboratifs
#### Échelle:
- 100 pages
- 20 coauteurs
- 3000 e-mails; 400 dans les deux derniers jours
Ne même pas penser à Word, Dropbox, ...
----
### Collaborative tools: some recommendations
#### Proposal writing
- Version control and forge: e.g. [GitHub](github.com) or [GitLab](gitlab.com)
- Automate whatever you can: tables, graphics, computation of the budget, ...
- Source files in text: for example LateX + [proposal](http://www.ctan.org/tex-archive/macros/latex/contrib/proposal) style file
<p style="margin-bottom:2ex;"></p>
#### Communication
- General discussions and progress tracking: mailing list, e.g. on sympa
- Discussions on specific topics: e.g. GitHub issues
- Videoconferences: [appear.in](appear.in) / [hubl.in](hubl.in) / [framatalk.org](framatalk.org)
- Chat: gitter / slack / ...
- Live text editing and note taking: e.g. [hackmd.io](hackmd.io)
<p style="margin-bottom:2ex;"></p>
<center>Train, train, train your team; learn, learn, learn</center>
----
### Obtenir de l'aide
Écoutez attentivement tous les conseils ...
Et faites à votre sauce! C'est **votre** projet.
#### Déléguer à une compagnie externe?
- Coût? Liberté? Contrôle?
- Pour nous: de bons outils collaboratifs ont fait l'affaire
<p style="margin-bottom:2ex;"></p>
<p style="margin-bottom:2ex;"></p>
#### Quelques bons conseillers!
- Votre service Europe
- Des collègues ayant de l'expérience
<p style="margin-bottom:2ex;"></p>
<p style="margin-bottom:2ex;"></p>
#### Un administrateur (par ex. à temps partiel)
---
## Montage et gestion ouverte du projet: une clé du succès
- Éthique
- Qualité
- Fraîcheur et sincérité
- Esprit d'équipe et confiance réciproque
- Soutien de la communauté
---
## Lancer ou ne pas lancer un projet européen?
### Coût
#### Préparation
- Trois mois à temps plein pour le coordinateur
- Quelques mois réparti entre les autres participants
- 3000 € (déplacements + une rencontre)
#### Lancement
- Deux mois à temps plein pour le coordinateur
- 1000 € (site web, déplacements, ...)
#### Coordination
- Un bon quart de mon temps: administration et coordination
- Plus l'implication dans le projet lui-même: ingénierie, recherche, animation
- 15k mails, Coauteur de 21 rapports, éditeur de 40 rapports.
----
### Bénéfices
<p style="margin-bottom:2ex;"></p>
- Des moyens pour ce qui vous tient à cœur: financement et personnel
<p style="margin-bottom:2ex;"></p>
- Opportunité d'étendre ses compétences, son réseau, sa vision
<p style="margin-bottom:2ex;"></p>
- Une expérience humaine exceptionnelle
<p style="margin-bottom:10%"></p>
- Incidemment: pas trop mauvais pour votre carrière
Mais: on vous demandera de prendre plus de responsabilités
----
## Lancer ou ne pas lancer un projet Européen?
- Quels sont les besoins de votre communauté?
- Est-ce que des financements peuvent vraiment y faire quelque chose?
<p style="margin-bottom:2ex;"></p>
- Avez-vous un support administratif solide?
- Avez-vous le temps? L'énergie?
- Avez-vous de l'expérience pour animer une équipe?
<p style="margin-bottom:2ex;"></p>
- Avez-vous un rêve?<br>Qui vous tient suffisamment à cœur pour y
consacrer quelques années de votre vie? de votre recherche?
- Êtes vous prêt pour une aventure?
---
## Bonne chance!
### Et si vous êtes heureux lauréat, n'oubliez pas:
- Vous avez eu beaucoup de chance
- **Les citoyens** vous confient des moyens **pour votre communauté**
- Vous n'en êtes pas propriétaire; simplement responsable du meilleur usage
<center><img src="/public/logos/odk-elected-logo.svg" width="10%" alt="OpenDreamKit"/></center>
---
## Mieux soutenir les chercheurs?
### Les limitations des financements par projet européens
- Tension entre financement par projet et carrières des personnels
- Surcoût financier et humain de la gestion de projet
- Granularité: on a pas tous les jours besoin de viser la lune
- Aléatoire (des appels à projets, des réussites)
<p style="margin-bottom:2ex;"></p>
### Vraie solution: des financements récurrents!
---
## Mesures de mitigation
~~Soutien aux carrières~~ Faire gagner du temps aux chercheurs!!!
### Cellules Europe solides
- Conseil et aide au montage et à la gestion
- Mise à disposition d'ingénieurs projets contre finance
- Soutien financier léger au montage (bonus)
<p style="margin-bottom:2ex;"></p>
- À disposition de tous les chercheurs (sur site!)
- Interlocuteur unique / privilégié
- Personnel stable
- Vrais systèmes d'information (gestion financière, mission, ...)
<p style="margin-bottom:2ex;"></p>
### Outils collaboratifs institutionnels (type Renater, ...), Mentors
---
### Accompagnement au lobbying?
- Financement de la maintenance des logiciels scientifiques libres<br>
(the road and bridges of modern science)
- Projets atypiques / hors poids lourds
- Écosystème Jupyter pour le calcul interactif et la science ouverte
- Écosystème des logiciels de maths
----
## More reading
- [On OpenDreamKit's open and collaborative proposal writing](http://opendreamkit.org/2015/01/31/open-proposal-writing/)
- The [developer's perspective](/about-developers) on OpenDreamKit
- [About OpenDreamKit](/about)
</script>
</section>
Retour d'expérience: OpenDreamKit2020-02-26T00:00:00+00:00http://opendreamkit.org/2020/02/26/odk-feedback-from-experience<section data-markdown="" data-separator="^---\n" data-separator-vertical="^----\n">
<script type="text/template">
## Plan
1. OpenDreamKit?
1. Montage et gestion ouverte
1. Lancer, ou ne pas lancer, un projet Européen?
1. Mieux soutenir les chercheurs
Note:
This is a talk I delivered at a meeting organized by platform Wiskunde
Netherlands, NWO, and Yellow Research to encourage Dutch
mathematicians to apply for EU funding.
---
## OpenDreamKit (2015-2019) #676541
<center>
Open Digital Research Environment Toolkit<br>
for the Advancement of Mathematics<br>
[OpenDreamKit.org](OpenDreamKit.org)
</center>
**Objectif**: soutenir l'écosystème des logiciels libres pour les **maths pure et applications**
- Ingénierie logicielle: portabilité, interoperabilité, distribution, HPC, ...
- Environnements interactifs basés sur Jupyter
- Animation de la communauté
**Consortium**: [15 sites, 50 participants](http://opendreamkit.org/partners), ... together with the international community!
**Programme**: [Horizon 2020](https://ec.europa.eu/programmes/horizon2020/), [European Research Infrastructures](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures)
**Appel à projet**: Environnements Virtuels de Recherche
**Budget**: 7.6 M€
---
## Le montage du projet
<center><img src="/public/images/haut2.jpg" alt="just a nice picture" width="60%"/></center>
Note:
- Depuis 25 ans, je suis engagé dans le développement de logiciels de
calcul pour les maths -- parce que j'en ai besoin -- et dans la
promotion de la mutualisation, du partage, du logiciel libre. Bref,
de la science ouverte.
Aussi longtemps que j'ai pu mon cœur de stratégie c'était comment
fonctionner sans financement spécifique. Pour préserver mon temps
pour le développement et l'animation de communauté. Parce que je
pense que ce n'est pas un bon système de financement de la
recherche. Et pour préserver ma liberté.
Chercher des sous, c'est risquer de vendre son âme.
----
## L’étincelle initiale
<p style="margin-bottom:15%"></p>
**Une question de Bruce Westbury at [FPSAC 2013](http://fpsac.org) in Paris:**
<center>
*Given unlimited funding, what would you do with it for Sage?*
</center>
----
## Le montage du projet: approche
### Suivre mes rêves
- Logiciel libre, données ouvertes, publications ouvertes, **projet ouvert**
- Construction du bas vers le haut:
- Analyser les besoins de la communauté:<br>
quelles tâches requièrent un gros investissement?
- Chercher un appel à projet adéquat
- Trouver une histoire sincère qui les lie tous<br>
«a smart and creative interpretation of the call»
----
### Mobiliser la communauté et favoriser l'intelligence collective
<p style="margin-bottom:2ex;"></p>
#### Projet ouvert
- Invitation large à la participation à la définition du projet
- Écriture publique du projet
<p style="margin-bottom:2ex;"></p>
#### Développer une **vision partagée** du project
- Qui sommes nous?
- Quels sont nos objectifs?
- De quoi partons nous?
- Quelle est notre stratégie?
- Comment notre projet se positionne dans le paysage?
- Pour quoi sommes nous l'équipe rêvée pour la tâche?
----
### Impliquer les participants
#### Encourager
- à définir ensemble la vision du project
- à agir selon leurs convictions: c'est eux les experts!
<p style="margin-bottom:2ex;"></p>
#### Faciliter
- À quel stade en est la rédaction du projet?
- Que reste-t'il à faire?
- Comment peuvent-ils aider?
<p style="margin-bottom:2ex;"></p>
#### Bienveillance, exigence et joie
#### Sourire, courir devant, espérer ... et remercier
<p style="margin-bottom:2ex;"></p>
<center>[Quand cela fonctionne](https://www.youtube.com/watch?v=kM9zcfRtOqo)</center>
----
### Les outils collaboratifs
#### Échelle:
- 100 pages
- 20 coauteurs
- 3000 e-mails; 400 dans les deux derniers jours
Ne même pas penser à Word, Dropbox, ...
----
### Collaborative tools: some recommendations
#### Proposal writing
- Version control and forge: e.g. [GitHub](github.com) or [GitLab](gitlab.com)
- Automate whatever you can: tables, graphics, computation of the budget, ...
- Source files in text: for example LateX + [proposal](http://www.ctan.org/tex-archive/macros/latex/contrib/proposal) style file
<p style="margin-bottom:2ex;"></p>
#### Communication
- General discussions and progress tracking: mailing list, e.g. on sympa
- Discussions on specific topics: e.g. GitHub issues
- Videoconferences: [appear.in](appear.in) / [hubl.in](hubl.in) / [framatalk.org](framatalk.org)
- Chat: gitter / slack / ...
- Live text editing and note taking: e.g. [hackmd.io](hackmd.io)
<p style="margin-bottom:2ex;"></p>
<center>Train, train, train your team; learn, learn, learn</center>
----
### Obtenir de l'aide
Écoutez attentivement tous les conseils ...
Et faites à votre sauce! C'est **votre** projet.
#### Déléguer à une compagnie externe?
- Coût? Liberté? Contrôle?
- Pour nous: de bons outils collaboratifs ont fait l'affaire
<p style="margin-bottom:2ex;"></p>
<p style="margin-bottom:2ex;"></p>
#### Quelques bons conseillers!
- Votre service Europe
- Des collègues ayant de l'expérience
<p style="margin-bottom:2ex;"></p>
<p style="margin-bottom:2ex;"></p>
#### Un administrateur (par ex. à temps partiel)
---
## Montage et gestion ouverte du projet: une clé du succès
- Éthique
- Qualité
- Fraîcheur et sincérité
- Esprit d'équipe et confiance réciproque
- Soutien de la communauté
---
## Lancer ou ne pas lancer un projet européen?
### Coût
#### Préparation
- Trois mois à temps plein pour le coordinateur
- Quelques mois réparti entre les autres participants
- 3000 € (déplacements + une rencontre)
#### Lancement
- Deux mois à temps plein pour le coordinateur
- 1000 € (site web, déplacements, ...)
#### Coordination
- Un bon quart de mon temps: administration et coordination
- Plus l'implication dans le projet lui-même: ingénierie, recherche, animation
- 15k mails, Coauteur de 21 rapports, éditeur de 40 rapports.
----
### Bénéfices
<p style="margin-bottom:2ex;"></p>
- Des moyens pour ce qui vous tient à cœur: financement et personnel
<p style="margin-bottom:2ex;"></p>
- Opportunité d'étendre ses compétences, son réseau, sa vision
<p style="margin-bottom:2ex;"></p>
- Une expérience humaine exceptionnelle
<p style="margin-bottom:10%"></p>
- Incidemment: pas trop mauvais pour votre carrière
Mais: on vous demandera de prendre plus de responsabilités
----
## Lancer ou ne pas lancer un projet Européen?
- Quels sont les besoins de votre communauté?
- Est-ce que des financements peuvent vraiment y faire quelque chose?
<p style="margin-bottom:2ex;"></p>
- Avez-vous un support administratif solide?
- Avez-vous le temps? L'énergie?
- Avez-vous de l'expérience pour animer une équipe?
<p style="margin-bottom:2ex;"></p>
- Avez-vous un rêve?<br>Qui vous tient suffisamment à cœur pour y
consacrer quelques années de votre vie? de votre recherche?
- Êtes vous prêt pour une aventure?
---
## Bonne chance!
### Et si vous êtes heureux lauréat, n'oubliez pas:
- Vous avez eu beaucoup de chance
- **Les citoyens** vous confient des moyens **pour votre communauté**
- Vous n'en êtes pas propriétaire; simplement responsable du meilleur usage
<center><img src="/public/logos/odk-elected-logo.svg" width="10%" alt="OpenDreamKit"/></center>
---
## Mieux soutenir les chercheurs?
### Les limitations des financements par projet européens
- Tension entre financement par projet et carrières des personnels
- Surcoût financier et humain de la gestion de projet
- Granularité: on a pas tous les jours besoin de viser la lune
- Aléatoire (des appels à projets, des réussites)
<p style="margin-bottom:2ex;"></p>
### Vraie solution: des financements récurrents!
---
## Mesures de mitigation
~~Soutien aux carrières~~ Faire gagner du temps aux chercheurs!!!
### Cellules Europe solides
- Conseil et aide au montage et à la gestion
- Mise à disposition d'ingénieurs projets contre finance
- Soutien financier léger au montage (bonus)
<p style="margin-bottom:2ex;"></p>
- À disposition de tous les chercheurs (sur site!)
- Interlocuteur unique / privilégié
- Personnel stable
- Vrais systèmes d'information (gestion financière, mission, ...)
<p style="margin-bottom:2ex;"></p>
### Outils collaboratifs institutionnels (type Renater, ...), Mentors
---
### Accompagnement au lobbying?
- Financement de la maintenance des logiciels scientifiques libres<br>
(the road and bridges of modern science)
- Projets atypiques / hors poids lourds
- Écosystème Jupyter pour le calcul interactif et la science ouverte
- Écosystème des logiciels de maths
----
## More reading
- [On OpenDreamKit's open and collaborative proposal writing](http://opendreamkit.org/2015/01/31/open-proposal-writing/)
- The [developer's perspective](/about-developers) on OpenDreamKit
- [About OpenDreamKit](/about)
</script>
</section>
A rewarding Final Project Review2019-10-30T00:00:00+00:00http://opendreamkit.org/2019/10/30/final-project-review<p>On October 30th of 2019, OpenDreamKit underwent its final formal
review by the European Commission, represented by three reviewers and
our finance and project officers. About twenty of us went to
Luxembourg for
<a href="/meetings/2019-10-30-Luxembourg/">two days of rehearsals</a>
before an
<a href="/meetings/2019-10-30-Luxembourg/ProjectReview/">intense day of presentation</a>
of our achievements <a href="/project/reports/#reporting-period-3">in the last reporting period</a>,
and in the project as a whole.</p>
<p>After the presentations and in-depth discussions, the feedback from
the reviewers was enthusiastic, with comments such as “[you have] done extremely
good job” or “heroic management” and strong encouragements to keep
applying and shooting for the moon. As in previous reviews, we
highlighted that OpenDreamKit’s work was but the tip of the iceberg;
all we did was to exploit the special resources the EU entrusted us to
knock down some tough hurdles that were preventing the ball to roll.</p>
<p><strong>The pride shall go to our communities!</strong></p>
<h2 id="content">Content</h2>
<p>Nicolas launched the day by showcasing some of our
<a href="/tag/use-case">use cases</a>, before
<a href="https://nbviewer.jupyter.org/github/OpenDreamKit/demo-semigroup-representation-theory/blob/master/demo.ipynb">telling a story</a>
of his successive PhD students, to illustrate how the very diverse
achievements of OpenDreamKit and our communities were coming together
to enable novel experimental research.</p>
<p>Min <a href="/meetings/2019-10-30-Luxembourg/ProjectReview/WP4.pdf">presented</a>
our contributions to Jupyter-based user interfaces and
virtual environments, reflecting along the way on the future of the
Jupyter community. The impact was illustrated by software
demonstrations by Marijan (micromagnetism), Olexandr (<a href="/meetings/2019-10-30-Luxembourg/ProjectReview/WP4_lightning_talk-Jupyter_and_GAP.pdf">GAP</a>), Olivier
(training), and Marcin (3D).</p>
<p>Clément – supported by software demonstrations of Steve (GAP), Bill
(Pari), Daniel (Singular) –
<a href="/meetings/2019-10-30-Luxembourg/ProjectReview/WP5.pdf">reviewed</a>
the achievements toward higher
performance on modern architectures, both within computational
components and when combining them. He highlighted algorithmic
advances, common trends, lessons learned and perspectives.</p>
<p>Izabela and Nicolas <a href="/meetings/2019-10-30-Luxembourg/ProjectReview/WP1.pdf">detailed</a> the project management: use of
resources, explanations of deviations, risks management, follow up to
recommendations, etc; then we received feedback from our financial
officer on how to polish our financial and technical reports.</p>
<p>Luca <a href="https://nbviewer.jupyter.org/github/OpenDreamKit/OpenDreamKit.github.io/blob/master/meetings/2019-10-30-Luxembourg/ProjectReview/WP3.ipynb">showcased</a>
the variety of Virtual Research Environments that can
be constructed from our toolkit. He revealed the massive plumbing
efforts (modularity, packaging, …) that underlined it, and their
impact on usability and sustainability.</p>
<p>Michael <a href="/meetings/2019-10-30-Luxembourg/ProjectReview/WP6.pdf">explained</a> the research that was conducted on exploiting
mathematical knowledge for interoperability between mathematical
computational systems, databases, and even proof systems; concrete
outcomes range from proof of concepts to production grade tools, notably
to support FAIR mathematical data.</p>
<p>Viviane, with the help of Erik, <a href="/meetings/2019-10-30-Luxembourg/ProjectReview/WP2">outlined</a> our massive community
building, training, and dissemination activities. 110 events touching
1800 trainees and 5000 students, and involving a majority of the
consortium. The reviewers were amazed and said so, adding that the
level of outreach went far beyond anything they imagined for this
project; they specifically congratulated Viviane for the strong
diversity actions.</p>
<p>Nicolas was left with the tough task of <a href="/meetings/2019-10-30-Luxembourg/ProjectReview/wrapup-final.pdf">wrapping up</a>
four years of hard work by 50+ participants. He highlighted the keys
to success, notably the open management and bottom-up approach, the
impact of Research Software Engineers, the needs of the involved
communities, and the ongoing efforts toward future funding. He
concluded with his deep gratitude to the consortium for this amazing
human endeavor, <strong>with a special thought to all those that entrusted
years of their career to temporary positions in this project</strong>.</p>
<h2 id="aftermath">Aftermath</h2>
<p>After the reviewers comments and departure, we all were left with an
intense feeling of having built together a community. Hence, before we
all went for a well deserved celebration dinner, we spontaneously
started to brainstorm about the future of that community, beyond
ephemera and geographic artifacts such as EU projects.</p>
<p>Stay tuned :-)</p>
<p><img src="/public/images/rp3-meeting.jpg" alt="Our photo after the review meeting" class="img-responsive" /></p>
Which architecture for which application2019-10-28T00:00:00+00:00http://opendreamkit.org/2019/10/28/use-case-which-architecture-for-which-application<h2 id="scenario">Scenario</h2>
<p>Archibald has a challenge mathematical computation to run for which he received a grant. He wonders how to spend it most efficiently in hardware and/or compute hours on a high performance computing cluster.</p>
<h2 id="suggestion-of-solution">Suggestion of solution</h2>
<p><a href="/public/images/use-cases/which_infrastructure.png"> <img src="/public/images/use-cases/which_infrastructure.png" alt="picture of the suggested solution" /></a></p>
<ol>
<li>He draws the resource consumption profile of his computation as precisely as possible:
<ul>
<li>identifies the main computational bottlenecks</li>
<li>then for each of them identifies:
<ul>
<li>the overall memory footprint (memory space required by Input and Output, together with the allocations of intermediate computations)</li>
<li>whether the computation is memory bound (about the same amount of computation as data) or compute bound (significantly more computation than data).</li>
<li>the level of parallelism between tasks</li>
<li>the regularity pattern of the computation.</li>
</ul>
</li>
</ul>
</li>
<li>
<p>Investigate ways (algorithmic, problem reformulation), if any, to make most of the computational effort rely on a regular, compute intensive, kernels.</p>
</li>
<li>Select the architecture best suited bor the main bottlenecks among the following:
<ul>
<li>High speed single core (or few-cores) machine.</li>
<li>Large scale multi-core server</li>
<li>high end GPU on a standard multi-core server</li>
<li>hours on a large scale distributed cluster of heterogeneous servers (academia)</li>
<li>hours on a large scale high end homogeneous distributed cluster
depending on the type of computations:</li>
</ul>
</li>
</ol>
<table class="table table-hover">
<thead>
<tr>
<th> </th>
<th style="text-align: center">Fast single core</th>
<th style="text-align: center">Large scale multi-core</th>
<th style="text-align: center">High-end GPU</th>
<th style="text-align: center">Heterogeneous cluster</th>
<th style="text-align: center">Uniform cluster</th>
</tr>
</thead>
<tbody>
<tr>
<td>Compute intensive</td>
<td style="text-align: center">✓</td>
<td style="text-align: center">✓</td>
<td style="text-align: center">✓</td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
</tr>
<tr>
<td>Regularity (SIMD)</td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
<td style="text-align: center">✓</td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
</tr>
<tr>
<td>Memory intensive</td>
<td style="text-align: center">✓</td>
<td style="text-align: center">✓</td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
</tr>
<tr>
<td>Large data (near 1Tb)</td>
<td style="text-align: center"> </td>
<td style="text-align: center">✓</td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
</tr>
<tr>
<td>Huge data (several Tb)</td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
<td style="text-align: center">✓</td>
<td style="text-align: center">✓</td>
</tr>
<tr>
<td>Embarassingly parallel</td>
<td style="text-align: center"> </td>
<td style="text-align: center"> </td>
<td style="text-align: center">✓</td>
<td style="text-align: center">✓</td>
<td style="text-align: center">✓</td>
</tr>
</tbody>
</table>
<h2 id="design-of-high-performance-code-for-mathematical-computing">Design of high performance code for mathematical computing</h2>
<p>Before throwing any effort in code developpment, Archibald tries to make use of any specialised library providing high performance implementations for some kernels he is focusing on.</p>
<p>Overall the design of new code solving the problem should focus on optimizing the following points incrementally.</p>
<ol>
<li>optimizing the memory access patterns: cache optimization</li>
<li>optimizing the use of in-core parallelism: using SIMD instructions, arithmetic units, etc</li>
<li>shared memory parallelization:
<ul>
<li>applied from the coarsest to the finest grain</li>
<li>optimizing data placement and memory allocation (numactl, tcmalloc, etc)</li>
<li>technology choice:</li>
</ul>
<ul>
<li>pthreads: full control, often too low level w.r.t. application</li>
<li>OpenMP: good level of abstraction, now take advantage of task based dependency DAGs. Still struggles with recursive tasks and large task queues</li>
<li>Cilk, XKaapi, SMPSS, star-PU: more powerfull, but risk of discontinued support</li>
</ul>
</li>
<li>Message passing based distributed computing (MPI):
<ul>
<li>tune the serialization layer</li>
<li>use hybrid MPI-OpenMP: exploiting shared memory within nodes.</li>
</ul>
</li>
</ol>
Collaborating with Jupyter notebooks2019-10-24T00:00:00+00:00http://opendreamkit.org/2019/10/24/rtc<h2 id="introduction">Introduction</h2>
<p>Jupyter notebooks are documents that enable the recording and sharing
of computational ideas.
Often, such ideas are developed and shared by teams of people,
distributed throughout the world,
collaborating in multiple ways at multiple time scales.
Sometimes one person makes changes and proposes them to the project
via a version control system such as git,
which we can call “long-term collaboration”.
At other points during the work,
contributors may want to work together on the document at the same time,
which we call “real-time collaboration.”
Collaboration is a key part of the development and dissemination of
computational ideas,
so it is important for Jupyter notebooks to be well suited to these tasks.</p>
<h2 id="scenario">Scenario</h2>
<p>Ash and Fatima are collaborating on a project.
They have developed a model,
and have some demonstrations to run, illustrating how their model works,
and running it through some sample tasks with visualisations.
Fatima has some changes to make to the demonstration and would like Ash’s feedback.</p>
<p>For this, Ash needs to:</p>
<ul>
<li>See Fatima’s changes in context of the previous version of the document</li>
</ul>
<p>Ash reviews Fatima’s changes and would like to discuss them further,
iterating on the changes to come to a conclusion that both can agree on.</p>
<p>For this second part, Ash and Fatima need to:</p>
<ul>
<li>Both be able to execute and edit the document together</li>
</ul>
<h2 id="suggestion-of-solution">Suggestion of solution</h2>
<ol>
<li>They use a Jupyter notebook for the demonstration of the model,
to encapsulate prose, code, and outputs in one place.</li>
<li>They publish the work as a git repository,
enabling review of changes step-by-step as they are made.</li>
<li>Ash uses nbdime to review Fatima’s changes in the repository.</li>
<li>They use nbval to validate each change to notebooks via Continuous Integration service such as TravisCI.</li>
<li>Ash and Fatima use JupyterLab for real-time collaboration to work through
rapid feedback and come to a resolution.</li>
</ol>
<h2 id="discussion">Discussion</h2>
<p>By using Jupyter notebooks, the illustration of the model
can be a single container for prose explanation,
implementation in code, and visualisations of results.
Further, it can be shared as an interactive document
for others to download, execute, and modify,
either in their own environments, or online with zero installation
via services such as <a href="https://mybinder.org">Binder</a>.</p>
<h2 id="time-and-expertise-required">Time and expertise required</h2>
<p>Assuming the team is familiar with version control and Jupyter (basic lab
skills taught at <a href="http://software-carpentry.org/">Software Carpentry</a>,
and is fluent in using Jupyter notebook, the initial setup of the
collaboration could be done in a few minutes.
For live collaboration,
a notebook server to which both contributors can connect would need to exist.
<a href="https://jupyterhub.readthedocs.io">JupyterHub</a> can be used for this,
or for short-term collaboration, <a href="https://mybinder.org">mybinder.org</a> may be used with zero installation.</p>
<h2 id="whats-new-since-opendreamkit-started">What’s new since OpenDreamKit started</h2>
<ul>
<li>Expansion of the <a href="https://jupyter.org">Jupyter</a> technology;</li>
<li>Introduction of <a href="https://nbdime.readthedocs.io">nbdime</a> package for reviewing notebooks in version control;</li>
<li>Introduction of <a href="https://nbval.readthedocs.io">nbval</a> package for reviewing notebooks in version control;</li>
<li>Prototype of real-time collaboration in <a href="https://jupyterlab.readthedocs.io">JupyterLab</a></li>
</ul>
<h2 id="opendreamkit-contribution">OpenDreamKit contribution</h2>
<ul>
<li>Development and contributions to Jupyter interfaces (kernels) for
math software (GAP, Pari/GP, SageMath, Singular) and C++;
see <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/96">D4.7</a>.</li>
<li>Development of <a href="https://nbdime.readthedocs.io">nbdime</a> for reviewing notebooks in version control,
see <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/95">D4.6</a></li>
<li>Development of <a href="https://nbval.readthedocs.io">nbval</a> for testing notebooks,
see <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/98">D4.8</a></li>
<li>Prototype of real-time collaboration in JupyterLab,
see <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/89">D4.15</a></li>
</ul>
Micromagnetics with Ubermag2019-10-24T00:00:00+00:00http://opendreamkit.org/2019/10/24/MicromagneticsWithUbermag<h2 id="scenario">Scenario</h2>
<p>Tom is running micromagnetic simulations in order to explore how does
the result change when he varies one of many input paraneters. For
every value of an input parameter he has to write a separate
configuration file for OOMMF, run them all individually, and then
extract the results he is interested in for every single run. Finally,
he wants to share his entire computational workflow as a supplementary
information to his paper. In other words, Tom wants to automate the
entire process and make the entire computational workflow easily
reproducible by anyone.</p>
<h2 id="context">Context</h2>
<p>Computational Science is emerging as the third pillar of research and
development in academia and in industry across all science and
engineering disciplines. Computational studies complement experimental
and theoretical studies, and are at times the only feasible way to
address research challenges, effective industrial design and
engineering of various products and systems.</p>
<p>In the field of magnetism, micromagnetic simulations have become well
established and are often the only possible technique for the
exploration of different magnetic phenomena. Their use becomes more
widespread and reliable as the micromagnetic models, simulation
techniques, and the processing power of computers advance.</p>
<p>Computational science brings its own challenges: results based on
computer simulation should be reproducible, ideally by the whole
research community but at the very least by the authors of the
publication. This requires, amongst other things, tracking of all
input parameters for a simulation, and all post-processing steps,
often for very many simulation runs.</p>
<p>By exposing Object Oriented MicroMagnetic Framework (OOMMF)
simulations to the general purpose programming language, all the
benefits of various Python scientific libraries become readily
available. In addition, by integrating micromagnetic simulations into
Jupyter notebook, the entire computational study can be preserved as a
single document, containing the code, its output, visualisation, and
human-readable text in a single shareable document.</p>
<h2 id="suggestion-of-solution">Suggestion of solution</h2>
<p>He uses <a href="https://ubermag.github.io">Ubermag</a>, developed as a part of
OpenDreamKit, where OOMMF micromagnetic simulations are exposed to all
existing Python libraries and embedded in Jupyter. This way, in a
single document he can iterate through the entire parameter space he
is interested in, he can extract the results, analyse them, and
visualise them in a single shareable document.</p>
<p>Apart from that, the computational study he created in a single
document can be run “in the cloud” using Binder, so that anybody can
run his simulations without the need to have anything installed on the
hosting machine; see the <a href="https://opendreamkit.org/2017/11/02/use-case-publishing-reproducible-notebooks/">publishing reproducible notebooks</a> use case.</p>
<h2 id="ubermag-instances">Ubermag instances</h2>
<ul>
<li><a href="https://mybinder.org/v2/gh/ubermag/oommfc/master?filepath=docs%2Fipynb%2Findex.ipynb">Ubermag Binder</a></li>
</ul>
<h2 id="discussion">Discussion</h2>
<p>By running his micromagnetic study inside the Jupyter notebook, Tom
achieved the following:</p>
<ul>
<li>He can easily set up his simulation using the Python-based domain specific langauge.</li>
<li>He can use all the benefits of Python in running his simulations, such as running multiple simulations over large parameter spaces.</li>
<li>He can easily access the resulting data and analyse them.</li>
<li>He can visualise the data using both static and interactive plots (k3d).</li>
<li>He can easily share his entire workflow as a single Jupyter notebook.</li>
</ul>
<h2 id="whats-new-since-opendreamkit-started">What’s new since OpenDreamKit started</h2>
<p>Before, for every value of an input parameter he had to write a
separate configuration file for OOMMF, run them all individually, and
then extract the results he is interested for every single run. Later,
sharing his computational workflow was difficult and that affected the
reproducibility of his work.</p>
Authoring interactive books with Jupyter notebooks2019-10-19T00:00:00+00:00http://opendreamkit.org/2019/10/19/books-authoring<h2 id="introduction">Introduction</h2>
<p>The ease of authoring manuscripts is a key to wide application of any
new technology in educational practice. Generally speaking, academic
teachers lecturing on computer science tend to be fluent in all kinds
of computer technology and happily adopt new ones. However, as topics
get further and further from computer science as in theoretical
mechanics or dynamical systems technology, technology literacy becomes
a barrier. It can often happen that such subjects are not integrated
with computer technologies during teaching at all.</p>
<h2 id="scenario">Scenario</h2>
<p>John is lecturing theoretical mechanics and wants to integrate computer
algebra calculation in his course. He already has examples and ideas
how to use SageMath to creatively improve problem solving in
mechanics. He wants to use a single system for both his own
experiments with algebra involved in mechanics and for authoring
materials for students. Additionally, he wants students to be able to:</p>
<ul>
<li>
<p>work with examples which are published and create own
solutions of given problems and return then for grading.</p>
</li>
<li>
<p>perform simple and short interaction with a code or user interface
during a lecture or a tutorial</p>
</li>
<li>
<p>read printed version of lecture notes.</p>
</li>
</ul>
<h2 id="suggestion-of-solution">Suggestion of solution</h2>
<ol>
<li>
<p>He uses <a href="http://jupyter.org">Jupyter notebook</a> with SageMath kernel,
mixing prose, code, visualization, together with resources: source
code, data, media;</p>
</li>
<li>
<p>He publishes them on a publicly hosted repository using <a href="https://github.com/OpenDreamKit/authoring_cookie_cutter">a
template</a>. In
this example the repository is <a href="/tag/binder">Binder-ready</a>; for
details, see the <a href="https://mybinder.readthedocs.io/en/latest/using.html#preparing-a-repository-for-binder">Binder documentation</a></p>
</li>
<li>
<p>He shares the link to the repository with students.</p>
</li>
</ol>
<h2 id="some-instances">Some instances</h2>
<ul>
<li>
<p><a href="https://github.com/fangohr/introduction-to-python-for-computational-science-and-engineering">Introduction to Python for Computational Science and Engineering</a>,
Hans Fangohr
<a href="https://mybinder.org/v2/gh/fangohr/introduction-to-python-for-computational-science-and-engineering/library-current-versions?filepath=index.ipynb"><img src="https://mybinder.org/badge.svg" alt="Binder" /></a></p>
</li>
<li>
<p><a href="https://github.com/marcinofulus/Mechanics_with_SageMath">Mechanics with SageMath</a>,
Marcin Kostur, Jerzy Łuczka
<a href="https://mybinder.org/v2/gh/marcinofulus/Mechanics_with_SageMath/master?filepath=index.ipynb"><img src="https://mybinder.org/badge_logo.svg" alt="Binder" /></a></p>
</li>
</ul>
<h2 id="alternative-solutions">Alternative solutions</h2>
<p>There are many other possibilities to write interactive textbooks. This workflow can be adapted to use automatic tests that can reexecute all the material in the notebook to verify that the newly calculated outputs match up to trivial differences, with the recorded outputs. This uses the NBVAL tool – developed as part of OpenDreamKit and is implemented in Introduction to Python for Computational Science and Engineering textbook.</p>
<p>One can use more sophisticated typesetting and documentation tools. One possibility is Sphinx system, which is used to build documentation for Python packages. It is capable of better control of the format of the book. Examples are interative textbooks on <a href="http://visual.icse.us.edu.pl/LA/">Linear Algebra</a> and <a href="http://visual.icse.us.edu.pl/NPB/">Nonlinear Processes in Biology</a> which have been developed as part of OpenDreamKit <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/49">D2.9</a>. In this scenario, authoring is done using reStructuredText format. It provides the ability to create interactive code cells using <a href="https://sagecell.sagemath.org/">SageCell</a> or <a href="https://github.com/minrk/thebelab">thebelab</a>. It requires more effort to setup the environment and to author materials, but one gain better interactivity features and more control onver the output. Addtionally, using the sphinx system one can produce high quality “static” PDF format.</p>
<h2 id="discussion">Discussion</h2>
<p>By publishing the course materials in the form of notebooks and using
provided the template repository, John easily achieves following:</p>
<ol>
<li>
<p>He can compile selected notebooks to a book form using LaTeX
typesetting software. It can be printed and used in non-interactive
way as a regular book.</p>
</li>
<li>
<p>Using Binder, notebooks can be launched at any time without any
authorization. In this way is it possible to use in during lecture or
a tutorial as live interactive examples.</p>
</li>
<li>
<p>The repository can be cloned and on students computers. This allows
for individual work of students on their assignments or exploration of
the subject.</p>
</li>
</ol>
<h2 id="time-and-expertise-required">Time and expertise required</h2>
<p>Assuming John is familiar with version control and Jupyter (basic lab
skills taught at <a href="http://software-carpentry.org/">Software Carpentry</a>,
and is fluent in using Jupyter notebook, the initial setup of the
course repository could take few hours the first time. Further
development should not generate any significant overhead compared to
regular use of Jupyter notebook.</p>
<h2 id="whats-new-since-opendreamkit-started">What’s new since OpenDreamKit started</h2>
<ul>
<li>Apparition of <a href="http://mybinder.org">Binder</a>;</li>
<li>Expansion of the <a href="http://jupyter.org">Jupyter</a> technology;</li>
<li>Providing demonstrators as examples of using interactive books
technology in practice.</li>
<li>Providing 3d visualization software which can extent application of Jupyter
notebook rely on 3d visualisation (e.g. fluid dynamics)</li>
</ul>
<h2 id="opendreamkit-contribution">OpenDreamKit contribution</h2>
<ul>
<li>Development and contributions to Jupyter interfaces (kernels) for
math software (GAP, Pari/GP, SageMath, Singular) and C++;
see <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/96">D.47</a>.</li>
<li>Contributions to the packaging of math software (GAP, Pari/GP,
SageMath, Singular, …); see
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/58">D3.1</a>
and
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/59">D3.10</a>;</li>
<li>Early adoption of Binder;</li>
<li>Contributions to the
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/238">deployment of new Binder instances</a>;</li>
<li>Advertising, training,</li>
<li>Providing <a href="https://github.com/OpenDreamKit/authoring_cookie_cutter">a template</a>
of interactive book repository,</li>
<li>Development of 3D visualization software for Jupyter notebook
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/86">D4.12</a></li>
<li>Development of demonstrators of interactive books technology:
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/39">D2.14</a></li>
</ul>
MathHub Data launch2019-08-31T00:00:00+00:00http://opendreamkit.org/2019/08/31/MHD-launch<p>After ten months of work, the FAU group launched the <a href="https://data.mathhub.info">MathHub Data</a> portal.</p>
<p>MathHub Data is a unified infrastructure to support deep FAIR relational mathematical data.
It builds on our MathHub system, a portal for narrative and symbolic mathematical data.
MathHub Data is a part of the MathHub portal and provides storage and hosting with integrated support for deep FAIR.
In the future, this will also allow for the development of mathematical query languages
(i.e., queries that abstract from the encoding) and mathematical validation
(e.g., type-checking relative to the mathematical types, not the database types).</p>
<p>We have imported a few real world datasets into the system.
These include:</p>
<ul>
<li><a href="https://data.mathhub.info/collection/ab/">additive bases</a>
(by <a href="http://emis.ams.org/journals/JIS/VOL17/Kohonen2/kohonen5.pdf">Jukka Kohonen</a>),
the full collection of extremal additive bases of up to 2525 elements;</li>
<li><a href="https://data.mathhub.info/collection/lat/">lattices</a>
(by <a href="https://doi.org/10.1007/s11083-018-9475-2">Jukka Kohonen</a>);</li>
<li><a href="https://data.mathhub.info/collection/maniplexes/">regular maniplexes</a>
(by Gabe Cunningham);
a collection of all regular maniplexes of rank 3 with automorphism groups of order 200 or less;</li>
<li><a href="https://data.mathhub.info/collection/polyhedra/">regular polyhedra</a>
(by Gabe Cunningham),
a collection of all regular abstract polyhedra with automorphism groups of order 2000 or less.</li>
</ul>
<p>We are also preparing two larger imports to test the system.</p>
Final reports sprint2019-08-26T00:00:00+00:00http://opendreamkit.org/2019/08/26/FinalReportsSprint<p>This workshop will bring together OpenDreamKit participants just
before the end of the project to collectively:</p>
<ul>
<li>Finish up writing our final reports</li>
<li>Prepare demos for the final review</li>
<li>Any other collaborative ODK work</li>
</ul>
<p>And last but not least:</p>
<ul>
<li>Celebrate the last week of our joint ODK adventure!</li>
</ul>
<p>All ODK participants (or close collaborators) are welcome. Funding is
as usual: Paris-Sud will pay for local expenses (food + lodging);
sites pay for the travel expenses of their members.</p>
<p>Like our usual Cernay workshop, this will be rather informal: we will
cook for each other, share rooms, etc. See the notes on the <a href="https://hackmd.io/4b8zBeKLS6i9w84G0ChanA">local
organization</a>.</p>
<p><a href="https://hackmd.io/_qX_YnOpT6eG6kl2xRqJAQ">Report writing coordination page</a></p>
<p>For registration, please fill in the <a href="https://framadate.org/tfuHjZgcSU8pHI45">date
poll</a> where date N is to be
interpreted as “staying overnight from date N to date N+1”. Leave a
comment on the poll if you care about having a single room. When you
know them, please also specify your arrival and departure time on the
<a href="https://hackmd.io/HR0KV8IJT2qlAOqRBhqW6A">pad</a>.</p>
Report on the Workshop on Data in Mathematics, Cernay-la-Ville2019-08-25T00:00:00+00:00http://opendreamkit.org/2019/08/25/MathDataWorkshop-Report<p>The <a href="https://opendreamkit.org/2019/08/17/WorkshopOnDataInMathematics/">workshop on data in mathematics</a> brought together fourteen participants.
They represented authors of mathematical datasets,
interested users of such datasets, data framework developers,
knowledge engineers, and experts interested in integrating mathematical databases
with computer algebra systems.</p>
<p>The organisers had to limit participation partly because of the early stages of the project.
Because the system is not mature yet, many developers were needed during the workshop
to support the dataset authors.
This number will decrease quickly as the system matures, and we expect to be able to invite
more dataset authors to future installments of the workshop.</p>
<p>Testimonials</p>
<p>AK: We had a super intense workshop with people with very different, complementing competencies all strongly willing to work together towards the same goal of pushing the data on MathHub idea: a real team evolved and with that a push beyond my expectations.</p>
<p>JK: For me this was a great way of getting to know people working towards open math data. It was also an opportunity to see the general idea from different perspectives, including concrete databases, user interfaces, and formal and informal logic interconnecting the data. All this and good concrete steps taken within a superbly leisurely setting – I was absolutely enjoying my time!</p>
<p>GC: By bringing together the developers of the MathHub with the mathematicians who are producing and analyzing math data, the workshop accomplished in a week what would have otherwise taken months. It was immensely gratifying to see how quickly the system evolved into something that is already better than the state of the art for most mathematicians.</p>
<h2 id="workshop-results">Workshop results</h2>
<h3 id="rewrite-and-mayor-improvements-of-infrastructure">Rewrite and mayor improvements of infrastructure</h3>
<p>The FAU group had developed a working prototype for an infrastructure
for mathematical data in early 2019.
In the two weeks leading up to the workshop and during the workshop,
Tom Wiesing had completely rewritten the system in Django.
The decision to switch backends was made to allow for a more
agile development in the future.</p>
<p>We named the system <a href="https://data.mathhub.info">Data.MathHub</a>, to indicate integration with <a href="https://mathhub.info">MathHub</a>,
a portal for active mathematical documents and an archive for flexiformal mathematics.
MathHub Data provides a unified infrastructure supporting deep FAIR relational mathematical data.
It provides storage and hosting for mathematical datasets.
The infrastructure produced includes everything necessary to display
a simple website with basic search functionality.</p>
<p>During the workshop, the developers made significant improvements
in the architecture and the data model.
The workshop participants sketched out a submission and editorial process for the platform</p>
<h3 id="analysis-of-interface-requirements">Analysis of interface requirements</h3>
<p>Dr. Andrea Kohlhase tested the user experience of the existing web interface
through user interviews.
She also produced a clickable prototype of an updated interface and used that
in a few interviews combined with an eye-tracker test.</p>
<p>Discussions on the user interface showed that the system will need to support from
contributors beyond dataset authors and infrastructure developers.
The interface</p>
<ul>
<li>would reach more users if it were available in multiple languages,</li>
<li>needs to show definitions of concepts,</li>
<li>needs to have good defaults for ordering of conditions and columns, and</li>
<li>needs to have good defaults for which columns are visible.
All of these features are valuable, since they make the system more usable.</li>
</ul>
<h3 id="building-ties-with-the-mathematical-community">Building ties with the mathematical community</h3>
<p>The mathematicians taking part in the workshop were enthusiastic to see
the quick results and usefullness of the MathHub Data system,
despite the system not being mature yet.</p>
<h3 id="discussions-about-a-possible-journal">Discussions about a possible journal</h3>
<p>The participants discussed the wider problems that members of the
mathematical community face when it comes to authoring software and datasets.
There is a real need in the community to give some sort of academic recognition
to software and dataset authors.
It was pointed out that senior researchers should at this point advise young people
to not invest time in dataset libraries or software, because they are not going
to get credit for it.
At the same time, it is precisely the young people who are contributing most
to datasets libraries and software.</p>
<p>To provide a basis for giving academic recognition, Michael Kohlhase pointed out
that a variety of personal resources will be needed:</p>
<ul>
<li>mathematicians with good standing in the field who can become guardian angels,</li>
<li>people who can provide hosting, so we can point to some landing page,</li>
<li>people who are actually willing to review datasets.</li>
</ul>
<h3 id="conclusions">Conclusions</h3>
<p>The workshop showed that it can be fruitful to bring together researchers who
know what kind of datasets their community would benefit from and researchers who know how to generate such datasets.
One such collaboration started at the workshop.
Finally, but probably most importantly, the workshop showed that the research community
accepts the idea of a system like data.mathhub.info and is taking it seriously to the extent
that several participants offered help with tasks such as dataset reviews.</p>
<h2 id="future">Future</h2>
<p>The workshop was very successful even with the somewhat limited outside participation
(indeed that was probably a factor in the success). As an effect, an outside member of the
math data community has bid for a second installment in spring of 2020 in Slovenia and
organisation is underway for that.</p>
Sharing a mathematical dataset online2019-08-24T00:00:00+00:00http://opendreamkit.org/2019/08/24/SharingAMathematicalDataset<h2 id="introduction">Introduction</h2>
<p>Mathematicians produce data as part of their research.
They do this to find patterns and test conjectures.
Because of the complexity of the data, the datasets tend to be hard
to share with other researchers.
A lot of effort needs to go into developing a suitable infrastructure,
particularly so for the larger projects such as
the <a href="https://oeis.org">Online Encyclopedia of Integer functions</a> (OEIS), the
[L-Functions and Modular Forms Database](https://lmfdb.org (LMFDB), the <a href="https://www.gap-system.org/Packages/smallgrp.html">Small Groups Library</a>, and the <a href="https://hog.grinvin.org/">House of Graphs</a>,
to name just a few.
Resources needed to make a mathematical dataset useful to other researchers
are typically beyond what a typical researcher has available to them.</p>
<h2 id="scenario">Scenario</h2>
<p>Gabe works in the area of abstract polytopes and would like to share
a small dataset he has produced with his colleagues.
He has generated all regular maniplexes of rank 3 with
automorphism groups of order 200 or less.
In addition to the maniplexes, he has computed a few mathematical properties
for each of them.
These properties are helpful in understanding the structure of objects.
While Gabe is comfortable using a database which he can use
to filter and search, his colleagues are not.
He wishes there were an easier to use interface he could point them to.
In particular, he would like his dataset to be avalable online,
so that the users can</p>
<ul>
<li>list all the contents, and</li>
<li>filter maniplexes by their properties.</li>
</ul>
<h2 id="suggestion-of-solution">Suggestion of solution</h2>
<p>He submits his dataset to the <a href="https://data.mathhub.info/">Data.MathHub</a> platform.</p>
<p>Once his data is imported, he shares the link with his colleagues.</p>
<h2 id="some-instances">Some instances</h2>
<ul>
<li><a href="https://data.mathhub.info/collection/maniplexes/">Regular maniplexes</a></li>
<li><a href="https://data.mathhub.info/collection/smallgp/">A subset of the Small Groups Library</a></li>
</ul>
<h2 id="discussion">Discussion</h2>
<p>By publishing his dataset on MathHub Data, Gabe has achieved the
<a href="https://docs.mathhub.info/data/">following</a>:</p>
<ul>
<li>other researchers can easily interact with his dataset,</li>
<li>his data will stay available in the long-term,</li>
<li>user interfaces, APIs, and interoperability at the database level and the mathematical level,</li>
<li>his data will reach a broader audience more easily.</li>
</ul>
<h2 id="time-and-expertise-required">Time and expertise required</h2>
<p>Gabe needs to describe the structure of the dataset in JSON format.
Similarly, he need to submit his data in JSON.</p>
<h2 id="whats-new-since-opendreamkit-started">What’s new since OpenDreamKit started</h2>
<p>Before OpenDreamKit, Gabe would have had few options if his data was not
in the area covered by one of the larger projects.
He could have contacted the authors of DiscreteZOO, but that project
had little resources and was not maintained on a regular basis.
The other alternative would be to set up a website himself, with
whatever resources he had available himself.</p>
<p>Since the end of <a href="https://opendreamkit.org">OpenDreamKit</a>, everyone with basic skills for working with data
can use <a href="https://data.mathhub.info">Data.MathHub</a> to share their datasets.
Indeed, datasets such as Gabe’s are the focus of the project.</p>
Report Writing Workshop Cernay-la-ville, France, 24th-31th of August 20192019-08-24T00:00:00+00:00http://opendreamkit.org/2019/08/24/ReportWritingSprint<p>## Event summary</p>
<p>This workshop brought together OpenDreamKit
participants just before the end of the project to collectively:
write the project reports, prepare demos for the final review, and
other collaborative OpenDreamKit work.</p>
<p>## OpenDreamKit implication</p>
<p>13 OpenDreamKit members participated to this
workshop. Paris-Sud paid for local expenses (food and
accommodation); Other sites paid for the travel expenses of their
members.</p>
<p>## Results and impact</p>
<p>The OpenDreamKit members worked on the writing
of 13 deliverables and also made progress on the technical report
writing. 7 deliverables were submitted during that week. In
addition, the already submitted deliverable D5.13 was improved and
re submitted.</p>
<p><img src="/public/images/events/2019-08-24-ReportWritingSprint.jpg =75%" alt="" /></p>
<p>Report Writing Workshop</p>
Workshop on Data in Mathematics2019-08-17T00:00:00+00:00http://opendreamkit.org/2019/08/17/WorkshopOnDataInMathematics<p>This community building workshop will bring together interested users and authors of mathematical datasets,
data framework developers, and experts interested in integrating mathematical databases with computer algebra systems.</p>
<p>The workshop is organized by the H2020 European project <a href="http://opendreamkit.org">OpenDreamKit</a>.</p>
<p>We will continue adding information as it becomes available to this page.</p>
<p><strong><em>Dates:</em></strong> August 17th-24th</p>
<p><strong><em>Organising team:</em></strong> Katja Berčič, Nicolas Thiéry, and Primož Potočnik</p>
<h2 id="program">Program</h2>
<p>We will predominantly be working in small groups performing code/development sprints. To
structure the day and ensure communication, we adopt the following daily schedule.</p>
<ul>
<li>09:00–10:00: Planning Plenary</li>
<li>10:00–12:30: Code Sprints/Tutorials</li>
<li>12:30–14:00: Joint Lunch</li>
<li>14:00–17:00: Development Sprints/Tutorials</li>
<li>17:00–18:00: Planning Plenary</li>
<li>18:00–19:00: Joint Dinner</li>
</ul>
<h2 id="funding">Funding</h2>
<p>OpenDreamKit will cover the local expenses for all participants and
the travel expenses of most to all participants.</p>
<h2 id="participants">Participants</h2>
<p>We are excited to announce the following (tentatively) confirmed attendees:</p>
<ul>
<li>Odile Bénassy</li>
<li>Katja Berčič</li>
<li>E. Madison Bray</li>
<li>Gabe Cunningham</li>
<li>Dennis Müller</li>
<li>Andrea Kohlhase</li>
<li>Michael Kohlhase</li>
<li>Jukka Kohonen</li>
<li>Samuel Lelièvre</li>
<li>Florian Rabe</li>
<li>Julian Rüth</li>
<li>Nicolas Thiéry</li>
<li>Michael Torpey</li>
<li>Tom Wiesing</li>
</ul>
<h2 id="arrivals-and-departures">Arrivals and departures</h2>
<p>See the <a href="https://hackmd.io/HR0KV8IJT2qlAOqRBhqW6A">arrivals and departures pad</a>.</p>
Data in Mathematics Workshop Cernay-la-ville, France, August 17th to 24th2019-08-15T00:00:00+00:00http://opendreamkit.org/2019/08/15/MathDataWorkshop<h2 id="main-goals">Main goals</h2>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>This event was organized and funded by OpenDreamKit (Paris Sud, FAU).
OpenDreamKit funded accommodation for all participants, as well as travel expenses for all but two.</p>
<h2 id="event-summary">Event summary</h2>
<p>This workshop brought together interested users and authors of mathematical datasets,
data framework developers, and experts interested in integrating mathematical databases with computer algebra systems.
Participants discussed general issues related to data in mathematics,
as well as working on concrete steps towards improving the status.</p>
<h2 id="demographic">Demographic</h2>
<p>The workshop was attended by
two PhD students,
a postdoc,
four research software engineers,
and seven researchers from various areas of mathematics and computer science.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop allowed for a lot of time for free discussion.
Several pain points experienced by mathematicians that work with data came up this way,
were noted, and several cases already acted upon.
In particular, there appears to be a real need for a journal dedicated to
mathematical datasets and mathematical software.</p>
<p>Discussions on provenance of data in mathematics,
initiated by Dr. Michael Kohlhase, resulted in
a draft of a standard and formalisation of math data provenance.</p>
<p>Similarly, participants collaborated on how the web interface
for datasets in mathematics should look like.
This effort was led by Dr. Andrea Kohlhase and resulted in
a clickable prototype in addition to a long list of requirements,
sorted into must–have, should–have, and can’t–have.</p>
<p>Participants from FAU are working on an infrastructure for math data and
significant effort went into improving this prototype system.
As planned, we were able to import several real-life datasets.
This included writing up descriptions, metadata (including provenance),
formalising the mathematics.
In addition to the datasets that were planned for import,
several participants identified the need for new datasets for their
research communities, and have started to work on them.
The diverse backgrounds of the participants made for a helpful
environment for this.</p>
<p>Participants also worked on side projects related to data in mathematics.
Odile Bénassy was able to quickly produce a basic Jupyter interface to the system,
opening up new interface possibilities.
Several user stories were collected by Gabe Cunningham, and will serve as use cases for the system.
Participants also added content to a multilingual math glossary.</p>
<p><img src="/public/images/events/math-data.jpg =75%" alt="" />
Data in Mathematics Workshop, Cernay, France</p>
<p>The following testimonials indicate the degree of progress made in a single week.</p>
<p>Gabe Cunningham</p>
<p><strong>``By bringing together the developers of the MathHub with the mathematicians who are producing and analyzing math data, the workshop accomplished in a week what would have otherwise taken months. It was immensely gratifying to see how quickly the system evolved into something that is already better than the state of the art for most mathematicians.’‘</strong></p>
<p>Andrea Kohlhase</p>
<p><strong>``We had a super intense workshop with people with very different, complementing competencies all strongly willing to work together towards the same goal of pushing the data on MathHub idea: a real team evolved and with that a push beyond my expectations.’‘</strong></p>
<p>Jukka Kohonen</p>
<p><strong>``For me this was a great way of getting to know people working towards open math data. It was also an opportunity to see the general idea from different perspectives, including concrete databases, user interfaces, and formal and informal logic interconnecting the data. All this and good concrete steps taken within a superbly leisurely setting – I was absolutely enjoying my time!’‘</strong></p>
GAP Singular Meeting and School2019-08-15T00:00:00+00:00http://opendreamkit.org/2019/08/15/GAPSingularMeeting<p>We are pleased to announce a GAP-Singular-Meeting, taking place
at the PfalzAkademie, Lambrecht, Germany.</p>
<p>The meeting consists of three parts: A school,
a mini conference, and several workshops.</p>
<p>The school is meant for beginners as well as slightly advanced people
and aims to give an introduction to the using GAP and Singular in research.<br />
At the mini conference, participants can present their
computer algebra related work.<br />
Afterwards, participants are encouraged to attend workshops,
in order to work with expert computer algebra system developers
on various topics, including their own research.</p>
<p>Please see the <a href="https://opendreamkit.org/meetings/2019-04-02-GAPSingularMeeting/">meetings page</a> for details.</p>
GAP/Singular School and Meeting PfalzAkademie, Lambrecht, Germany, 15--23 August, 20192019-08-15T00:00:00+00:00http://opendreamkit.org/2019/08/15/GAP-Singular-school-meeting<h2 id="main-goals">Main goals</h2>
<p>The main goals were to finish OpenDreamKit related developer tasks as well as
to introduce beginners to GAP and Singular, present
computer algebra–related work, and share ideas through workshops.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The event was fully funded by OpenDreamKit. Costs are still pending due to
overseas travel claims.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event consisted of three parts:</p>
<ul>
<li>School: a sequence of workshops to introduce beginners to GAP, Singular,
and open-source development principles.</li>
<li>Mini-conference: talks to allow participants to present computer
algebra–related work to each other.</li>
<li>Workshops: a set of workshops dedicated to specific computer algebra
problems.</li>
</ul>
<h2 id="demographics">Demographics</h2>
<p>The workshop had just under 50 registered participants, not all of which
were able to attend (VISA delays).
The participants came from more than 10 countries on 4 continents.
We had a total of 8 women attending.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>In the School, tutorials were given that introduced participants to:</p>
<ul>
<li>Best practices in software development (Max Horn);</li>
<li>Basic use of GAP (Michael Torpey);</li>
<li>Advanced Topics in GAP (Thomas Breuer);</li>
<li>Basic use of Singular (Christian Eder, Andreas Steenpaß, Isabel Stenger);</li>
<li>Parallel modular algorithms in Singular (Christian Eder, Andreas Steenpaß, Isabel Stenger);</li>
<li>CAP: Categories, algorithms, programming (Sebastian Posur).</li>
</ul>
<p>These tutorials were complemented by informal discussions, and chances for
learners to answer questions. The <code class="language-plaintext highlighter-rouge">Best practices'' and</code>Basic use of GAP’’
sessions followed material from appropriate Software Carpentry courses
(https://software-carpentry.org/lessons/): <code class="language-plaintext highlighter-rouge">Version Control with Git''
and</code>Programming with GAP’’ respectively.</p>
<p>The workshops of basic Singular and GAP were well received - several
participants were either novices in GAP or Singular.
For the non-beginners, the advanced workshops on modular algorithms in
Singular and on CAP (a GAP package for categories, in particular in,
but not restricted to, algebraic geometry) proved very intersting.</p>
<p>The next part, contributed talks, covered only a single day. However
the talks originated in many different areas of mathematics and covered
many successful challenging applications of components (GAP, Singular)
of OpenDreamKit.</p>
<p>The final part, comprised of workshops on GAP, Singular and CAP
allowed us to close off remaining issues with regard <br />
to the integration of the OpenDreamKit work into the various packages, e.g. <br />
improvements to the integration of the fast multivariate polynomial <br />
arithmetic in Singular. The workshop was in particular extremely useful in <br />
bringing participants together to discuss some of the remaining technical <br />
challenges and in understanding the mathematical challenges real world <br />
researchers were facing and how they could be solved using tools developed <br />
during OpenDreamKit. For example, it was discovered during the workshop that a <br />
bottleneck experienced by one of the participants was down to fast <br />
multivariate polynomial arithmetic (via rational functions) which was then <br />
directly worked on by participants of the meeting. There was also a project <br />
to improve integration of Singular into the Singular.
subsystem of the <br />
Oscar computer algebra system specifically so that Gr"{o}ber bases could <br />
be computed over coefficient fields implemented by OpenDreamKit components. This <br />
project was finalized and merged at the workshop after months of <br />
unsuccessful previous attempts.</p>
<p>The GAP workshop was part of the GAP 4.11 release process which is critical
for the dissemination of OpenDreamKit components such as \libGAP. At the workshop,
changes to the \libGAP system were documented and significant progress was
made towards rapping up the release to reach users. This included
dissemination of GAP into the Debian distribution. There were also lively
discussions about the GAP-Jupyter integration.</p>
<p><img src="/public/images/events/gap-singular-participants =50%" alt="" /></p>
<p>GAP–Singular meeting, PfalzAkademie, Lambrecht, Germany</p>
ESSLLI 2019 Course on Formalizing the Zoo of Logical Systems Riga, Latvia, 05th-09th of August 20192019-08-04T00:00:00+00:00http://opendreamkit.org/2019/08/04/ESSLLI-Zoo<h2 id="main-goals">Main goals</h2>
<p>In this course, we disseminated methods and solutions developed during OpenDreamKit to young researchers.
In particular, this involved the systematic modular formalization of formal languages in which mathematical knowledge is expressed.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The course was organised by Michael Kohlhase (FAU) and Florian Rabe (FAU).</p>
<h2 id="event-summary">Event summary</h2>
<p>The course consisted of 5 sessions of 90 minutes each, one each from Monday to Friday.
It was attended by undergraduate, graduate, and PhD students from mathematics, philosophy, and computer science.
Attendance was very big with 43 participants in the first sessions and 25 in the last.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The main result of the workshop was the dissemination of key OpenDreamKit ideas and methods.</p>
Report on Sage Days 102, University of Ibadan, Nigeria2019-07-29T00:00:00+00:00http://opendreamkit.org/2019/07/29/SageDays102<p>During the week of July 15 - 19 we held <a href="https://wiki.sagemath.org/days102">Sage Days
102</a> at the University of Ibadan, in
Ibadan, Oyo State, Nigeria. This was (to our knowledge) the first ever
week-long workshop dedicated primarily to SageMath in Nigeria (although not
the first ever in West Africa; that honor belongs to <a href="https://wiki.sagemath.org/days34.5">Sage Days
34.5</a> in Burkina Faso, October 2011),
and there was immense interest, with 80 registered participants, and many
others who had to be wait-listed.</p>
<p>The primary focus of the workshop was to introduce students and faculty to
computational mathematics with SageMath, as well as to on-board them to open
source mathematical and scientific computing in general. To that end we also
held sessions on GAP and R with the help of local experts.</p>
<p>All of the on-site organization and funding was provided by the University
of Ibadan Department of Mathematics, while OpenDreamKit provided travel
funding for four instructors (Erik Madison Bray from France, Yaé Ulrich Gaba
from South Africa, Evans Doe Ocansey from Austria, and Dr. Chimere Stanley
Anabanti from Nigeria), as well as for two PhD students (Olivia Adu-Poku
from Ghana and Gilda Bansimba from Congo). The instructors were also joined
by Dr. Olalekan K. Obisesan, a professor of statistics local to the
University of Ibadan.</p>
<p>The genesis of this workshop was at the <a href="https://opendreamkit.org/2019/02/11/FreeComputationalMathematicsConference/">Free Computational
Mathematics</a>
conference, held at CIRM in France, February 2019, also sponsored by
OpenDreamKit. That conference was attended by Ini Adinya, a lecturer in
mathematical finance at the University of Ibadan, along with two of her
colleagues. They were impressed by how much they were learning at that
conference, and wanted to replicate it, or something like it, at the
University of Ibadan. Ini and Erik had worked together during the
conference on some of her programming problems, and Erik agreed to come help
out and teach at such a workshop if they organized it. And so, shortly
afterwards we got the ball rolling, with a big kick of help from Viviane
Pons who helped with much of the initial organizing, and especially the
search for additional instructors who would be willing to travel to (or from
within) Nigeria, as without several instructors there was no way the
workshop could be successful. One of the IT managers at AIMS, the African
Institute for Mathematical Sciences, saw Viviane’s e-mail on one of the Sage
mailing lists and forwarded it to Evans and Yaé, who had prior experience
teaching Sage together in South Africa. They in turn agreed it would be fun
to team up to help teach Sage in Nigeria, and we couldn’t have done it
without them.</p>
<p>The following are Erik’s notes on the workshop.</p>
<h1 id="the-workshop">The Workshop</h1>
<p>The workshop was held during the week, Monday through Friday, with official
times beginning at 9:00 am and ending at 5:00 pm, with a coffee break and a
lunch break, with catering provided by the department. (While this schedule
gave everyone a chance to breathe at the end of the day, as the exit-survey
suggests it left many people wanting for more, as a something like an
intensive “boot camp” retreat might have provided; an idea for a future
project.) Some teaching time was also lost to infrastructure problems, most
notably a power outage on Monday that lasted almost the entire afternoon
after lunch. Even then we persevered, stretching laptop batteries as long as
possible and using the whiteboard.</p>
<p>Despite that and other challenges (e.g. spotty internet access, technical
problems with people’s computers and the projector, and not overall enough
instructors for the size of the group) there was a lot of spirit and
excitement for the subject matter, and most attendees got a lot out of it
and wished it could go even longer.</p>
<p>We started off Monday with a brief opening ceremony lead by leadership from
the Department of Mathematics and from the Faculty of Science including
Head of Department Dr. Deborah O. Ajayi who co-organized the workshop, and
Prof. A. Ayodele, Dean of the Faculty of Science. Followed by this was a
brief talk to introduce Sage, the community around it, and to provide some
of the background to free and open source research computing in general, and
why it can be advantageous. Originally this talk was supposed to have some
slides and live demos, but as we did not have the projector working yet I
(Erik) gave the talk off-the-cuff to keep things moving while the technical
kinks were worked out.</p>
<p><img src="/public/images/SageDays102/opening.jpg" alt="Opening speech by Dr. Ajayi" class="img-responsive" /></p>
<p>Then, after the coffee break, I lead a brief introduction to using the
command-line shell, based on the <a href="https://swcarpentry.github.io/shell-novice/">Software Carpentry
lesson</a> but only with time to
cover basic filesystem navigation and manipulation, in which we wanted to
make sure all attendees had some common grounding. We had ambitions to then
teach some basic Git, in part that we might use it to distribute files, and
updates to those files, from the <a href="https://github.com/gabayae/sage-days-102">Git repository for the
workshop</a>.
However, due in large part to inconsistent network access between the
attendees, we had to abandon that idea and resort to distributing files on
USB drives, of which I brought nearly two dozen (more on that in Technical
Notes). The math department did provide a 4G modem/Wi-Fi router for
attendees to connect to, but with around 80 attendees it was difficult to
disseminate connection information to everyone, especially for people who
came on later days and missed the notice that they could use that access
point. In any case the tiny router would not have been able to handle 60+
simultaneous connections.</p>
<p>Here it should be noted that the workshop was being held in a new lecture
hall building being constructed by the University, which when completed
(which it nearly is) will be as state-of-the-art as most I’ve ever been in,
with copious power outlets and ethernet ports at each desk, and back rooms
for the instructors to use as offices, and to gain access to the racks of
ethernet switches that the desk ports will be hooked up to. At the time of
this workshop it just wasn’t fully hooked up yet, and was so new that there
was still shrink-wrap on the furniture and even on the whiteboard, which we
only even figured out on the second day.</p>
<p><img src="/public/images/SageDays102/whiteboard.jpg" alt="The whiteboard was not working well for some time before we realized it
still had its protective wrapping
on" class="img-responsive" /></p>
<p>By the time another such workshop is held, the lecture halls are expected to
be completed and fully functional. Even as-is I felt it worked quite well as
a facility. The only suggestion I might have made would be for a speaker
system and microphone, as even with shouting it was hard to reach people in
the very back of the long room.</p>
<p>We finished Monday by getting Sage installed on as many machines as
possible, ensuring that most attendees could run the Jupyter Notebook, and
giving them an introduction to Python led by Yaé. Tuesday continued largely
with more of the same, with Yaé and Evans teaching introductory Python
programming (including basic types and data structures, loops and other
control structures, and how to define functions), as well as some basic
features of Sage itself.</p>
<p>At the end of each day (even as soon as Monday, though nobody volunteered on
the first day) we also set aside some time for “lightning talks”. We wanted
to give attendees a chance to speak to the whole group about their research
problems, or just something interesting they are working on. There were too
many people to give <em>everyone</em> an opportunity to speak, but we opened slots
up so that those who wanted to could. Originally these talks were only
intended to be 5 to 10 minutes each, but people were shy about signing up,
and we only had about two per day. But that gave those people more time to
speak and their talks were successful. I think, maybe, if we had
communicated this plan to hold lightning talks further in advance, maybe
more people would have been excited to try to give a talk (they are short
and do not require much preparation).</p>
<p><img src="/public/images/SageDays102/sage-session.jpg" alt="Evans teaches introduction to Sage" class="img-responsive" /></p>
<p>Our original plan was that by Wednesday, attendees should break off into
more interest area-specific groups and possibly work on some specific
problems or group projects. In particular, Evans has introduced us to
creating interactive widgets in the Jupyter Notebook, with the hope that
attendees would be inspired to participate in a contest to develop the most
creative and interesting interacts. However, it was clear that we needed to
spend more time in the first half of the day on getting everyone up to speed
on Sage and Python basics, which was well appreciated.</p>
<p>For the second half of Wednesday we split into four separate, parallel
sessions for different topic interests:</p>
<ul>
<li>
<p>Graph theory and discrete math with Sage, headed by Erik and Evans.</p>
</li>
<li>
<p>Numerical analysis and calculus with Sage, headed by Yaé.</p>
</li>
<li>
<p>Group theory and algebra with GAP, headed by Chimere.</p>
</li>
<li>
<p>Statistical analysis with R, headed by Olalekan.</p>
</li>
</ul>
<p><img src="/public/images/SageDays102/sessions.png" alt="Attendance by session: Graph theory: 11; Calculus: 34; Group theory: 11; Statistics: 18" class="img-responsive" /></p>
<p>It was good to have a representative from the GAP community, in part to
drive home the point I was making that SageMath could not exist without all
the technologies it is built on top of, and to give special emphasis to one
of them. Similarly with R, although R is included as part of Sage, and Sage
includes an interface to R, the two are less tightly coupled than Sage and
GAP. Nevertheless, although Sage does have useful facilities for
statistics, it has not often been the focus of most (although some) Sage
developers. So this provided some of the best of both worlds for attendees
who were most interested in statistical analysis.</p>
<p>Chimere also provided a <a href="/public/images/SageDays102/days102-gap-report.pdf">separate
report</a> on the workshop and on what
was taught in his session. It also includes a breakdown of his total
expenses for travel within Nigeria (from Enugu) which may be useful
information to have for planning future workshops.</p>
<p><img src="/public/images/SageDays102/analysis-session.jpg" alt="Yaé teaches
Sage in analysis session" class="img-responsive" /></p>
<p><img src="/public/images/SageDays102/r-session.jpg" alt="Olalekan teaches
R" class="img-responsive" /></p>
<p>This pattern seemed to be the most helpful to the most people, so for
Thursday we repeated it: In the morning we continued the general SageMath
and Python programming instruction from the tutorial notebooks we were
using, and in the afternoon split into the same break-out sessions, although
Olalekan was not able to attend Thursday, so most of his session’s attendees
split off between the other sessions, or worked on their own.</p>
<p>Finally, on Friday, we spent a little more time following up with exercises
that were given in the break-out sessions, following up on attendees’
questions, and giving individual help where possible. This continued most
of the morning. Just before lunch, the whole class had a 30 minute Skype
session with Jessica Striker, an assistant professor at North Dakota State
University, and one of the organizers of the <a href="https://opendreamkit.org/2017/04/06/WomenInSage/">Women in Sage
workshop</a> held in Paris in
2017, also sponsored by OpenDreamKit. She spoke to us about that workshop
and how it came about, as well as others like it since, and answered several
questions. Although I did not get an exact gender breakdown of the
attendees, based on a rough head count it was just about 33% women in
attendance, if not slightly more. There was considerable interest among them
in attending a workshop like this in the future. In fact, at least one
woman told me that she (along with two others from Nigeria) had been
accepted to attend the <a href="https://opendreamkit.org/2019/06/28/WomenInSage/">second Women in Sage
workshop</a> held in June of
2019 in Crete, but had been denied visas, or at least were unable to obtain
them in time. If a similar workshop were held in Nigeria, or elsewhere in
West Africa, it would likely be well-attended.</p>
<p><img src="/public/images/SageDays102/family-friendly.jpg" alt="Attendees of all ages were welcome" class="img-responsive" /></p>
<p>We closed Friday first with a meeting with the Vice Chancellor of the
University, Prof. Abel Idowu Olayinka, attended by the workshop organizers,
instructors, and a few of the students. Then we had a brief but fun closing
ceremony, but many people were not ready for the workshop to be over,
despite losing power shortly afterwards. We continued for nearly another
hour giving brief talks, and closed off by having three students present (as
best they could without a projector) their submissions for the interacts
contest.</p>
<p><img src="/public/images/SageDays102/interacts.jpg" alt="Students presented their interacts" class="img-responsive" /></p>
<h1 id="impact">Impact</h1>
<p>Many attendees expressed immense gratitude that we (the instructors) could
come and hold this workshop. Nigeria is the largest country in Africa by
population, and the seventh largest in the world, with an estimated
population of about 200,000,000 in 2019, and ever growing, and the
University of Ibadan is a very large university of approximately 35,000
students and 1,500 academic staff. It offers a deep well of highly
motivated and talented individuals. Nevertheless, funding for research and
workshops is very limited, and spread thin across the country, inhibiting
opportunities for cooperation and collaboration. Several attendees also
decried Nigeria’s “poor reputation” on the world stage inhibiting
opportunities for funding and for international collaboration. Indeed, the
number of international visitors to the University number in the dozens
annually.</p>
<p>So opportunities like this one, for students and faculty to receive expert
training on new technologies, are not nearly as frequent as they should be
(even despite there being some local experts such as Chimere and Olalekan)
and we tried to make as much of this one as we could.</p>
<p>As previously stated, we needed to spend more time bringing the whole group
up to speed both on Python, and programming skills in general. Had we done
a more formal pre-workshop survey we might have realized this earlier and
planned more teaching time in the first place, as 70% of the attendees did
not rate their programming knowledge highly (that said, this is far from a
perfect measure, as different people have different internal scales for how
much they think they know).</p>
<p><img src="/public/images/SageDays102/programming-knowledge.png" alt="Programming knowledge prior to the workshop: 1: 34.8%; 2: 36.4%; 3: 12.1%; 4: 6.1%; 5: 10.6%" class="img-responsive" /></p>
<p>This suggests that the idea of having a more programming skills-focused
Software Carpentry workshop leading into Sage Days would have been useful
for many attendees. Unfortunately there was just not enough time, as that
alone is typically two full days, and we wanted to get right into the
mathematics as soon as possible. One of the most common
suggestions/wishes we saw in exit survey was just for more time. E.g.</p>
<p><cite>A little bit of days be extended for the workshop.</cite></p>
<p><cite>Increase the number of days and the instructor so that they can move
at our pace and impact us better.</cite></p>
<p><cite>The number of days for the workshop should be extended to allow wider
coverage of the concepts discussed and to aid assimilation</cite></p>
<p>Nevertheless, people came away feeling like they learned more about Sage,
which was the most important thing. We neglected to ask about GAP and R
in the exit survey, but those sessions were also very energetic and engaged,
and were likely similarly successful in imparting new knowledge.</p>
<p><img src="/public/images/SageDays102/sage-knowledge-before.png" alt="Sage knowledge prior to the workshop: 1: 80%; 2: 7.7%; 3: 4.6%; 4: 6.2%; 5: 1.5%" class="img-responsive" /></p>
<p><img src="/public/images/SageDays102/sage-knowledge-after.png" alt="Sage knowledge following to the workshop: 1: 0%; 2: 3.1%; 3: 44.6%; 4: 41.5%; 5: 10.8%" class="img-responsive" /></p>
<p>We also made an effort to ensure that attendees felt like they had resources
to use for further learning on their own, and that they knew where to go to
find help with Sage, and to remain involved in the community.</p>
<p><img src="/public/images/SageDays102/confidence-learning.png" alt="Confidence of ability to find help with and learn more about Sage: 1: 0%; 2: 0%; 3: 16.7%; 4: 47%; 5: 36.4%" class="img-responsive" /></p>
<p>These numbers are encouraging, as they give the impression that attendees of
the workshop will be able to carry on their work with Sage, as well as help
their classmates and colleagues get started with it: Indeed the best
teachers for beginners with new software are often other beginners who have
had just enough expert-guided exposure, which is why having workshops like
this is so important, even if we can’t reach everybody we’d like to in a
single workshop.</p>
<p>In the end, the majority of attendees felt strongly that the workshop was a
good opportunity to build new relationships with their peers, and that what
they learned and experience in the workshop would be of help to their future
careers. We just need more workshops like this, and more frequently.</p>
<p><img src="/public/images/SageDays102/future-collaboration.png" alt="Met interesting people for future collaboration: 1: 0%; 2: 3.1%; 3: 10.9%; 4: 35.9%; 5: 50%" class="img-responsive" /></p>
<p><img src="/public/images/SageDays102/future-impact.png" alt="Learned things that will have an impact on future career/research: 1: 1.6%; 2: 0%; 3: 1.6%; 4: 17.2%; 5: 79.7%" class="img-responsive" /></p>
<h1 id="technical-notes">Technical Notes</h1>
<p>The discussion here is based in part on data collected in the exit survey,
and in part from direct experience helping attendees install and configure
Sage on their personal laptops.</p>
<p>One of the major challenges in the past of running Sage workshops in some
parts of the world such as Africa and parts of Asia has been the
overwhelming dominance of Windows on attendees personal laptops. This is a
challenge, as Sage and most of the computer algebra systems it wraps were
developed in the free and open-source software ecosystem, which
traditionally has been oriented around UNIX-like operating systems such as
GNU/Linux and BSD, as well as MacOS (which presents its own challenges, but
is still compatible with POSIX, the main standard for UNIX-like OSes).
Windows, however, has never been POSIX-compatible, which can make it an
enormous challenge to port complex software to Windows. Thus, Windows users
were long relegated to klunky and slow workarounds, such as a running Sage
in a Linux virtual machine. Even this is not an option in many cases, as
attendees laptops may lack–or barely meet–the hardware specifications
needed to run a VM efficiently.</p>
<p>That situation has vastly improved since the August 2017 release of
<a href="https://opendreamkit.org/2017/10/11/SageWindows/">SageMath for Windows</a>,
thanks to work funded by OpenDreamKit
(<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/50">T3.1</a>). This has
made Sage installation on Windows 7 (64-bit) and up relatively easy, like
installing any other Windows application. Performance on Windows is still
lacking–in particular start-up time of Sage–especially on older machines.
But once it is up and running it provides a fully usable Sage, with
relatively few caveats (and continuing improvement).</p>
<p>Of the attendees of this workshop, almost all were using Windows as their
primary OS. Four people were using Linux. Zero had MacOS. We had asked
attendees to please try on their own to install SageMath before coming, so
that we could have as much time as possible to help those who did not.
Unfortunately very few people did this, in large part just because the
SageMath installer, even while heavily compressed, is nearly 1 GB in size
which was prohibitive for many attendees to download. In anticipation of
this, I brought nearly two dozen USB drives, many of them pre-loaded with
the Sage installers for most supported OSes. By passing these around, most
attendees were quickly able to copy the installer to their computers and run
it. In retrospect, before anything else, we should have led the entire
class in a tutorial on this and do it all together, as there was a
large-enough portion who needed help, and those who had already done it
could take that opportunity to help their neighbors as well. Instead, much
time was spent, especially the first two days of the workshop, on individual
support. Though as some attendees learned more and got more comfortable with
the software, they also began enthusiastically helping each other as we’d
hoped.</p>
<p><img src="/public/images/SageDays102/cooperation.jpg" alt="Attendees worked closely with each other to overcome technical
challenges" class="img-responsive" /></p>
<p>The Sage for Windows installer has improved enough in the last year that
most attendees did not encounter any problems with running the installer, or
starting Sage once it was installed. By far, the largest technical
challenge was presented by the plethora of different virus scanners and
other PC security software installed on peoples’ computers. Sage for
Windows works on top of <a href="https://www.cygwin.com/">Cygwin</a>, the venerable
POSIX emulation system for Windows. A long-known issue is that certain
virus scanners can interfere with Cygwin in different ways: Sometimes that
interference occurs at runtime resulting in strange bugs. Other times it
happens passively: For example some virus scanners would delete files from
the Sage/Cygwin installation in which it found false-positives. In almost
all cases where users were experiencing strange bugs, either during
installation or runtime, adding an exception for Sage to their virus
scanners, or even temporarily disabling their virus scanners, would resolve
the problem. So a major take-away was that we need better documentation of
this issue for Sage users on Windows, and instructions for how to create
exceptions for Sage in various popular virus scanners.</p>
<p>The other major issue was that due to technical difficulties inherent to
both Windows, and the large size of Sage, the Windows version still does not
work at all for 32-bit installations of Windows. We hope that the number of
installed 32-bit Windows will decrease over time, but it is still more
common than we would like, especially on some older laptops we saw with low
amounts (2 GB) of RAM. This is even despite the machines having 64-bit CPU
architectures.</p>
<p>For now, the only available workaround for 32-bit Windows users was to run a
Linux distribution with Sage pre-installed. But rather than have them run
it in a VM, they could use a “live Linux” USB drive, which boots their
computer directly (without installing anything) into a Linux distribution
called <a href="https://desktop.aims.ac.za/">AIMS Desktop</a>. Developed by the IT
department of AIMS South Africa, AIMS Desktop provides a user-friendly Linux
distribution designed specifically for running open source mathematics and
science software, and includes a working version of Sage out-of-the-box.
This allowed users to boot into a fully working and efficient OS, and start
up Sage, often in as little as a minute.</p>
<p>Using AIMS Desktop is a double-edged sword: On one hand it puts users in a
new, possibly unfamiliar computing environment, without access to all their
normal programs or files (though we did have time to show some users how to
access their Windows files from within AIMS Desktop). On the other hand,
for explicit purpose of running software like Sage, it provides an arguably
better overall user experience, and can run better than Windows on older
computers, as the operating system itself is less resource-intensive. It
was designed with users throughout Africa in mind, though nothing about it
is <em>specific</em> to Africa. Of the 13 attendees who tried AIMS Desktop, at
least 5 liked it enough to install it permanently on their computers,
side-by-side with Windows in a dual-boot configuration. Of those we helped
with AIMS Desktop installation, the installation only failed on one machine
(an older Compaq laptop), which was unable to boot it after installation.
Based on a count of the USB drives that were returned, at least 9 attendees
also kept their AIMS Desktop USB drives, and might decide to install it
later (most of them asked for permission to keep it, but this was partly the
intent in bringing so many in the first place so it was expected and even
hoped that people would keep them). It could even be said that those who
were using AIMS Desktop were some of the most engaged, as not being on their
normal Windows installations provided fewer distractions.</p>
<p>There were still more 32-bit Windows users than expected, however, and it
was unfortunate to have to tell them that they couldn’t install Sage. It
would be best if they could seek local tech support to upgrade to a 64-bit
Windows, where possible. But it might also be worth re-visiting the idea of
creating a 32-bit version of Sage for Windows. It’s possible that a
stripped-down version of Sage, that does not work in all edge cases, but is
at least good enough to run the introductory Sage tutorials, could be
developed.</p>
<p>These issues aside, we got Sage installed on about 65 computers for people
who otherwise had mostly never used Sage, and did not have reliable internet
access (and will be able to share it with their friends and colleagues, as
most of them kept a copy of the installer on their desktop). That alone is
a big success!</p>
Sage Days 100: Geometry and dynamics of surfaces Bonn (Germany), July 22 -- July 27, 20192019-07-22T00:00:00+00:00http://opendreamkit.org/2019/07/22/SageDays100<h2 id="main-goals">Main goals</h2>
<p>The Sage Days 100 workshop gathered a variety of public
(from master student with no prior knowledge
of Sage, to senior lecturer with 10 years of contribution to
Sage source code). The goals were to introduce OpenDreamKit softwares
to beginners, introduce good practice software development for
most advanced users and have programmers present their specialized
packages.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The event was organized by Vincent Delecroix.
The OpenDreamKit member Samuel Leli`evre participated in the event and
provided individual help to participants. The on-site facilities and
logistics were funded by the University of Bonn. OpenDreamKit funded the
travel and lodging of 11 of the participants that came from across
Europe (France, Germany and United Kingdom).</p>
<h2 id="event-summary">Event summary</h2>
<p>The week was organized so that half of the
time was left for people to work on their own project while advanced
users can answer questions. During the tutorial sessions we had
a mix of specialized package presentation, learning development tool
(control version system, testing, debugging, etc). We also had a brief
experience with pair programming where an advanced programmer was
paired with a beginner to solve a challenge.</p>
<h2 id="demographics">Demographics</h2>
<p>Out of approximately 25 participants approximtely
half of them where from Germany and the other half from neighbouring
countries (Switzerland, France, United Kingdom). We had an almost parity
of male and female participants which is rare in software development
workshops in general. The level of studies spread from one sixth master students,
one sixth PhD students, one sixth postdocs and one half of senior participants
(professor or associate professor, retired engineer).</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>One of the primary goal of the workshop
was to introduce Sage to students. I believe that we were successful
given the feedback we obtained from participants. All of the participants
answered to have learn something.</p>
Sage Days 102: Nigeria University of Ibadan (Nigeria), July 15 -- July 19, 20192019-07-15T00:00:00+00:00http://opendreamkit.org/2019/07/15/SageDays102<p><img src="/public/images/events/days102_group.jpg = 75%" alt="" /></p>
<p>Sage Days 102, University of Ibadan, Nigeria</p>
<h2 id="main-goals">Main goals</h2>
<p>The purpose of this event was to introduce students,
lecturers, and reaserchers local to Nigeria, and some of the
neighboring countries, to OpenDreamKit software, particularly Sage and GAP,
as well as to improve their programming skills in general and foster
interest in contributing to open source mathematics software.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Much of the initial animation for the event was
animated by Viviane Pons, though the only OpenDreamKit member to attend was
Erik Madison Bray, whose travel was funded by the project. OpenDreamKit also
funded travel for three other instructors from outside the project, as
well as two PhD students who attended from neighboring countries
outside Nigeria. All on-site facilities and logistics were funded by the
University of Ibadan.</p>
<h2 id="event-summary">Event summary</h2>
<p>Significant time was spent helping people install such
software on their personal machines: A particular challenge in Nigeria
where, as many attendees lacked reliable internet connections, even
downloading the software could pose a challenge. The workshop was
primarily organized as a series of introductory lectures, including on
using the UNIX shell and basic Python programming, then moving on to
more application-specific uses of Sage. This included
topic-specific break-out sessions on numerical analysis, graph
theory, and algebraic combinatorics. There were also successful
break-out sessions on group theory and abstract algebra with GAP, and
statistical analysis with R. Participants also gave short talks on some
of their personal research and computational problems, and
participated in a contest to design interactive widgets for the Jupyter
notebook.</p>
<p>This workshop was a result of the Free Computational Mathematics conference at
CIRM earlier the same year, which had three visitors from the University
of Ibadan Department of Mathematics, and wanted to repeat the success of
that workshop at their home institution.</p>
<h2 id="demographics">Demographics</h2>
<p>Out of approximately 80 participants, most came from the
University of Ibadan as well as a few other institutions in Nigeria,
mostly in the southern and south-eastern regions of the country. Two
students came from Ghana, as well as one from Congo. One instructor
(originally from the United States) came from France, one (originally
from Ghana) came from Austria, and one (originally from Benin) came
from South Africa. About one third of the attendees were women, and
there was a roughly equal mix of PhD students, postdocs, graduate
students, and a smaller number of professors an undergraduate students.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>A full report on the impact of this
workshop can be read on our website:
\centerline{https://opendreamkit.org/2019/07/29/SageDays102/}</p>
<p>As there are relatiely few opportunities for many students in Nigeria to travel
outside the country, as there are few opportunities to meet with
outside lecturers, it’s difficult to meet the demand for such
opportunities. Thus one of the primary goals of this workshop was to
disseminate the software itself, and to give a strong-enough
introduction to it that attendees could in turn introduce their peers
who were not able to attend, and to feel tapped into a broader
international community of researchers interested in computational
mathematics. To that end I believe we were successful, with 80
attendees rating their knowledge of Sage at 1 out of 5 before the
workshop, and over 90
workshop. 83
been given the resources to continue to learning on their own, and to
find help with Sage in the future, as well as that the workshop was
helpful for them to build opportunities for collaboration with their
peers.</p>
<p>Also, recognizing that one workshop was not enough to reach the demand for such
training, a group of the attendees spent the week incorporating an
organization (the strategy of which is still being developed) to build
local sustainability for this kind of training in the future, without
relying as much on (unfortunately rare) external support like that provided
by OpenDreamKit.</p>
CICM 2019: Workshop on Large Mathematics Libraries Prague, Czech Republic, 08th-12th of July 20192019-07-10T00:00:00+00:00http://opendreamkit.org/2019/07/10/CICM-LML<h2 id="main-goals">Main goals</h2>
<p>Large formal and semiformal mathematics libraries are needed to support mathematics research, mathematics education, rigorous software development, and formal proof development.
This workshop explored methods for designing, constructing, and maintaining large mathematics libraries as well as for finding, comparing, and applying the knowledge residing in these libraries.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The workshop was co-organised by Dennis M"uller (FAU).
Florian Rabe (FAU) was a member of the programme committee.
Michael Kohlhase (FAU) gave a talk presenting OpenDreamKit results.</p>
<h2 id="event-summary">Event summary</h2>
<p>The workshop featured two invited talks by Makarius Wenzel and Claudio Sacerdoti Coen on making large libraries (Isabelle resp. Coq) available for system integration.
It also included contributed talks on Logipedia by the Dedukti group, which closely collaborates with OpenDreamKit members, and mathematical datasets.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The main result of the workshop was to strengthen the awareness of and to support the community for managing large formal libraries.
This has been a central topic in WP 6 of OpenDreamKit, and the discussions allowed presenting OpenDreamKit results.</p>
<p>In particular, the talks be Wenzel and Sacerdoti Coen presented results that were developed in collaboration with OpenDreamKit members in WP 6.</p>
Sage Days 105 Free and Practical Software for Algebraic Combinatorics University of Ljubljana, Slovenia, 8th-12th of July 20192019-07-08T00:00:00+00:00http://opendreamkit.org/2019/07/08/SageDays105<h2 id="main-goals">Main goals</h2>
<p>This workshop was organized as satellite event to
the main yearly international conference on algebraic combinatorics
FPSAC (Formal Power Series and Algebraic Combinatorics). It aimed at
gathering researchers from this community interested in computation
for training, community building, and coding sprints.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Nicolas M. Thiéry was one of the organizers.
There was no registration fee. OpenDreamKit funded meals and nights taken by the participants
at the local Youth Hostel (Dijaski dom Vic, Gerbičeva ulica 51a, Ljubljana).</p>
<h2 id="event-summary">Event summary</h2>
<p>The workshop featured a total of 16 tutorial
sessions, 8 demos and 6 presentations, including the following
presentations by OpenDreamKit participants: <code class="language-plaintext highlighter-rouge">Best practice for computer
exploration'',</code>Live online notebooks with Binder’’, <code class="language-plaintext highlighter-rouge">Object
oriented programming in Sage'',</code>Authoring SageMath packages’’,
<code class="language-plaintext highlighter-rouge">free/libre software is good and what you can do about it'',
</code>Sage-Combinat-Widgets, Francy’’.</p>
<p>The first plenary talk was about inspiring the next generation,
notably women, about the impact using and developing computational
software can have on conducting one’s research in algebraic
combinatorics. The main purpose of the remaining plenary talks was to
pave the path for the implementation of new features in SageMath and
related software, to ignite, inspire and fuel brainstorms and coding
sprints with other participants. Training took the form of short demos
to give an overview of the OpenDreamKit toolkit, tutorials that were
presented to point participants to resources to explore according to
their pace and taste, with continuous support from instructors, and
guided tutorials/longer presentations in separate rooms. Participants
were encouraged to skip the parts that were irrelevant to them (e.g.
tutorials on material they already master) to engage into parallel
collaborative activities such as coding sprints.</p>
<h2 id="demographics">Demographics</h2>
<p>Out of 56 participants, 19 were PhD students
and 37 researchers or professors, 15 from european countries
(Germany, France, Slovenia, Portugal, Netherlands, Austria, Iceland), 21 from North America (US, the Carribbean, Canada)
1 from Australia and 19 from all over the world (Israel, Korea, Turkey, India, China, Nigeria, Chile).
We had 15 female participants to this workshop. Although still far
from satisfactory with respect to the last four years
this gender gap in the software development area tends to shrink, with
several bright young women taking on lead roles.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>More than one fifth of the FPSAC
participants decided to participate to this event; they actively
engaged into the collaborative atmosphere; a lot of expertise was
shared, not counting new software contributions. One of the
participant wrote back ``Thanks again for a very useful Sage Days. I
was happy that the Dynamics package got finished … Its fantastic how
you’ve shepherded this community over many years now, and the
mathematical community is much better for it.’’. Another decided to
organize a similar satellite event to FPSAC’20.</p>
CICM 2019: Conference on Intelligent Computer Mathematics Prague, Czech Republic, 08th-12th of July 20192019-07-08T00:00:00+00:00http://opendreamkit.org/2019/07/08/CICM<h2 id="main-goals">Main goals</h2>
<p>Digital and computational solutions are becoming the prevalent means for the generation, communication, processing, storage and curation of mathematical information.
Separate communities have developed to investigate and build computer based systems for computer algebra, automated deduction, and mathematical publishing as well as novel user interfaces.
While all of these systems excel in their own right, their integration can lead to synergies offering significant added value.
The Conference on Intelligent Computer Mathematics (CICM) offers a venue for discussing and developing solutions to the great challenges posed by the integration of these diverse areas.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Florian Rabe (FAU) was a member of the steering committee.
A workshop was co-organized by Dennis M"uller (FAU).
There were no costs to OpenDreamKit other than travel costs for OpenDreamKit members.</p>
<h2 id="event-summary">Event summary</h2>
<p>CICM featured 3 invited talks, 5 days of presentations, a doctoral program, 3 workshops, and 1 tutorial.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>OpenDreamKit members presented about 10 talks at the conference, various workshops, and the doctoral programme, describing various OpenDreamKit results.
This triggered a number of discussions with researchers from adjacent fields in computer science as well as a few mathematicians.</p>
<p>Florian Rabe and Yasmine Sharoda (who visited FAU for one month during OpenDreamKit) were awarded the Best Paper Award for his paper on diagram operators — a new method for organizing large libraries of mathematics that was motivated by OpenDreamKit.</p>
<p>Sylvain Corlay (QuantStack) gave an invited talk on ``Jupyter: From IPython to the Lingua Franca of Scientific Computing’’.
Jupyter being central to OpenDreamKit, this talk mentioned various technologies related to OpenDreamKit results.</p>
<p>Makarius Wenzel gave an invited talk on ``Interaction with Formal Mathematical Documents in Isabelle/PIDE’’, in which he also presented some of the technologies for which he was subcontracted by OpenDreamKit.</p>
FPSAC 2019: International Conference on Formal Power Series and Algebraic Combinatorics Ljubljana (SI), 2019-07-01 to 2019-07-052019-07-01T00:00:00+00:00http://opendreamkit.org/2019/07/01/FPSAC<p>OpenDreamKit participants: Nicolas M. Thi'ery, Katja Ber\v{c}i\v{c}, Viviane
Pons.</p>
<h2 id="event-summary">Event summary</h2>
<p><a href="http://fpsac.org">FPSAC</a> – Formal Power
Series and Algebraic Combinatorics – is the major yearly
international conference on Algebraic; topics of presentations covered
all aspects of combinatorics and their relations with other parts of
mathematics, physics, computer science, and biology. This year’s
edition took place in Ljubljana, Slovenia, from July 1st to 5th. About
250 mathematicians attended the event. One fourth of them also
attended the computational workshop Sage Days 105 organized on the
following week by OpenDreamKit.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>N Thiéry is member of the permanent
committee of the conference since three years and has been leading an
effort to increase the strength of software demonstrations, as a way
to advertise and give back credit to the software development efforts
of its members. Indeed, this mathematical community has a long
tradition of using computer experimentation which plays a fundamental
role in the research. N. Thiéry chaired the software demonstration
sessions which held a record of six presentations, instead of a
handful for the previous years combined.</p>
<p>One presentation was delivered by K. Ber\v{c}i\v{c}, and presented an
early version of a website generator for mathematical databases, an
integral component of the \textsf{data.math-hub.info} system
prototype (reported on in D6.10). All the materials for K.
Ber\v{c}i\v{c}’s software demonstration are available online:</p>
<ul>
<li>slides http://fpsac2019.fmf.uni-lj.si/resources/Slides/202slides.pdf and</li>
<li>extended abstract http://fpsac2019.fmf.uni-lj.si/resources/Proceedings/202.pdf</li>
</ul>
<p>OpenDreamKit funded the participation of Nicolas M. Thiéry, K. Ber\v{c}i\v{c}
to the event (about 1100 Euro). Coordinated efforts were undertaken
with the FPSAC organizers to encourage joint participation to both
events and optimize the funds used on both sides for supporting young
researchers.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The software demonstrations were well
received by the attendees. More than one half of them were
present at the session, and a quick poll showed that most of them were
using computer experimentation on a regular basis, while a large
fraction was often writing code for their research.</p>
<p>All of the presentations used OpenDreamKit components, and all the
demonstrations about computational software used Sage and were made
available online using the Binder service, with links from the
<a href="http://fpsac2019.fmf.uni-lj.si/schedule/">schedule</a>.</p>
<p>There was a notable interest for the \textsf{data.math-hub.info}
system prototype, which confirmed that there is a real need for such
infrastructure in the combinatorics community.</p>
<p><img src="/public/images/events/FPSAC.jpg =75%" alt="" />
International Conference on Formal Power Series and Algebraic Combinatorics, Ljubljana, Slovenia</p>
Report on second Women In Sage2019-06-28T00:00:00+00:00http://opendreamkit.org/2019/06/28/WomenInSage<p>We organized our second <a href="https://wiki.sagemath.org/days98">Women In Sage event</a> this April in Crete. We were 22 women staying in the village of Archanes working on the mathematical software SageMath.</p>
<h1 id="the-workshop">The Workshop</h1>
<p>The participants arrived to Archanes over the week-end and the event started officially on the Monday morning. Viviane Pons opened the event with a Sage introduction. In the afternoon, we worked on installing Sage on everyone’s machine.</p>
<p>We organized short research presentations on the Tuesday morning before moving to individual and group projects. On Tuesday afternoon, Viviane Pons gave a presentation of Sage contribution, version control and good practice for sharing code. We started a “contributor group” whose goal was to get a development version of Sage and start their first contribution to the software.</p>
<p>In the rest of week, we worked both in group and individually on our different projects involving Sage. You can read our final status reports on our <a href="https://wiki.sagemath.org/days98">webpage</a>.</p>
<p>The event also included a visit of Archanes archaeological site Fourni and a social dinner at a local restaurant.</p>
<p><img src="/public/images/womenInSage2/dinner.jpg" alt="Group picture at the restaurant" class="img-responsive" /></p>
<h1 id="impact">Impact</h1>
<p><em>The data presented here come from a post-event questionnaire sent to the participants.</em></p>
<p>This event followed our previous <a href="https://opendreamkit.org/2017/04/06/WomenInSage/">OpenDreamKit Women in Sage</a>. The co-organizer Eleni Tzanaki was a participant of the first event. We also had 3 participants of the first event who also attended this one.</p>
<p>Our participants came from 8 different countries (France, Belgium, Germany, Greece, UK, US, Romania and Peru) and 15 of them were PhD or Master students. We were supposed to welcome three participants from Nigeria but they were sadly not able to obtain their visas in time.</p>
<p>As before, the goal was to motivate women to participate and improve their self confidence regarding coding. Even though advanced developers were welcome, we tried to make the event welcoming for new coders and mathematician women without any experience in Sage.</p>
<p>The women who attended the conference had various level of programming experience
ranging from 1 (no experience) to 5 (a lot of experience).</p>
<p><img src="/public/images/womenInSage2/programming-knowledge.png" alt="1: 1 people, 2: 5 people, 3: 8 people , 4: 2 people, 5: 2 people" class="img-responsive" /></p>
<p>This disparity also reflected in their knowledge of Sage.</p>
<p><img src="/public/images/womenInSage2/sage-knowledge.png" alt="1: 4 people, 2: 4 people, 3: 4 people , 4: 4 people, 5: 2 people" class="img-responsive" /></p>
<p>As for contributions, only 3 participants had contributed to Sage in the past . Also, a majority of participants had never attended
a Sage Days before. Actually, 9 of them had never even heard of Sage Days and 3
of them said they did not think it was “for them”.</p>
<p><img src="/public/images/womenInSage2/sage-days-attendance.png" alt=""I had never heard of Sage Days": 50%, "I thought Sage Days were not for me": 16.7%, "I had heard of them but never had the time nor the occasion": 0% ,
"I attended once": 16.7%, "I attended many times": 16.7%" class="img-responsive" /></p>
<p>To the question “How did the fact that the event was targeted to women impact your decision to come? (Would you have participated in a classical SageDays)”,
Many participants answered that it was indeed a factor a their decision.</p>
<p><cite>It helped, made me feel more encouraged to apply</cite></p>
<p><cite>I may have participate anyway but the fact that it was only for women made me want to participe even more. </cite></p>
<p><cite>It was my first workshop in Europe so I felt more comfortable working with women first but now I think I can go to any kind of workshop.</cite></p>
<p><cite>It is the reason that I applied to go. Along with the fact that it was open to participants with limited experience, it made me feel that I was a good fit. </cite></p>
<p><cite>I think I also would have participated in a classical SageDays - but the fact that this event was targeted only to women made it more attractive for me. I hoped for a nice(r) atmosphere, and this was definitely the case.</cite></p>
<p>The event helped building up the confidence of the participants, 11 of them said
they felt more confident to attend classical Sage Days after the event.</p>
<p><img src="/public/images/womenInSage2/sage_days_confident.png" alt=""I already felt comfortable attending 'normal' Sage Days": 23.5%, "I feel more comfortable now": 64.7%, "I still don't feel confident enough": 5.9%, "Depends if I make a progress with coding.":5.9% " class="img-responsive" /></p>
<p>We had a working group specifically dedicated to getting started with Sage contribution. As a result, 6 participants got a development version of Sage installed for the first time. We worked on <a href="https://trac.sagemath.org/search?q=days89">5 tickets</a> during the week,
3 of those which have been merged since the conference. All participants said they
had learned new things and it would impact their careers.</p>
<p><img src="/public/images/womenInSage2/learn_new_things.png" alt="Did you learn new things at this conference? 1:0 people, 2: 0 people, 3: 5 people, 4: 4 people, 5: 9 people" class="img-responsive" /></p>
<p><img src="/public/images/womenInSage2/impact.png" alt="Did you think what you learned will have an impact on your future research/career? 1:0 people, 2: 0 people, 3: 4 people, 4: 5 people, 5: 9 people" class="img-responsive" /></p>
<p>This also was an occasion to start projects and form more research and development collaborations for future.</p>
<p><img src="/public/images/womenInSage2/people.png" alt="Did you meet interesting people? 1:0 people, 2: 0 people, 3: 0 people, 4: 0 people, 5: 18 people" class="img-responsive" /></p>
<p><img src="/public/images/womenInSage2/projects.png" alt="Have you started (or carried on) any project that could lead to more research and development in the future? I have started things and will try to follow up: 44.4%. I have started things and will definitely follow up: 38.9%. Nothing: 11.1%. carrying on things that should be follow up: 5.6%" class="img-responsive" /></p>
<p>All of this happened in a very casual and welcoming atmosphere. We used the common
room of the main house to work. We cooked international, vegetarian friendly meals using delicious ingredients from Crete. We took nice walks around the village and had perfect ice-creams. We got to know
each other and shared more than code. All participants agreed that it was a very
positive experience.</p>
<p><img src="/public/images/womenInSage2/atmosphere.png" alt="How did you like the general atmosphere of the conference? 1:0 people, 2: 0 people, 3: 1 people, 4: 0 people, 5: 17 people" class="img-responsive" /></p>
<p><cite>The group was amazing!</cite></p>
<p><cite>There was a great atmosphere at the conference. I never felt intimidated or that I should be worried about my level of knowledge. It was easy to talk with everyone there and to ask/give help. Overall it was very positive and supportive.</cite></p>
<p><cite>This was the first time I didn’t have imposter syndrome since starting my PhD. Everyone was so welcoming and there was an understanding that we were all from different backgrounds and with different levels of experience. It felt very easy to make suggestions and ask questions without the fear of being judged.</cite></p>
Sage Gap Days 101 Cernay-la-ville (FR), 2019-06-17 to 2019-06-212019-06-15T00:00:00+00:00http://opendreamkit.org/2019/06/15/SageGapDays101<h2 id="main-goals">Main goals</h2>
<p>This developer meeting was the third in a row
with a focus on OpenDreamKit tasks related to packaging, portability and
documentation tools for GAP and SageMath.</p>
<h2 id="event-summary">Event summary</h2>
<p>An intensive week of brainstorms and coding
sprints, structured by regular planning and debriefing sessions, and
interspersed with a few presentations:</p>
<ul>
<li>Sage Combinat Widgets and Francy, Sage Explorer by O. Benassy</li>
<li>Semantic aware Sage interface to GAP by Nicolas M. Thiéry</li>
<li>Jupyter Viz by professor Nathan Carter from St Andrews
university,</li>
<li>Cppyy by Julian Rüth, PhD.</li>
<li>RISE by Tomer Bauer, PhD student from Bar-Ilan University, in
Israel</li>
</ul>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>This event was organized and funded by OpenDreamKit
(Paris Sud). Accommodation and meals for all participants were covered
by OpenDreamKit funding sources. OpenDreamKit participants covered their own
travel expenses.</p>
<h2 id="demographic">Demographic</h2>
<p>Eight OpenDreamKit participants (N. Thiéry, V.
Delecroix, F. Rabe, V. Klein, S. Lelièvre, L. de Feo, O. Benassy, E.M.
Bray) from four sites, together with twelve other participants from
nine different institutions</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop was the occasion of a major advance for packaging
SageMath and dependencies with Conda. A first version of SageMath had
been packaged two years ago in good parts by independent conda expert
Isuru Fernando. However the packages had not been updated since due to
a series of hurdles. Thanks to the coming of Isuru and brainstorms
with local experts of SageMath and GAP, many of the hurdles have been
lifted, leading to the packaging of the latest version of SageMath and
paving the way for more regular updates in the future.</p>
<p>In addition, a lot of work was targeted at continuous integration,
Python 3 support, docker packaging and interfaces between systems.</p>
<p>Finally the workshop was the occasion for a joint coding sprint
between OpenDreamKit participants and Nathan Carter, around visualization
(Francy, Jupyter Viz) and live browsing of mathematical data (Sage
Explorer, Group Explorer). This led to the sharing of much vision and
know how, and opened the door for potential future convergences
between the related projects.</p>
Micromagnetism Workshop: Ubermag and mumax$^{3}$ European XFEL, Schenefeld, Germany, 11-14 June 20192019-06-11T00:00:00+00:00http://opendreamkit.org/2019/06/11/XFEL-mumax<h2 id="main-goals">Main goals</h2>
<p>The main goal of this workshop was to develop a
micromagnetic calculator for driving mumax$^{3}$ micromagnetic
simulation tool.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The main implication of this workshop was
that another computational backend can be used (apart from OOMMF) to
run micromagnetic simulations. mumax$^{3}$ has certain advantages over
OOMMF, mostly in terms of its capabilities of running micromagnetic
simulations on GPU. This would enable us to reach a much larger target
audience.</p>
<h2 id="event-summary">Event summary</h2>
<p>The first day of the workshop consisted of
presentations from all its participants in order to familiarise all
participants about the basics of Ubermag and mumax$^{3}$. Another 3
days were focused on coding and the implementation of the mumax$^{3}$
calculator, setting up the repository’s infrastructure, and
discussions of future developments.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>A micromagnetic calculator, based on
mumax$^{3}$, was developed and covers the most important capabilities
of mumax$^{3}$:</p>
<p>\centerline{https://github.com/ubermag/mumax3c}</p>
Nbgrader Hackathon Edinburgh, May 29th-311st2019-05-29T00:00:00+00:00http://opendreamkit.org/2019/05/29/nbgrader<p>## Event summary</p>
<p>Within the Jupyter Community Workshop series
funded by Bloomberg, the University of Edinburgh hosted a three-day
event. Its core aspect was a hackathon focused on adding
improvements, fixes and extra documentation for the course
management tool \texttt{nbgrader}. Alongside this it held an
afternoon of talks highlighting how Jupyter can be used in education
at varying levels.</p>
<p>## OpenDreamKit implication</p>
<p>OpenDreamKit participant Nicolas M. Thiéry was
invited to participate to the hackathon and deliver a talk on
Jupyter for teaching introductory programming.</p>
<p>## Results and impact</p>
<p>The hackathon resulted in many
improvements to nbgrader, including some contributions by Nicolas,
that make it easier to integrate in a variety of environments.</p>
<p>Nicolas used the occasion to visit St Andrews, for brainstorms on
upcoming deliverable reports and collaboration on Sage-GAP
interfacing. This visit also led to the invitation of Nathan Carter
to Sage Days 101 for collaboration on mathematical visualization and
live data browsing in Jupyter.</p>
ICMS Workshop: Big Proof Bayes Centre, Edinburgh, UK, 27th-31st of May 20192019-05-27T00:00:00+00:00http://opendreamkit.org/2019/05/27/BigProof<h2 id="main-goals">Main goals</h2>
<p>The Isaac Newton Institute programme on Big Proof during the summer of 2017 drew a great deal of interest from mathematicians, philosophers, and computer scientists. In May 2019 a week-long follow-up workshop funded by the Isaac Newton Institute will take place at the International Centre for Mathematical Sciences in Edinburgh. This workshop will take forward a number of the key initiatives from Big Proof:</p>
<ul>
<li>Pragmatic foundations for the formalization of mathematical proofs.</li>
<li>Social processes that support research, exposition, and learning in mathematics.</li>
<li>Interchange formats and repositories for formal mathematical knowledge</li>
<li>Scalable proof automation</li>
<li>Applications of Big Proof technology</li>
</ul>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Michael Kohlhase (FAU) and Florian Rabe (FAU) were invited participants. The former also gave an invited talk.</p>
<h2 id="event-summary">Event summary</h2>
<p>The workshop consisted of 5 days of 1 hour talks and ample discussion.
Due to the invite-inly format, a slew of top researchers from computer science, mathematics, and social sciences were present.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The main result of the workshop from the OpenDreamKit perspective was Kohlhase’s presentation of the Math Data initiative developed within OpenDreamKit.
Together with OpenDreamKit’s Big Computation theme, these complement and integrate with the Big Proof topic.</p>
Sage Days 98: Women in Sage Archanes (Greece), April 8 -- April 12, 20192019-04-08T00:00:00+00:00http://opendreamkit.org/2019/04/08/WomenInSage<h2 id="main-goals">Main goals</h2>
<p>The main goal of the event was to initiate more women to the software Sage to reduce the gender gap in mathematics software
development. Each participant had to propose a mathematic development project to be carried out during the week.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The event was initiated by Viviane Pons from OpenDreamKit and co-organized with Eleni Tzanaki (University of Crete). It was funded solely by OpenDreamKit which covered: lodging for the participants (rented houses), food, and transportation for many of the participants.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event was organized as a workshop where every participant could work on their own project to develop their coding skills. We started the week with an introduction to Sage and some tutorials. Then each participant gave a 5 minute talk about their own research. After that, we worked on different projects, organizing status reports every day. In particular, we ran a group specifically to produce new contributions to Sage.</p>
<h2 id="demographics">Demographics</h2>
<p>All participants were women coming from 8 different countries (France, Belgium, Germany, Greece, UK, US, Romania and Peru), as per institutions, and more if we count nationalities (Australia, Lebanon, Spain). About half of them could be considered Sage beginners. We had 1 master student, 14 PhD students, 2 postdocs, and 5 <strong>Maîtresses de conférences</strong> or assistant professor or lecturer.</p>
<p>We were supposed to also welcome 3 women from Nigeria but they were sadly not able to obtain their visas on time.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>A full report on the impact of this
workshop can be read on our website:
https://opendreamkit.org/2019/06/28/WomenInSage/
The main goal was to make the participants more confident into their programming skills and more prone to become Sage contributors and attend classical Sage Days. It was a big success in that regard. Indeed, before the conference, only 17</p>
<p><img src="/public/images/events/group_photo_head.jpeg =50%" alt="" /></p>
<p>Women in Sage, Archanes, Crete</p>
Atelier PARI/GP 2019b Roma (IT), 2018-04-09 to 2018-04-102019-04-04T00:00:00+00:00http://opendreamkit.org/2019/04/04/Atelier-PARI-GP<h2 id="main-goals">Main goals</h2>
<p>This was a teaching and dissemination meeting, by
invitation from the Roman Number Theory Association as a satellite
event for their 5th mini-symposium.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: B. Allombert and A. Page from
Bordeaux.</p>
<p>OpenDreamKit funded travel and accomodation costs for B. Allombert for about
1.5k Euro. The ALGANT consortium, LIA LYSM (CNRS) and University Roma Tre
co-funded the event.</p>
<h2 id="event-summary">Event summary</h2>
<p>This Atelier PARI/GP took place in Roma (Italy) on
April 9th and 10th. It was followed by a 3-day international research
conference on Number Theory. There were 40 participants for the
Atelier.</p>
<p>The 2-day Atelier followed the same pattern as the previous Roma Atelier
in 2018,
featuring a general introduction to PARI/GP and two
specialized courses (graduate level) in the mornings:</p>
<ul>
<li>Bill Allombert ``Finite fields’’,</li>
<li>Aurel Page ``Algebraic number theory’’.</li>
</ul>
<p>Afternoons were devoted to practice and problem sessions.</p>
<p>Slides for all talks are available at
http://pari.math.u-bordeaux.fr/Events/PARI2019b/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a successful teaching and dissemination
event, with positive feedback from the participants and organizers.</p>
<p>\begin{figure}[ht]
\includegraphics[width=.75\textwidth]{pari2019b.jpg}
\caption*{Atelier PARI/GP, 5th Roman Number Theory Association mini-symposium, Rome}
\end{figure}</p>
GAP Days Spring 2019 Martin Luther University of Halle-Wittenberg (Germany), 18--22 Mar 20192019-03-18T00:00:00+00:00http://opendreamkit.org/2019/03/18/GAPDaysSpring2019<h2 id="main-goals">Main goals</h2>
<p>This was a meeting for experienced GAP developers and users
to discuss and contribute to the GAP project. The overarching theme was
algorithms for permutation groups.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Michael Torpey and Markus Pfeiffer attended, and
worked on software while learning more about GAP and permutation groups.</p>
<p>OpenDreamKit participants: M.~Torpey and M.~Pfeiffer</p>
<h2 id="event-summary">Event summary</h2>
<p>This was a meeting for experienced developers and users
of GAP to discuss and influence the future development of GAP. There was also
an overarching theme of algorithms for permutation groups, with discussions and
exercises relating to stabiliser chains and the Schreier–Sims algorithm.
There were also various coding sprints for the GAP system and various GAP
packages.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>M.~Torpey learned more about the core GAP system
and permutation group algorithms. He also made progress with work on
PackageManager, which he was able to advertise to others at the meeting.</p>
SageMath classes in Crete Heraklion, March. 4-6, 20192019-03-04T00:00:00+00:00http://opendreamkit.org/2019/03/04/heraklion<h2 id="main-goals">Main goals</h2>
<p>We organized a series of SageMath classes in the mathematical department of the Univeristy of Crete.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit funded the trip of Viviane Pons to give the classes.</p>
<h2 id="event-summary">Event summary</h2>
<p>A series of three classes was organized throughout the week. This was open both to students (from undergrad to PhD) and researchers of the math department of the Univeristy of Crete. The classes consisted of SageMath tutorials to allow the attendees to discover the software at their own pace. We used Jupyter notebooks with a focus on math related problems.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The students enjoyed the tutorials a lot. It was actually used as a starting point for more regular sessions tutored by their local teacher Eleni Tzanaki. We are now confident that SageMath will be taught at Univeristy of Crete as part of the math program. The meeting was also an occasion to prepare the up-coming Women in Sage event also organized in Crete.</p>
CIRM Conference: Cohomology of arithmetic groups, lattice and number theory Luminy (FR), 2019-03-24 to 2019-03-292019-03-03T00:00:00+00:00http://opendreamkit.org/2019/03/03/CIRM<h2 id="main-goals">Main goals</h2>
<p>This was a research conference on the cohomology of
arithmetic groups, with a focus on computational techniques.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Bill Allombert was invited to give a 1-hour
introduction to PARI/GP for a software session during the conference.
He gave a tutorial on the manipulation of lattices, $L$-functions,
modular forms and curves of small genus in the system.</p>
<p>OpenDreamKit participants: B. Allombert from Bordeaux.</p>
<p>OpenDreamKit funded the participation of B. Allombert to the event (about 600 Euro).</p>
<h2 id="event-summary">Event summary</h2>
<p>This conference took place in Luminy (France)
from March 24th to 29th, with the participation of about 70 mathematicians.</p>
<p>Slides for the PARI/GP presentation are available at
http://pari.math.u-bordeaux.fr/Events/CIRM2019/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a successful teaching and dissemination
event towards a community (arithmetic geometry, representation theory)
for which computer-aided calculations is less natural than in other
areas of mathematics: the talk was well-received with interesting
feedback.</p>
Free Computational Mathematics2019-02-11T00:00:00+00:00http://opendreamkit.org/2019/02/11/FreeComputationalMathematicsConference<p>This community building and training conference aims to bring
together users and developers of (open source) (pure) mathematics
software, including GAP, Linbox, MPIR, Pari/GP, SageMath, or Singular.
Following a long trend of highly productive workshops within the various communities (e.g. the Sage-Days series), this conference will consist of keynote talks and hands on tutorials with a focus on experimental research and computational and development best practices; and plenty of free time for interactions and collaborative work.</p>
<p>This conference is organized by the H2020 European project <a href="http://opendreamkit.org">OpenDreamKit</a>, of which
it will be the main public closing event.</p>
<h2 id="keynote-speakers">Keynote speakers</h2>
<ul>
<li><a href="https://irmar.univ-rennes1.fr/en/interlocutors/marie-francoise-roy">Marie-Françoise Roy</a> (Université de Rennes 1) A historical perspective on contributions of researchers and teachers to (open source) mathematical software</li>
<li><a href="https://www.quendi.de/en/math.html">Max Horn</a> (Justus-Liebig-Universität Gießen) Collaborative software development in a large system (GAP)</li>
<li><a href="http://fperez.org/">Fernando Perez</a> (Berkeley Institute for Data Science, California) Scientific computing, user interfaces, and community building</li>
<li><a href="https://www.math.ucdavis.edu/~anne/">Anne Schilling</a> (UC Davis California) Impact of computer assisted experimentation in combinatorics</li>
<li><a href="http://www.texmacs.org/joris/main/joris.html">Joris Van Der Hoeven</a> (Ecole Polytechnique) General purpose mathematical software design</li>
</ul>
<h2 id="preregistration"><a href="https://www.cirm-math.fr/preRegistration/index.php?EX=menu0&id_renc=1978">Preregistration</a></h2>
<p><a href="https://www.cirm-math.fr/preRegistration/index.php?EX=menu0&id_renc=1978">Preregistration</a>
officially ended on January 7th. However, we may have
some additional spots, which we will fill on a first come first
serve basis. If you are insterested, please keep preregistering.</p>
<h2 id="funding">Funding</h2>
<p>OpenDreamKit will cover the local expenses for all participants and
the travel expenses of some participants, with priority given to
students and participants from developing countries. You can apply for
such funding upon preregistration; CV and similar information will
help us make an informed decision.</p>
<h2 id="schedule"><a href="https://hackmd.io/zkXWN_cwTjy46dyBSJKaug?view#">Schedule</a></h2>
<h2 id="additional-information-on-cirms-web-page">Additional information on <a href="https://conferences.cirm-math.fr/1978.html">CIRM’s web page</a></h2>
CIRM Conference: Free Computational Mathematics CIRM, Luminy, France, 11--15 Feb 20192019-02-11T00:00:00+00:00http://opendreamkit.org/2019/02/11/CIRM-FreeComputationalMathematics<h2 id="main-goals">Main goals</h2>
<p><strong>Free Computational Mathematics</strong> was
a community building and training conference for a wide mathematical audience:
it was one of the main dissemination events for OpenDreamKit and our public closing event.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>This conference was organized jointly by
OpenDreamKit’s sites PS and UG, under the lead of PS.
The cost was about 60\,000\, Euro, covered by UK.</p>
<h2 id="event-summary">Event summary</h2>
<p>We brought together users and developers of most OpenDreamKit software components
(GAP, Jupyter, \Linbox, \MPIR, \PariGP, Sage, Singular).
The workshop followed a long trend of highly productive workshops that we organised in the past,
consisting of keynote talks and hands-on tutorials with a focus on experimental research
and computational and development best practices.
There was plenty of time for interactions and collaborative work.
Part of the schedule was intentionally made up during the workshop itself,
taking input from the participants on what trainings they wanted to see.</p>
<p>Do you have demographic information? If so, please share!</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>On the dissemination side, many
participants wished for the training to extend for more than five
days, and for similar events to be organized at their location. In
particular our three participants from Nigeria went on to organize a
similar event at the University of Ibadan, with support and trainers
from OpenDreamKit (See Sage Days 102 below). On the community building
side, this was the first time that such a wide mix of developers from
PARI, Singular, GAP, LMFDB, SageMath, Oscar, and Jupyter were gathered
together and closely collaborating, which was invaluable for long term
community building.</p>
<p><img src="/public/images/events/CIRM.jpg =75%" alt="" /></p>
<p>Free Computational Mathematics, CIRM, Luminy, France</p>
JNCF 2019: Journées Nationales de Calcul Formel Luminy (FR), 2019-02-04 to 2019-02-082019-02-04T00:00:00+00:00http://opendreamkit.org/2019/02/04/JNCF<h2 id="main-goals">Main goals</h2>
<p>This is the yearly meeting of the French community in
Computer Algebra, open to an international audience, with lectures and
contributed talks given mostly in English.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Luca De Feo co-organized the JNCF 2019. The
organization of the workshop was coordinated with the organization of
the ``Free Computational Mathematics’’ conference, taking place the
following week in the same venue, to encourage cross-participation.</p>
<p>Luca De Feo and Clément Pernet gave presentations on Monday on the
achievements of OpenDreamKit relevant to the Computer Algebra
community.</p>
<p>OpenDreamKit participants: Luca De Feo from University of Versailles,
Jean-Guillaume Dumas and Clément Pernet from University Grenobles
Alpes.</p>
<p>OpenDreamKit sponsored the event (3000 euros</p>
<h2 id="event-summary">Event summary</h2>
<p>This conference took place in Luminy (France) from
February 4th to 8th. About 58 mathematicians and computer scientists
participated to the event. Three of the participants also participated
in the ``Free Computational Mathematics’’ conference the following
week, while four other noted members of the French community in
Computer Algebra, Joris van der Hoeven, Marie Françoise Roy, Fredrik
Johansson and Bill Allombert, elected to only participate in the latter
due to time constraints.</p>
<p>Slides for Luca De Feo and Clément Pernet’s presentations are
available online from the conference web page:
http://www.jncf2019.uvsq.fr/edt.html.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>Computer Algebra is the birth place of
computer-aided Mathematics, and all of OpenDreamKit software owe something to
the field. Over the years, OpenDreamKit has constantly used the JNCF as an
occasion to disseminate its achievements relevant to the community,
through contributed talks.</p>
<p>The last edition of JNCF to happen concomitantly with the project was
the occasion to double the dissemination efforts, and organize the
JNCF in tandem with the main OpenDreamKit conference <code class="language-plaintext highlighter-rouge">Free Computational
Mathematics'': two invited talks at the</code>Free Computational
Mathematics’’ were delivered by noted members of the French Computer
Algebra community, while one of the invited lectures at JNCF was
delivered by Mohamed Barakat, a GAP developer and founder of the OSCAR
consortium, close to the OpenDreamKit community. A full session was devoted to
mathematical software on Monday afternoon, with two presentations
given by OpenDreamKit members, and one by a MapleSoft executive, which
triggered visible interest in the audience.</p>
<p>This was a successful workshop, and a great occasion to deliver
successful developments of OpenDreamKit to its user base.</p>
Atelier PARI/GP 2019 Bordeaux (FR), 2019-01-14 to 2019-01-182019-01-30T00:00:00+00:00http://opendreamkit.org/2019/01/30/Atelier-PARI-GP<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: B. Allombert, K. Belabas, J.
Cremona, V. Delecroix, J. Demeyer, L. de Feo.</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and some travel expenses), for about 13k Euro.</p>
<h2 id="event-summary">Event summary</h2>
<p>The 12th Atelier PARI/GP took place in Bordeaux
(France) from january 14h to 18th.</p>
<p>There were 57 registered participants from 31 different institutions
(no registration fees).</p>
<p>A typical day of the workshop had introductory talks and tutorials
in the morning; afternoons allowed ample time for hacking sessions,
discussions and training.</p>
<p>The Atelier featured 10 morning talks on mathematical topics and
implementation projects including 4 talks by OpenDreamKit members</p>
<ul>
<li>Bill Allombert <code class="language-plaintext highlighter-rouge">New GP features'' and</code>Parallel GP programming’’.</li>
<li>John Cremona ``Computing classical modular forms for the LMFDB’’.</li>
<li>Jeroen Demeyer ``\texttt{cypari2}: Python bindings for PARI/GP’’.</li>
</ul>
<p>Slides for all talks are available at
http://pari.math.u-bordeaux.fr/Events/PARI2019/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop was productive and a successful
teaching and dissemination event; 12 participants came from developping
countries (Algeria, Djibouti, Lebanon, Morroco, Senegal, Tunisia,
Turkey).</p>
<p>It provided final feedback on recent PARI/GP developments that paved the way
for the release of \texttt{pari-2.12-alpha} (2019/06).</p>
<p>\begin{figure}[ht]
\includegraphics[width=.75\textwidth]{pari2019.jpg}
\caption*{PARI/GP Atelier 2019, Bordeaux, France}
\end{figure}</p>
ODK continuation Proposal: FAIRmat (FAIR Mathematical Data for the European Open Science Cloud)2019-01-29T00:00:00+00:00http://opendreamkit.org/2019/01/29/FAIRmat<p>A subgroup of the OpenDreamKit consortium centered in WP6 has just submitted a proposal in
the European research infrastructures topic: Implementing the European Open Science Cloud:
<strong>FAIR Mathematical Data for the European Open Science Cloud</strong> (FAIRmat).</p>
<p>The problem this proposal wants to solve is that the <a href="https://www.eosc-portal.eu/">European Science Cloud</a> currently under-represents mathematical data and data-based services.</p>
<p>The proposal builds on the <a href="https://www.go-fair.org/fair-principles/">FAIR principles</a> (Findability, Accessiblity, Interoperability, and Reusability), and tries to realize them for mathematical data.
To make FAIR meaningful for mathematical data, we need to represent the meaning of all three data categories (<strong>deep FAIR</strong>):</p>
<ul>
<li>concrete data (i.e. mathematical knowledge represented as record or array data)</li>
<li>symbolic data (i.e. mathematical knowledge represented as terms)</li>
<li>linked data (aka. Metadata)
The aim of the FAIRmat proposal is to standardize mathematical data representations, scale existing mathematical services and make them interoperable, and integrate them into the <a href="https://www.eosc-portal.eu/">European Science Cloud</a> for sustainability.</li>
</ul>
<p>The FAIRmat consortium consists of the following partners:</p>
<table>
<thead>
<tr>
<th>#</th>
<th>Participant organisation name</th>
<th>Short name</th>
<th>Country</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>Friedrich-Alexander Universitaet Erlangen/Nuernberg (coordinator)</td>
<td>FAU</td>
<td>DE</td>
</tr>
<tr>
<td>2</td>
<td>Universite Paris-Sud</td>
<td>UPSud</td>
<td>FR</td>
</tr>
<tr>
<td>3</td>
<td>Chalmers University of Technology</td>
<td>CHA</td>
<td>SE</td>
</tr>
<tr>
<td>4</td>
<td>Univerza v Ljubljani</td>
<td>UL</td>
<td>SI</td>
</tr>
<tr>
<td>5</td>
<td>CAE Tech Limited</td>
<td>CAE</td>
<td>UK</td>
</tr>
<tr>
<td>6</td>
<td>FIZ Karlsruhe – Leibniz Institute for Information Infrastructure</td>
<td>FIZ</td>
<td>DE</td>
</tr>
<tr>
<td>7</td>
<td>European Mathematical Society</td>
<td>EMS</td>
<td>FI</td>
</tr>
</tbody>
</table>
<p>Like the OpenDreamKit Proposal Process, the <a href="https://gl.kwarc.info/mathhub/data-proposal/raw/master/Proposal/final.pdf?inline=false">FAIRmat proposal</a> was developed on a <a href="https://gl.kwarc.info/mathhub/data-proposal">public repository</a>.</p>
<p>We include the abstract for convenience below.</p>
<p>The scientific community increasingly considers datasets as their own kind of resource that should be shared and published individually, either in conjunction with a traditional paper or as a standalone digital artefact. Recently this trend has formed a positive feedback loop with the rise of deep learning methods, which require large datasets as input. Multiple national and international Open Science initiatives have been started to ensure the open availability, easy sharing, and reliable reproducibility of datasets in particular and data-driven research in general. The FAIR principles in particular have been developed as a goalpost for the openness of datasets, including in particular the sharing across disciplines and between research and industry.</p>
<p>As a rule, mathematicians strongly support the Open Science movement and happily make their datasets public for both practical and ethical reasons. This is accompanied by a vibrant and growing community of Open Source software for computational mathematics. However, the systematic FAIR sharing of mathematical data is very difficult due to the inherent complexity of the data. Therefore, today most mathematical data collections are shared in an ad hoc manner that is limited in scope and suffers from a lack of interlinking of digital artefacts across platforms. Thus, FAIR mathematics, while widely welcomed, is effectively non-existent today. A similar argument applies to related sciences to the extent that they make heavy use of mathematical data, e.g., the mathematical modeling of cyber-physical systems.</p>
<p>Generally, reusing shared data requires that the reuser be able to understand the semantics of the data. This is particularly difficult for system interoperability where the semantics must not only be evident but must itself be accessible for automated processing, and it is particularly critical where data is used in safety-critical systems. While this problem exists for all data, it is particularly challenging for mathematical and similar data in related disciplines where the semantics is very difficult to specify. Therefore, today there are virtually no mathematical datasets whose semantics is itself accessible.</p>
<p>This is evidenced by the wide gap in the service offerings of the EOSC when it comes to semantics-aware services in general and services for mathematical data in particular.</p>
<p>FAIRMat (pronounced “Fermat”) will deliver a framework and prototype service for the FAIR and semantics- aware sharing of mathematical data. It will meet the needs of mathematics research and education and will also bring added value to other disciplines and industry that work with data that exhibits complex structure or semantics. It will support all phases of the research life-cycle including the generation, publication, updating, extending, curation, search, reuse, and archival of mathematical data.</p>
<p>FAIRMat will be based on a sustainable software ecosystem including Open Source databases and services as well as nationally funded infrastructures. This includes TRL 6 and above technologies like the Modelica modeling language, the LMFDB database, the SageMath computer algebra suite, or the zbMATH publication information system.</p>
<p>Besides supporting the direct sharing of newly-produced data, FAIRMat will make existing datasets available in a uniform way. In particular, we will demonstrate the scalability of the FAIRMat infrastructure by integrating it with several large and important databases and services from diverse communities including databases of pure mathematical objects (graphs, integer sequences, elliptic curves, . . . ), formalized theorems and proofs, mathematical models used in engineering, linked data from Wikidata, and publication metadata. Finally, to maximize its long-term impact, FAIRMat is designed to culminate in the ISO standardization of its data representation format and the smooth integration of its services into the EOSC Hub at the end of the project lifetime.</p>
<p>FAIRMat will be carried out by a consortium of 7 sites with huge experience in designing and maintaining mathematical datasets and services. The majority of partners are long time Open Science promoters, with a strong experience in large open (software) project management. In particular, all produced documents (including this proposal itself), software, and data will be licensed under open licenses and freely available.</p>
<!-- LocalWords: FAIRmat Findability Accessiblity Universitaet Nuernberg Universite Univerza Ljubljani reuser Modelica zbMATH formalized standardization
-->
Logipedia - an OpenDreamKit for Theorem Proving?2019-01-24T00:00:00+00:00http://opendreamkit.org/2019/01/24/logipedia<p>Theorem proving is the sibling computation:
Both share the representation of mathematical objects.
But while computation aims at solving problems quickly and at large scale, theorem proving makes sure the solutions are correct.</p>
<p>Computation and theorem proving are critical for automating and scaling up STEM in the 21st century — but there are two problems.
Firstly, computation and theorem proving are mostly done separately in different communities and using different approaches and systems.
That’s because both problems are so hard by themselves already that solving them together is even harder.
And secondly, both areas struggle to acquire funding for maintaining and developing their systems: most rewards are given to novel applications while the daily work of fixing bugs and making releases is often overlooked.</p>
<p>OpenDreamKit has generated a lot of attention because it managed to solve the second problem for computation (at least for 4 years).
Now researchers in theorem proving have started similar efforts.
This week Gilles Dowek hosted a meeting in Paris of leading European theorem proving researchers.
Florian Rabe participated as a representative of the Erlangen-Nuremberg site (led by Michael Kohlhase) and of OpenDreamKit as a whole.</p>
<p>The goal is to launch <a href="http://logipedia.inria.fr/">Logipedia</a> as a central system for integrating theorem prover libraries.
An EU infrastructure project akin to OpenDreamKit may be started in the future.</p>
<p>Kohlhase and Rabe have collaborated with Dowek’s group for many years, including throughout OpenDreamKit.
They have held multiple workshops together and have recently submitted a joint French (ANR)/German (DFG) grant proposal of verifying deduction system results.</p>
<p>Many ideas of Logipedia are related to or inspired by innovations of OpenDreamKit.
In particular, the general Math-in-the-Middle approach developed OpenDreamKit and the use of
alignments for aligning corresponding (logical) concepts has been adopted in Logipedia.</p>
<p>Future collaboration between these projects will be critical to make progress on the first problem mentioned above - an integration of OpenDreamKit’s Math-in-the-Middle infrastructure with Logipedia is an exciting way towards integrating theorem proving and computation systems.</p>
<!-- LocalWords: Logipedia Dowek
-->
On the making of OpenDreamKit2018-12-14T00:00:00+00:00http://opendreamkit.org/2018/12/14/eu-proposal<section data-markdown="" data-separator="^---\n" data-separator-vertical="^----\n">
<script type="text/template">
## Plan
1. OpenDreamKit?
1. The making of: brief history (from coordinator's perspective)
1. The making of: approach
1. To run or not to run for a EU project?
1. Good luck and more reading
Note:
This is a talk I delivered at a meeting organized by platform Wiskunde
Netherlands, NWO, and Yellow Research to encourage Dutch
mathematicians to apply for EU funding.
---
## OpenDreamKit (2015-2019)
<center>
Open Digital Research Environment Toolkit<br>
for the Advancement of Mathematics<br>
[OpenDreamKit.org](OpenDreamKit.org)
</center>
**Work Programme**: [Horizon 2020](https://ec.europa.eu/programmes/horizon2020/), [European Research Infrastructures](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures)
**Call**: Virtual Research Environments
**Budget**: 7.6 M€
**Aim**: support the ecosystem of open source software for **pure maths and applications**
- Software engineering: portability, interoperability, distribution, HPC, ...
- Jupyter-based interactive environments
**Consortium**: [15 sites, 50 participants](http://opendreamkit.org/partners), ... together with the international community!
---
## The making of: for me it all started there
<center><img src="/public/images/haut2.jpg" alt="just a nice picture" width="60%"/></center>
Note:
- Since 20 years, I am doing research in algebraic combinatorics. And
like about everybody here, computer exploration is a fundamental tool.
- I am also am a big fan of free software, both from the ethics and practical
points of view
- From the beginning a big fan of software packages like GAP, Singular, Pari
demonstrated that, with a few dedicated leaders, whole communities
of researchers could get together, and share their work, and build
the tools they need.
- I wanted the same in my community. One constraint is that we
need a platform that integrates together tools from many different
areas of maths. So I ended up being a happy Sage developer.
- For a long time, to keep being happy, I strived hard to not
depend on funding so that I could focus on coding and community
building. But with time it became tricky.
----
## What really got me started
<p style="margin-bottom:15%"></p>
**A question of Bruce Westbury at [FPSAC 2013](http://fpsac.org) in Paris:**
<center>
*Given unlimited funding, what would you do with it for Sage?*
</center>
----
## Timeline
**Before 2014:**
Basic idea: seek funding for
- A couple full time developers
- Community building
Several unlucky attempts: ANR, ...
<p style="margin-bottom:2ex;"></p>
**Early 2014:** Our EU office suggests a call
** January 2015:** Proposal submitted
** May 2015:** Proposal accepted
** September 2015:** Project starts
----
## Detailed timeline
### January to August 2014
- Seek information on the H2020 Programm
- Reach the broad community
- First five page draft
- Choice of a potential call
<p style="margin-bottom:2ex;"></p>
### September 2014
- Meeting with 12 potential participants
- Analyze the needs
- First list of tasks, work packages
- Reconfiguration of the consortium
----
### October 2014 to January 2015
- Writing
- Reconfiguration of work packages
- More reaching toward the broad community
- Extension of the consortium consortium
- Writing, writing, writing, ...
<p style="margin-bottom:2ex;"></p>
### May 2015 to August 2015
- Acceptation, Préparation, Launch
<p style="margin-bottom:2ex;"></p>
### September 2015
- Kickoff
---
## The making of: approach
### Follow my dreams
- Open source, open data, open publications, **open proposal**
- Bottom up:
- analyze and collect the needs of our community
- choose a call accordingly
- find a story that binds them all
----
### Engage the community, foster collective intelligence
<p style="margin-bottom:2ex;"></p>
#### Open Proposal
- Broad invitation to participate to the definition of the project
- Public writing of the project
<p style="margin-bottom:2ex;"></p>
#### Develop and share the **vision** on the project
- Who are we?
- What are our aims?
- Where do we start from?
- What is our strategy?
- How does this project relate to other projects?
- Why are we the dream team for the task?
----
### Empower the participants
#### Encourage
- to jointly define the vision of the project
- to do what they think is right (they know better)
<p style="margin-bottom:2ex;"></p>
#### Enable
- What's the status of the proposal?
- What remains to do?
- How can they help?
<p style="margin-bottom:2ex;"></p>
#### Smile, run forward, and hope ... and be thankful
<center>[When it works video](https://www.youtube.com/watch?v=kM9zcfRtOqo)</center>
----
### Collaborative tools: a key to success
#### Scale
- 100 pages
- 20 coauthors
- 3000 e-mails; 400 on the last two days
Forget about: Word, Dropbox, ...
----
### Collaborative tools: some recommendations
#### Proposal writing
- Version control and forge: e.g. [GitHub](github.com) or [GitLab](gitlab.com)
- Automate whatever you can: tables, graphics, computation of the budget, ...
- Source files in text: for example LateX + [proposal](http://www.ctan.org/tex-archive/macros/latex/contrib/proposal) style file
<p style="margin-bottom:2ex;"></p>
#### Communication
- General discussions and progress tracking: mailing list, e.g. on sympa
- Discussions on specific topics: e.g. GitHub issues
- Videoconferences: [appear.in](appear.in) / [hubl.in](hubl.in) / [framatalk.org](framatalk.org)
- Chat: gitter / slack / ...
- Live text editing and note taking: e.g. [hackmd.io](hackmd.io)
<p style="margin-bottom:2ex;"></p>
<center>Train, train, train your team; learn, learn, learn</center>
----
### Getting external help
#### Delegate to a specialized company?
- They will crunch your freedom. And your money.
- Get good collaborative tools instead.
<p style="margin-bottom:2ex;"></p>
<p style="margin-bottom:2ex;"></p>
#### Get a few good counselors!
- Your local European Affair service
- People with previous experience running a project
- Needed as well for launching the project
- Plan for a part time admin if you can
---
## To run or not to run for a EU project?
### Costs
#### Preparation
- Three months full time for the coordinator
- A few months spread over the other participants
- 3000 € (travel + one meeting)
#### Launching
- Two months full time for the coordinator
- 1000 € (web site, travel, ...)
#### Running
- A solid fourth of my time: administration + coordination
- Plus doing the work itself: engineering + research
----
### Benefits
<p style="margin-bottom:2ex;"></p>
- Means to achieve what you care about: funding, manpower
<p style="margin-bottom:2ex;"></p>
- Opportunities to grow your skills, your network, your vision
<p style="margin-bottom:2ex;"></p>
- An amazing human experience
<p style="margin-bottom:10%"></p>
- Incidentally: not bad for your career
But: you **will** be asked to take more responsibilities
----
## To run or not to run for a EU project?
- What are the needs of your community?<br>
Can funding actually help?
- Do you have the time and energy?
- Do you have good administrative support?
- Do you have experience coordinating people?
<p style="margin-bottom:2ex;"></p>
- Do you have a dream?<br>Something you care enough about to devote a
good chunk of your life? of your research?
<p style="margin-bottom:2ex;"></p>
- Are you ready for an adventure?
---
## Good luck!
### And if it works, never forget:
- You have been very lucky
- You will be entrusted funds **by the citizen** and **for your community**
- You won't own them, but be responsible for their best use
<center><img src="/public/logos/odk-elected-logo.svg" width="10%" alt="OpenDreamKit"/></center>
----
## More reading
- [On OpenDreamKit's open and collaborative proposal writing](http://opendreamkit.org/2015/01/31/open-proposal-writing/)
- The [developer's perspective](/about-developers) on OpenDreamKit
- [About OpenDreamKit](/about)
</script>
</section>
LinBox Winter Days 2018 Richerenches (FR), 2018-12-04 to 2018-18-072018-12-04T00:00:00+00:00http://opendreamkit.org/2018/12/04/LinBoxWinterDays<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: A. Breust, J-G. Dumas, C. Pernet, H. Zhu.</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and some travel expenses), for about 3.5k Euro.</p>
<h2 id="event-summary">Event summary</h2>
<p>The LinBox Winter Days 2018 took place in Richerenches
(France) from December 4th to 7th.
There were 10 participants from either Université Grenoble Alpes or Université of Montpellier. The focus of the meeting
was twofold: advancing core development of the libraries in the LinBox ecosystem, and welcoming and integrating new
developpers within the community. The main activities of the group during the meeting were:</p>
<ul>
<li>group brainstorming on new designs (e.g. refactorization of the dense matrix classes, of lifting containers, etc),</li>
<li>pair programming of the resulting new designs,</li>
<li>close interraction in bug correction and code review,</li>
<li>tutorial presentations for introducing new developpers to the librairies.</li>
</ul>
<h2 id="results-and-impact">Results and impact</h2>
<p>A large number of issues and bugs have been successfully processed. New design for the
lifting container classes and the dense matrix classes have been finalized and their implementation drafted. This effort
was then successfully pursued and merged into the libraries.</p>
GAP in Algebraic Research Summer School Aachen, Germany, 19th-22nd of November 20182018-11-19T00:00:00+00:00http://opendreamkit.org/2018/11/19/Aachen_school<h2 id="main-goals">Main goals</h2>
<p>The aim of this ``summer’’ school was
to give an overview of different applications of GAP in mathematical research.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Olexandr Konovalov was one of the instructors.
OpenDreamKit funded his participation to the event: \pounds 560.</p>
<h2 id="event-summary">Event summary</h2>
<p>The speakers presented their areas of research,
the functionality that is available in GAP to investigate these areas
computationally, and guided participants through these tools during the
accompanying hands-on programming lab sessions.</p>
<p><img src="/public/images/events/Aachen_school_2018" alt="" /></p>
<p>GAP in Algebraic Research Summer School, Aachen, Germany</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>Olexandr Konovalov
gave an introductory GAP Tutorial based on the
Software Carpentry lesson ``Programming with GAP’’
(https://doi.org/10.5281/zenodo.597073),
demonstrated GAP Jupyter notebooks and parallel
distributed calculations using the SCSCP package.
The format of this part of the school’s programme
followed the Software Carpentry teaching approaches.
In particular, we collected the feedback on minute
cards after each of the half-day sessions. Some quotes
from the feedback written by the learners:</p>
<ul>
<li>
<p>Nice step-by-step introduction</p>
</li>
<li>
<p>Good organisation of materials and presentation</p>
</li>
<li>
<p>The content are quite well organised</p>
</li>
<li>
<p>Idea with stickers is cool. This really makes asking questions easier</p>
</li>
<li>
<p>I really like the use of stickers as tools for asking for help and give feedback</p>
</li>
<li>
<p>I really liked the tutorial, well explained, and differences between listing on a group elements or a group G!!!</p>
</li>
<li>
<p>Appropriate speed</p>
</li>
<li>
<p>Started from the beginning, easy to follow</p>
</li>
<li>
<p>The session was understandable</p>
</li>
<li>
<p>I learned a lot of interesting ‘short’ commands</p>
</li>
<li>
<p>Lots of shortcuts - I am very lazy</p>
</li>
<li>
<p>Very useful tips on optimising the working process (autocompletion etc)</p>
</li>
<li>
<p>Many nice details</p>
</li>
<li>
<p>“Real-life” example of application (average order of group elements) that demonstrate necessary syntax =)</p>
</li>
<li>
<p>Good to solve a simple problem, not only technical details</p>
</li>
<li>
<p>Programming with a goal in mind</p>
</li>
<li>
<p>Very informative and useful</p>
</li>
<li>
<p>Like presentation of different packages - also interesting for people knowing GAP</p>
</li>
<li>
<p>Excellent the Jupyter notebook!! Incredible! I would like to see more about this application.</p>
</li>
<li>
<p>References to “modern” interfaces like Jupyter instead of command line</p>
</li>
<li>
<p>Some interesting tricks for people who already know GAP</p>
</li>
<li>
<p>The graphical applications were awesome!</p>
</li>
</ul>
<p><img src="/public/images/events/Aachen_school_2018" alt="" />
GAP in Algebraic Research Summer School, Aachen, Germany</p>
JupyterLab Workshop Austin (Texas), Nov. 1, 20182018-11-01T00:00:00+00:00http://opendreamkit.org/2018/11/01/jupyterlab<h2 id="main-goals">Main goals</h2>
<p>To develop JupyterLab further and set agenda for future development.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Vidar Fauske attended to participate in real-time collaboration discussions.</p>
<h2 id="event-summary">Event summary</h2>
<p>Core stakeholders in JupyterLab working together on high-impact tasks and direction of the project.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>Significant progress was made on JupyterLab in general, with particular progress on the design and architecture of real-time collaboration support.</p>
How OpenDreamKit supported the RSE revolution2018-10-29T00:00:00+00:00http://opendreamkit.org/2018/10/29/ODK-RSE<p>The term ‘Research Software Engineer’ refers to the people in research who are roughly in the middle of a continuum with ‘Software Engineer’ at one end and ‘Researcher’ at the other. They have skills and responsibilities associated with both professions but don’t fully identify with either one. Such roles are prevalent in industry with Microsoft being one of the leading examples of a company who developed career pathways for them.</p>
<center><img src="/assets/rse_slide1.png" alt="What is a Research Software Engineer?" /></center>
<p>Prior to 2012, <a href="https://www.software.ac.uk/">The Software Sustainability Institute</a>, conducted a study of academic job adverts and discovered that there were 194 different job titles for what we now call ‘Research Software Engineer’.</p>
<center><img src="/assets/rse_slide2.png" alt="One job. 194 job titles" style="width: 800px;" /></center>
<p>This, among many other things, made it difficult for such people to progress their careers within academia. When everybody refers to what you do by almost 200 different titles, your CV ends up looking a little odd to say the least.</p>
<p>For these people, there’s almost no chance to get promoted since the very things that make them valuable are not recognised within the traditional career pathways. There’s not enough research or teaching to get promoted as a lecturer and not enough IT, computing or management experience to be promoted as a software engineer.</p>
<p>Thus, the SSI came up with the term ‘Research Software Engineer’ (RSE) and campaigned to bring a community of such people together. For more information on the RSE movement, see the video of my JuliaCon 2018 talk linked to below</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/8ZSaAM8hhJ4" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe>
<p>By 2015, there were a small number of central ‘Research Software Engineering Groups’ within UK Universities with my group at Sheffield (co-founded by Paul Richmond) being among the first (Ref: RSE State of the Nation 2017: https://zenodo.org/record/495360#.W9Yg5hP7TOQ). OpenDreamKit was one of the first projects we won that demonstrated that funders would support RSEs on major grants – this improved credibility of the new role a great deal and helped secure its future at Sheffield.</p>
<p>Part of the OpenDreamKit dissemination strategy was to explicitly target the emerging RSE community. If you get the people who advise researchers, your reach is enhanced compared to targeting researchers directly. We developed and delivered a workshop on using Jupyter notebooks for reproducible research (materials at <a href="https://github.com/trallard/JNB_reproducible">https://github.com/trallard/JNB_reproducible</a>) for the <a href="https://rse.shef.ac.uk/blog/a-successful-2nd-rse-conference/">2nd international RSE conference</a>. This focused on ODK-developed enhancements such as nbdime and nbval that allowed version control and unit testing of notebooks.</p>
<p>The workshop was one of the most popular across the entire conference and, as such, we were asked to deliver it twice in order to meet demand. The only workshop that was more popular was Microsoft Azure who were giving away free credits!!</p>
<p>The RSE community is now thriving with central RSE groups in many of the top UK Universities. The <a href="https://rse.ac.uk/">UK Association of RSEs</a> has around 1000 members and became an independent, formal society with full charitable status in 2018. OpenDreamKit member Tania Allard was one of the <a href="https://twitter.com/sjh5000/status/1007639883165454336">signatories to this historic document</a> for RSE thanks to her position as elected member of the RSE committee.</p>
<center><img src="/assets/rse_slide6.png" alt="One job. 194 job titles" style="width: 800px;" /></center>
A successfull 2nd Formal Project Review2018-10-28T00:00:00+00:00http://opendreamkit.org/2018/10/28/second-project-review<p>On October 30th of 2018, OpenDreamKit underwent its second formal review by
the European Commission, represented by three reviewers and our former
and new project officers. About twenty five of us came to Luxembourg to
present our achievements during the second period of the project
(March 2017 to August 2018), including
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3Adeliverable+label%3AReportingPeriod2+">18 deliverables</a>.
See our <a href="/project/reports/#reporting-period-2">reports</a>.</p>
<p>After
<a href="https://opendreamkit.org/meetings/2018-10-28-Luxembourg/">two days of rehearsals</a>,
we had one day of
<a href="https://opendreamkit.org/meetings/2018-10-28-Luxembourg/ProjectReview/">presentations and software demonstrations</a>,
which all came up as quite a show.</p>
<p>Up to a legitimate complaint about our financial reports being
delivered very late by our administrations and some other minor points,
<strong>the feedback was once again very positive</strong>, with language such as
“very good technical work”, “very good sustainability plan”, “very
good management work considering difficulty of managing large project
with many and challenging goals”, “you have done extremely well”, “we
urge you to apply for future funding”. They also pointed out the
spirit, the passion, and the unity of our team despite its size. This
is apparently not that common among EU projects.</p>
<p>As for the last review, we had made a strong point in our reports and
presentations that a vast majority of what’s happening comes from the
ecosystem we support. All we do is exploit the special resources the
EU is entrusting us to knock down some tough hurdles that are
preventing the ball to roll. <strong>Big kudos to our communities!</strong></p>
<p>We have now entered OpenDreamKit’s third and last period. In one year
day to day, we will be presenting the outcome of four years of hard
work. Hopefully by then we will have secured future funding in one
form or the other to keep supporting our ecosystem.</p>
3d interactive data visualisation with K3D-jupyter2018-10-28T00:00:00+00:00http://opendreamkit.org/2018/10/28/3d<p><strong>K3D-jupyter</strong> is a Jupyter notebook 3d visualization package created
within ODK project.</p>
<p>The primary aim of K3D is to be an easy tool for 3d visualization. We
designed it with native interoperation with existing powerful
libraries such as VTK however, it is not strictly depending on
them. K3D plots are ipywidgets, thus it assures a perfect interaction of
a browser side widget with Python kernel by a proven and standardized
protocol.</p>
<p>Let us start with a simple example. The following Python code will
make a surface plot of a function of two variables:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">k3d</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="n">np</span>
<span class="n">Nx</span><span class="p">,</span> <span class="n">Ny</span> <span class="o">=</span> <span class="mi">34</span><span class="p">,</span> <span class="mi">33</span>
<span class="n">xmin</span><span class="p">,</span> <span class="n">xmax</span> <span class="o">=</span> <span class="o">-</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span>
<span class="n">ymin</span><span class="p">,</span> <span class="n">ymax</span> <span class="o">=</span> <span class="o">-</span><span class="mi">0</span><span class="p">,</span> <span class="mi">3</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="n">linspace</span><span class="p">(</span><span class="n">xmin</span><span class="p">,</span> <span class="n">xmax</span><span class="p">,</span> <span class="n">Nx</span><span class="p">)</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="n">linspace</span><span class="p">(</span><span class="n">ymin</span><span class="p">,</span> <span class="n">ymax</span><span class="p">,</span> <span class="n">Ny</span><span class="p">)</span>
<span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="n">meshgrid</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="n">sin</span><span class="p">(</span><span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span>
<span class="n">plot</span> <span class="o">=</span> <span class="n">k3d</span><span class="p">.</span><span class="n">plot</span><span class="p">()</span>
<span class="n">plt_surface</span> <span class="o">=</span> <span class="n">k3d</span><span class="p">.</span><span class="n">surface</span><span class="p">(</span><span class="n">f</span><span class="p">.</span><span class="n">astype</span><span class="p">(</span><span class="n">np</span><span class="p">.</span><span class="n">float32</span><span class="p">),</span> <span class="n">bounds</span><span class="o">=</span><span class="p">[</span><span class="n">xmin</span><span class="p">,</span><span class="n">xmax</span><span class="p">,</span><span class="n">ymin</span><span class="p">,</span><span class="n">ymax</span><span class="p">])</span>
<span class="n">plot</span> <span class="o">+=</span> <span class="n">plt_surface</span>
<span class="n">plot</span><span class="p">.</span><span class="n">display</span><span class="p">()</span>
</code></pre></div></div>
<iframe src="/public/images/k3d/surface.html" height="550" width="100%">></iframe>
<p>The <code class="language-plaintext highlighter-rouge">plot</code> variable will be bound to an instance of the plot
widget. It contains a single plot object - <code class="language-plaintext highlighter-rouge">plt_surface</code>. In the
package there are many such objects available, covering most of the typical
needs for 3d data visualization. They can be added to the plot in
a similar way and they will appear on the same scene.</p>
<p>The sampled values of our function are available as an attibute of
<code class="language-plaintext highlighter-rouge">plt_surface.heights</code>. Thus we can write. for example:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">plt_surface</span><span class="p">.</span><span class="n">heights</span><span class="p">.</span><span class="n">shape</span>
</code></pre></div></div>
<p>Since, <code class="language-plaintext highlighter-rouge">plt_surface</code> is an ipywidget, its state on a javascript side
and python are connected. We can simply assign it to other values
(numpy array) and the plot will be automatically updated:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">plt_surface</span><span class="p">.</span><span class="n">heights</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="n">log</span><span class="p">((</span><span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span><span class="p">))</span>
</code></pre></div></div>
<p>In fact, such updates are efficient enough to allow for a high degree of
interactivity. One can produce animations:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">plot</span><span class="p">.</span><span class="n">camera_auto_fit</span> <span class="o">=</span> <span class="bp">False</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">50</span><span class="p">):</span>
<span class="n">plt_surface</span><span class="p">.</span><span class="n">heights</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="n">sin</span><span class="p">(</span><span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="n">y</span><span class="o">**</span><span class="mi">2</span><span class="o">+</span><span class="mf">1.6</span><span class="o">/</span><span class="mi">10</span><span class="o">*</span><span class="n">i</span><span class="p">)</span>
</code></pre></div></div>
<p>Such an approach can be desirable when one needs to combine server-side
data processing with 3d interactive visualization. For example, in K3D
we do not support unstructured volumetric meshes. There exist,
however, a very powerful library for processing such data -
VTK. An arbitrary unstructured mesh can be displayed at its surface which
consists of triangles. The latter can be easily visualized with
K3D. Therefore one can process data on server-side using VTK and send
its triangularized surface representation to the plot. Since the
surface data is in many cases much smaller than the volume, it allows
for building responsive web applications.</p>
<p>The effect of such data processing are following:</p>
<iframe src="/public/images/k3d/vtk_cutter.html" height="550" width="100%">></iframe>
<p>The web browser gives access to the graphics hardware and allows to
perform very computationally intensive visualizations. One such
technique is a volume rendering. It has been implemented in K3D
widget:</p>
<iframe width="900" height="500" src="https://www.youtube.com/embed/zCeQ_ZXy_Ps" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
<p>The flexibility
of network architecture allows to easily build an HPC data
visualization pipelines using jupyter notebook and webGL
frontend. One can preprocess volume of data on the server and send
small dataset to K3D widget for immediate inspection.</p>
<ul>
<li>https://github.com/K3D-tools/K3D-jupyter</li>
</ul>
Deploying a containerized JupyterHub server with Docker2018-10-17T00:00:00+00:00http://opendreamkit.org/2018/10/17/jupyterhub-docker<h2 id="medium-scale-jupyterhub-deployments">Medium-scale JupyterHub deployments</h2>
<p><a href="https://jupyterhub.readthedocs.io/en/stable/">JupyterHub</a> is the new
cool kid on the block. Everyone knows <a href="https://jupyter.org/try">Jupyter
notebooks</a> and how much they have
revolutionized the workflows of scientists and students alike.
Whereas Jupyter is meant to run on a personal computer, JupyterHub is
the solution that brings Jupyter to your own cloud, be it your team’s,
your university’s or your company’s cloud.</p>
<p>Not all clouds are born equal, though. The <a href="https://zero-to-jupyterhub.readthedocs.io/en/stable/">official JupyterHub
documentation</a>
targets very large clouds with multiple nodes, managed through
<a href="https://kubernetes.io/">Kubernetes</a>.<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">1</a></sup> This kind of deployment is
adapted for very large structures, such as universities, large
companies etc.</p>
<p><a href="https://the-littlest-jupyterhub.readthedocs.io/en/latest/">“The littlest
JupyterHub”</a>
adheres to a different philosophy: a JupyterHub deployment on a single
server, with no virtualization or containerization technology. This
kind of deployment is perfect for your small team, where one
administrator can manually create accounts, and users can share data.</p>
<p>So far, no documentation seems to have addressed the <em>medium-scale</em>
deployment, e.g., the one adapted for your university department, or
for your company with hundreds of employees. The goal of this tutorial
is to present a complete solution to deploy a JupyterHub server with
delegated authentication and containerized environments, based on
<a href="https://www.docker.com/">Docker</a>. If you have a powerful server lying
underused in your organization’s racks, chances are you will find this
deployment to your liking.</p>
<p>The set-up described in this tutorial has been deployed at <a href="http://www.uvsq.fr/">University
of Versailles</a> for the whole faculty of science.
It is used both for teaching and research. You can find the full
configuration scripts here:
<a href="https://github.com/defeo/jupyterhub-docker/">https://github.com/defeo/jupyterhub-docker/</a>. I will be glad to
receive your comments, suggestions, bug reports and help requests via
the <a href="https://github.com/defeo/jupyterhub-docker/issues">GitHub Issues</a>
section.</p>
<h2 id="the-architecture-of-jupyterhub">The architecture of JupyterHub</h2>
<p>A JupyterHub deployment has several components. Processes first:</p>
<ul>
<li>
<p>A configurable <em>Proxy</em>, responsible for receiving all web requests
and dispatching them to the appropriate component.</p>
</li>
<li>
<p>A <em>Hub</em>, responsible for handling authentication, <em>spawning</em>
single-user Jupyter notebook servers, and configuring the Proxy.</p>
</li>
<li>
<p>Single-user Jupyter servers. These are nothing else than the Jupyter
notebook servers you are used to run on your personal computer,
however they are started and stopped on demand by the Hub as users
come and go.</p>
</li>
</ul>
<p>More details can be found on the <a href="https://jupyterhub.readthedocs.io/en/stable/">official JupyterHub
documentation</a>. In our
deployment, we will put the Proxy and the Hub in the same container,
and the single-user server in another one.</p>
<p>Although JupyterHub can be directly exposed to the web without any
intermediary, and even handle TLS certificates for https, it will be
convenient for us to put it behind a reverse proxy, for efficiency,
security, and other reasons that will be apparent later. Typical
choices for a reverse proxy are <a href="https://httpd.apache.org/docs/2.4/mod/mod_proxy.html">Apache
2</a> and
<a href="https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/">Nginx</a>;
however in this tutorial we chose to use
<a href="https://traefik.io/">Traefik</a>, for its ease of use and nice
integration with Docker.</p>
<p>Thus, we will in total have to handle three container (images): one
for the Proxy+Hub, one for the reverse proxy, and one for the
single-use servers. To simplify the management, and automatizing the
deployment, we will use <a href="https://docs.docker.com/compose/">Docker
Compose</a>. All the configuration
will be kept in a single folder, with the following structure:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./
+-- .env
+-- docker-compose.yml
+-- jupyterhub/
| +-- Dockerfile
| +-- jupyter-config.py
+-- jupyterlab/
| +-- Dockerfile
+-- reverse-proxy/
+-- traefik.toml
</code></pre></div></div>
<p>The main configuration file for Docker Compose is
<code class="language-plaintext highlighter-rouge">docker-compose.yml</code>: it configures all containers (<em>services</em>, in
Compose jargon), and associated <em>volumes</em> and <em>networks</em>. It will look
like this:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">version</span><span class="pi">:</span> <span class="s1">'</span><span class="s">3'</span>
<span class="na">services</span><span class="pi">:</span>
<span class="c1"># Configuration for Hub+Proxy</span>
<span class="na">jupyterhub</span><span class="pi">:</span>
<span class="s">...</span>
<span class="c1"># Configuration for reverse proxy</span>
<span class="na">reverse-proxy</span><span class="pi">:</span>
<span class="s">...</span>
<span class="c1"># Configuration for the single-user servers</span>
<span class="na">jupyterlab</span><span class="pi">:</span>
<span class="s">...</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="s">...</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="s">...</span>
</code></pre></div></div>
<p>We now explain the configuration of each service, and the contents of
each additional file in detail.</p>
<h2 id="the-reverse-proxy">The reverse proxy</h2>
<p>This is the simplest one, since most of the configuration is pushed
into the other services. Traefik uses a single file for configuration,
named <code class="language-plaintext highlighter-rouge">traefik.toml</code>. The <a href="https://docs.traefik.io/#the-trfik-quickstart-using-docker">recommended
way</a> to
use it is to bind-mount it inside the default docker container. Using
<code class="language-plaintext highlighter-rouge">docker-compose.yml</code>:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="na">reverse-proxy</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">traefik</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">80:80"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">443:443"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">8080:8080"</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">./reverse-proxy/traefik.toml:/etc/traefik/traefik.toml</span>
<span class="pi">-</span> <span class="s">/var/run/docker.sock:/var/run/docker.sock</span>
<span class="pi">-</span> <span class="s">/etc/certs:/etc/certs</span>
</code></pre></div></div>
<p>Notice that we also bound some internet ports: ports 80 and 443 are
for http and https, while port 8080 is for the Traefik <em>dashboard</em>,
useful for analytics and debugging. You probably want to block access
to the 8080 port from outside your local network (e.g., via iptables
or some other firewall technology).</p>
<p>We also mounted two other <em>volumes</em> in the container. The first one,
<code class="language-plaintext highlighter-rouge">docker.sock</code>, is to let Traefik access the Docker server, this will
let it <em>automagically</em> configure routing web requests to other service
as they are started by Docker.</p>
<p>The second one is to let Traefik access your server TLS credentials,
for https. This assumes that you have put in <code class="language-plaintext highlighter-rouge">/etc/certs</code> on your host
machine a <code class="language-plaintext highlighter-rouge">server.crt</code> and a <code class="language-plaintext highlighter-rouge">server.key</code> file, containing your server
certificate and private key.</p>
<p>Now we are ready to write the <code class="language-plaintext highlighter-rouge">traefik.toml</code> configuration file. We
want to:</p>
<ul>
<li>redirect all http requests to https, configure TLS certificates,</li>
<li>activate automatic discovery of running Docker containers,</li>
<li>activate the Traefik <em>dashboard</em>.</li>
</ul>
<div class="language-toml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="py">debug</span> <span class="p">=</span> <span class="kc">false</span>
<span class="py">logLevel</span> <span class="p">=</span> <span class="s">"ERROR"</span>
<span class="py">defaultEntryPoints</span> <span class="p">=</span> <span class="nn">["https","http"]</span>
<span class="c"># Redirect HTTP -> HTTPS, install certificates</span>
<span class="nn">[entryPoints]</span>
<span class="nn">[entryPoints.http]</span>
<span class="py">address</span> <span class="p">=</span> <span class="s">":80"</span>
<span class="nn">[entryPoints.http.redirect]</span>
<span class="py">entryPoint</span> <span class="p">=</span> <span class="s">"https"</span>
<span class="nn">[entryPoints.https]</span>
<span class="py">address</span> <span class="p">=</span> <span class="s">":443"</span>
<span class="nn">[entryPoints.https.tls]</span>
<span class="nn">[[entryPoints.https.tls.certificates]]</span>
<span class="py">certFile</span> <span class="p">=</span> <span class="s">"/etc/certs/server.crt"</span>
<span class="py">keyFile</span> <span class="p">=</span> <span class="s">"/etc/certs/server.key"</span>
<span class="c"># Activate docker API</span>
<span class="nn">[docker]</span>
<span class="py">domain</span> <span class="p">=</span> <span class="s">"docker.local"</span>
<span class="py">watch</span> <span class="p">=</span> <span class="kc">true</span>
<span class="c"># Activate Traefik dashboard</span>
<span class="nn">[api]</span>
</code></pre></div></div>
<p>This is all that’s needed. You may be wondering why <code class="language-plaintext highlighter-rouge">traefik.toml</code>
contains no configuration specific to JupyterHub. This is because the
JupyterHub components will communicate with Traefik via the Docker
api.</p>
<h2 id="the-hub">The Hub</h2>
<p>The Hub is the centerpiece of our setup. When a user first lands on
our JupyterHub instance, it is directed to the Hub, which then:</p>
<ol>
<li>Authenticates the user by any of the configured methods (see
<a href="#authentication">below</a>);</li>
<li><em>Spawns</em> a single-user Jupyter notebook server for the user (in our
case, this will run in a Docker container);</li>
<li>Redirects the user to the single-user server.</li>
</ol>
<p>We will start from the <a href="https://hub.docker.com/r/jupyterhub/jupyterhub/">official JupyterHub
container</a>, and build
our own Hub on top of it. We’ll put all the configuration files in
the folder named <code class="language-plaintext highlighter-rouge">jupyterhub</code>, and we’ll tell Compose to build and
configure the container:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="na">jupyterhub</span><span class="pi">:</span>
<span class="na">build</span><span class="pi">:</span> <span class="s">jupyterhub</span> <span class="c1"># Build the container from this folder.</span>
<span class="na">container_name</span><span class="pi">:</span> <span class="s">jupyterhub_hub</span> <span class="c1"># The service will use this container name.</span>
<span class="na">volumes</span><span class="pi">:</span> <span class="c1"># Give access to Docker socket.</span>
<span class="pi">-</span> <span class="s">/var/run/docker.sock:/var/run/docker.sock</span>
<span class="na">environment</span><span class="pi">:</span> <span class="c1"># Env variables passed to the Hub process.</span>
<span class="na">DOCKER_JUPYTER_IMAGE</span><span class="pi">:</span> <span class="s">jupyterlab_img</span>
<span class="na">DOCKER_NETWORK_NAME</span><span class="pi">:</span> <span class="s">${COMPOSE_PROJECT_NAME}_default</span>
<span class="na">HUB_IP</span><span class="pi">:</span> <span class="s">jupyterhub_hub</span>
<span class="na">labels</span><span class="pi">:</span> <span class="c1"># Traefik configuration.</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.enable=true"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.frontend.rule=Host:jupyter.example.com"</span>
</code></pre></div></div>
<p>Here’s some highlights from the Compose configuration:</p>
<ul>
<li>
<p>We mount the Docker socket inside the container, so that it will be
able to spawn containers for the single-user Jupyter servers.</p>
</li>
<li>
<p>We set some environment variables for the Hub process, they will be
used in the Hub configuration file <code class="language-plaintext highlighter-rouge">jupyterhub_config.py</code>:</p>
<ul>
<li>
<p><code class="language-plaintext highlighter-rouge">DOCKER_JUPYTER_IMAGE</code> is the name of the Docker image for the
single-user servers; this must match the image configured in the
<code class="language-plaintext highlighter-rouge">jupyterlab</code> section of <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> (see
<a href="#the-jupyter-notebook-servers">below</a>).</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">DOCKER_NETWORK_NAME</code> is the name of the Docker network used by
the services; normally, this network gets an automatic name from
Docker Compose, but the Hub needs to know this name to connect the
single-user servers to it. To control the network name we use a
little hack: we pass an environment variable
<code class="language-plaintext highlighter-rouge">COMPOSE_PROJECT_NAME</code> to Docker Compose, and the network name is
obtained by appending <code class="language-plaintext highlighter-rouge">_default</code> to it.</p>
<p><strong>ATTENTION!</strong> Experience shows that this step is often missed or
misunderstood. A value (any name you like, really) for the
environment variable <code class="language-plaintext highlighter-rouge">COMPOSE_PROJECT_NAME</code> <strong>must be passed</strong> to
Docker Compose. You can do this either via the shell, or by adding
a <code class="language-plaintext highlighter-rouge">.env</code> file next to <code class="language-plaintext highlighter-rouge">docker-compose.yml</code>, with the following
content</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Name of our Docker Compose project</span>
<span class="nv">COMPOSE_PROJECT_NAME</span><span class="o">=</span>my_hub
</code></pre></div> </div>
<p>The <code class="language-plaintext highlighter-rouge">.env</code> file solution is the choice we made in the <a href="https://github.com/defeo/jupyterhub-docker/blob/master/.env">example
repo</a>.</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">HUB_IP</code> is the IP address of the Hub service within the docker
network. By using the <code class="language-plaintext highlighter-rouge">container_name</code> Compose directive, we can
set an alias for the IP, and use the same for <code class="language-plaintext highlighter-rouge">HUB_IP</code>.</p>
</li>
</ul>
</li>
<li>
<p>We set <em>labels</em> for configuring Traefik. These have no effect on
Docker, but are used by Traefik for configuring routing. In this
example, we are activating the Traefik API, and we are asking to
redirect all requests for <code class="language-plaintext highlighter-rouge">jupyter.example.com</code> to the Hub service.</p>
</li>
</ul>
<p>We can now move to the configuration of the container itself. We start
with the <code class="language-plaintext highlighter-rouge">Dockerfile</code>:</p>
<div class="language-docker highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Do not forget to pin down the version</span>
<span class="k">FROM</span><span class="s"> jupyterhub/jupyterhub:0.9.3</span>
<span class="c"># Copy the JupyterHub configuration in the container</span>
<span class="k">COPY</span><span class="s"> jupyterhub_config.py .</span>
<span class="c"># Download script to automatically stop idle single-user servers</span>
<span class="k">RUN </span>wget https://raw.githubusercontent.com/jupyterhub/jupyterhub/0.9.3/examples/cull-idle/cull_idle_servers.py
<span class="c"># Install dependencies (for advanced authentication and spawning)</span>
<span class="k">RUN </span>pip <span class="nb">install</span> <span class="se">\
</span> <span class="nv">dockerspawner</span><span class="o">==</span>0.10.0 <span class="se">\
</span> <span class="nv">oauthenticator</span><span class="o">==</span>0.8.0
</code></pre></div></div>
<p>It is important that we explicitly pin down a version of JupyterHub
(0.9.3 in the example), indeed the containers for the Hub and for the
single user servers must use the same version of JupyterHub.</p>
<p>Next to <code class="language-plaintext highlighter-rouge">Dockerfile</code>, we have <code class="language-plaintext highlighter-rouge">jupyterhub_config.py</code> to configure the
Hub. This plain Python file contains many different sections. We
start with the configuration of the <em>spawner</em>: we use the class
<code class="language-plaintext highlighter-rouge">DockerSpawner</code> to spawn single-user servers in a separate Docker
container. We use here the environment variables that we have set in
<code class="language-plaintext highlighter-rouge">docker-compose.yml</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">os</span>
<span class="n">c</span><span class="p">.</span><span class="n">JupyterHub</span><span class="p">.</span><span class="n">spawner_class</span> <span class="o">=</span> <span class="s">'dockerspawner.DockerSpawner'</span>
<span class="n">c</span><span class="p">.</span><span class="n">DockerSpawner</span><span class="p">.</span><span class="n">image</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">environ</span><span class="p">[</span><span class="s">'DOCKER_JUPYTER_IMAGE'</span><span class="p">]</span>
<span class="n">c</span><span class="p">.</span><span class="n">DockerSpawner</span><span class="p">.</span><span class="n">network_name</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">environ</span><span class="p">[</span><span class="s">'DOCKER_NETWORK_NAME'</span><span class="p">]</span>
<span class="n">c</span><span class="p">.</span><span class="n">JupyterHub</span><span class="p">.</span><span class="n">hub_ip</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">environ</span><span class="p">[</span><span class="s">'HUB_IP'</span><span class="p">]</span>
</code></pre></div></div>
<p>Optionally, we may want to stop the single-user servers after a
certain amount of idle time. Following <a href="https://github.com/jupyterhub/jupyterhub/blob/master/examples/cull-idle">this
example</a>,
we register a JupyterHub <em>service</em> like this:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">c</span><span class="p">.</span><span class="n">JupyterHub</span><span class="p">.</span><span class="n">services</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s">'name'</span><span class="p">:</span> <span class="s">'cull_idle'</span><span class="p">,</span>
<span class="s">'admin'</span><span class="p">:</span> <span class="bp">True</span><span class="p">,</span>
<span class="s">'command'</span><span class="p">:</span> <span class="s">'python /srv/jupyterhub/cull_idle_servers.py --timeout=3600'</span><span class="p">.</span><span class="n">split</span><span class="p">(),</span>
<span class="p">},</span>
<span class="p">]</span>
</code></pre></div></div>
<p>For a complete list of all available configuration options, you can
run the following command (answer <code class="language-plaintext highlighter-rouge">y</code> to the question):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose run --rm jupyterhub jupyterhub --generate-config -f /dev/stdout
</code></pre></div></div>
<h3 id="authentication">Authentication</h3>
<p>For the Hub configuration to be complete, we have to configure an
authentication method. By default, JupyterHub authenticates users
with the local system (more precisely, via
<a href="https://en.wikipedia.org/wiki/Linux_PAM">PAM</a>), but this is not
useful for us.</p>
<p>The quickest way to get a JupyterHub server running with a working
authentication, is to delegate to an authentication service such as
GitLab’s. This requires adding only two lines to
<code class="language-plaintext highlighter-rouge">jupyterhub-config.py</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">## Configure authentication (delagated to GitLab)
</span><span class="kn">from</span> <span class="nn">oauthenticator.gitlab</span> <span class="kn">import</span> <span class="n">GitLabOAuthenticator</span>
<span class="n">c</span><span class="p">.</span><span class="n">JupyterHub</span><span class="p">.</span><span class="n">authenticator_class</span> <span class="o">=</span> <span class="n">GitLabOAuthenticator</span>
</code></pre></div></div>
<p>Other preconfigured third party services are documented
<a href="https://github.com/jupyterhub/oauthenticator">here</a>. In principle,
any OAuth2 server can be used as third party authentication server.
If you have such a server in your institution, you can use it like
this:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">oauthenticator.oauth2</span> <span class="kn">import</span> <span class="n">OAuthLoginHandler</span>
<span class="kn">from</span> <span class="nn">oauthenticator.generic</span> <span class="kn">import</span> <span class="n">GenericOAuthenticator</span>
<span class="kn">from</span> <span class="nn">tornado.auth</span> <span class="kn">import</span> <span class="n">OAuth2Mixin</span>
<span class="c1"># OAuth2 endpoints
</span><span class="k">class</span> <span class="nc">MyOAuthMixin</span><span class="p">(</span><span class="n">OAuth2Mixin</span><span class="p">):</span>
<span class="n">_OAUTH_AUTHORIZE_URL</span> <span class="o">=</span> <span class="s">'https://oauth2.example.com/login'</span>
<span class="n">_OAUTH_ACCESS_TOKEN_URL</span> <span class="o">=</span> <span class="s">'https://oauth2.example.com/token'</span>
<span class="k">class</span> <span class="nc">MyOAuthLoginHandler</span><span class="p">(</span><span class="n">OAuthLoginHandler</span><span class="p">,</span> <span class="n">MyOAuthMixin</span><span class="p">):</span>
<span class="k">pass</span>
<span class="c1"># Authenticator configuration
</span><span class="k">class</span> <span class="nc">MyOAuthAuthenticator</span><span class="p">(</span><span class="n">GenericOAuthenticator</span><span class="p">):</span>
<span class="n">login_service</span> <span class="o">=</span> <span class="s">'Example'</span>
<span class="n">login_handler</span> <span class="o">=</span> <span class="n">MyOAuthLoginHandler</span>
<span class="n">userdata_url</span> <span class="o">=</span> <span class="s">'https://oauth2.example.com/userdata'</span>
<span class="n">token_url</span> <span class="o">=</span> <span class="s">'https://oauth2.example.com/token'</span>
<span class="n">oauth_callback_url</span> <span class="o">=</span> <span class="s">'https://jupyter.example.com/hub/oauth_callback'</span>
<span class="n">client_id</span> <span class="o">=</span> <span class="s">'...'</span> <span class="c1"># Your client ID and secret, as provided to you
</span> <span class="n">client_secret</span> <span class="o">=</span> <span class="s">'...'</span> <span class="c1"># by the OAuth2 service.
</span>
<span class="n">c</span><span class="p">.</span><span class="n">JupyterHub</span><span class="p">.</span><span class="n">authenticator_class</span> <span class="o">=</span> <span class="n">MyOAuthAuthenticator</span>
</code></pre></div></div>
<p>Alternatively, your institution may have an LDAP server, in which case
you can use the <code class="language-plaintext highlighter-rouge">LDAPAuthenticator</code> with the configuration described
<a href="https://github.com/jupyterhub/ldapauthenticator">here</a>: modify
<code class="language-plaintext highlighter-rouge">Dockerfile</code> to install <code class="language-plaintext highlighter-rouge">jupyterhub-ldapauthenticator</code>, then add (at
least) these lines to <code class="language-plaintext highlighter-rouge">jupyterhub_config.py</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">c</span><span class="p">.</span><span class="n">LDAPAuthenticator</span><span class="p">.</span><span class="n">server_address</span> <span class="o">=</span> <span class="s">'ldap.example.com'</span>
<span class="n">c</span><span class="p">.</span><span class="n">LDAPAuthenticator</span><span class="p">.</span><span class="n">bind_dn_template</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">c</span><span class="p">.</span><span class="n">JupyterHub</span><span class="p">.</span><span class="n">authenticator_class</span> <span class="o">=</span> <span class="s">'ldapauthenticator.LDAPAuthenticator'</span>
</code></pre></div></div>
<p>Another common authentication service in Universities is <a href="https://www.apereo.org/projects/cas">Apereo
CAS</a>, which is supported by
<a href="https://github.com/cwaldbieser/jhub_cas_authenticator">Jupyterhub CAS
Authenticator</a>.
You will find an example configuration
<a href="https://github.com/defeo/jupyterhub-docker/blob/master/jupyterhub/jupyterhub_config.py#L5-L17">here</a>.</p>
<h2 id="the-jupyter-notebook-servers">The Jupyter notebook servers</h2>
<p>The last component of our setup is the single-user Jupyter server.
This is the more versatile and configurable part, as it defines the
environment where your users are going to work.</p>
<p>The simplest way to get started, is to use one of the pre-packaged
<a href="https://jupyter-docker-stacks.readthedocs.io/en/latest/">Jupyter Docker
stacks</a>.
This only requires one line in <code class="language-plaintext highlighter-rouge">docker-compose.yml</code>:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="na">jupyterlab</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">jupyterlab/scipy-notebook:1145fb1198b2</span>
<span class="na">command</span><span class="pi">:</span> <span class="s">echo</span>
</code></pre></div></div>
<p>The extra line <code class="language-plaintext highlighter-rouge">command: echo</code> is there so that, when Docker Compose
starts the service, it terminates immediately. Indeed this image is
meant to be loaded by the Hub, not by Compose. Do not forget to edit
the <code class="language-plaintext highlighter-rouge">jupyterhub</code> section of <code class="language-plaintext highlighter-rouge">docker-compose.yml</code>, so that the
<code class="language-plaintext highlighter-rouge">DOCKER_JUPYTER_IMAGE</code> env variable matches this one.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="na">jupyterhub</span><span class="pi">:</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="na">DOCKER_JUPYTER_IMAGE</span><span class="pi">:</span> <span class="s">jupyterlab/scipy-notebook:1145fb1198b2</span>
</code></pre></div></div>
<p>Alternatively, you can use any of <a href="#todo">OpenDreamKit’s pre-built images</a>.</p>
<p>Finally, you can build your own image: the only requirement is that it
contains the <code class="language-plaintext highlighter-rouge">jupyterhub</code> Python package (and possibly some Jupyter
kernels, to make it interesting). I recommend starting from one of
the Jupyter Docker stacks images: create a file
<code class="language-plaintext highlighter-rouge">jupyterlab/Dockerfile</code> with, e.g., the following contents:</p>
<div class="language-docker highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">FROM</span><span class="s"> jupyter/scipy-notebook:1145fb1198b2</span>
<span class="k">RUN </span>conda <span class="nb">install</span> <span class="nt">--quiet</span> <span class="nt">--yes</span> <span class="se">\
</span> <span class="s1">'r-base=3.4.1'</span> <span class="se">\
</span> <span class="s1">'r-irkernel=0.8*'</span><span class="o">&&</span> <span class="se">\
</span> conda clean <span class="nt">-tipsy</span>
</code></pre></div></div>
<p>then modify <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> as follows:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="na">jupyterlab</span><span class="pi">:</span>
<span class="na">build</span><span class="pi">:</span> <span class="s">jupyterlab</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">jupyterlab_img</span>
<span class="na">command</span><span class="pi">:</span> <span class="s">echo</span>
</code></pre></div></div>
<p>It is recommended that you test your image alone, before starting
JupyterHub. You can do so by building it with Compose:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose build jupyterlab
</code></pre></div></div>
<p>then running a test server (adapt the ports to your configuration),
with:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">--rm</span> <span class="nt">-p</span> 8888:8888 jupyterlab_img
</code></pre></div></div>
<p>and copying in your browser the URL printed on the console (something
like <a href="http://127.0.0.1:8888/?token=012...">http://127.0.0.1:8888/?token=012...</a>). You can stop the test
server hitting <code class="language-plaintext highlighter-rouge">Ctrl-C</code>.</p>
<p>If everything works normally, you can try out the excellent JupyterLab
interface by replacing <a href="http://127.0.0.1:8888/user/.../tree">http://127.0.0.1:8888/user/.../tree</a> with
<a href="http://127.0.0.1:8888/user/.../lab">http://127.0.0.1:8888/user/.../lab</a>. You will not be disappointed,
<a href="https://jupyterlab.readthedocs.io/en/stable/">JupyterLab</a> is a game
changer: it transforms the old notebook interface in a full featured
IDE, with multiple split views, file explorer, Jupyter notebooks, and
much more, in a single browser window.</p>
<p>If you want to user JupyterLab as the default interface for your
JupyterHub users, configure the Hub by adding this line to
<code class="language-plaintext highlighter-rouge">jupyterhub-config.py</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Redirect to JupyterLab, instead of the plain Jupyter notebook
</span><span class="n">c</span><span class="p">.</span><span class="n">Spawner</span><span class="p">.</span><span class="n">default_url</span> <span class="o">=</span> <span class="s">'/lab'</span>
</code></pre></div></div>
<h2 id="running">Running!</h2>
<p>You are now ready to test your JupyterHub server. As superuser, or as
a user of the <code class="language-plaintext highlighter-rouge">docker</code> group, run</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose build
docker-compose up
</code></pre></div></div>
<p>The JupyterHub server loads up and, if all is configured properly,
starts waiting for connections. Test that authentication and starting
up single-user servers works properly, try playing with some
notebooks. When you are satisfied you can stop the server by hitting
<code class="language-plaintext highlighter-rouge">Ctrl-C</code>.</p>
<p>Be sure to read the documentation to <a href="https://docs.docker.com/compose/">Docker
compose</a>, and
<a href="https://jupyterhub.readthedocs.io/">JupyterHub</a> before going into
production. Customize the setup to your liking, then, when you are
ready to put the server online for good, launch it with</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose up -d
</code></pre></div></div>
<p>You can stop and destroy the server with</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose down
</code></pre></div></div>
<p>Note that single-user servers are not destroyed when they are
terminated: when a user returns, JupyterHub will look for a Docker
container named <code class="language-plaintext highlighter-rouge">jupyter-</code><em><code class="language-plaintext highlighter-rouge">username</code></em>, and will restart it with all
its data. This means that, even if you update the JupyterLab image,
returning users will not see the changes until you destroy their old
containers. You can list all containers, including inactive ones, with</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker ps -a
</code></pre></div></div>
<p>and you can remove them individually with <code class="language-plaintext highlighter-rouge">docker rm</code>. To remove all
containers at once (after the JupyterHub server has been stopped), you
can run a command like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker rm $(docker ps -qa -f "name=jupyter-")
</code></pre></div></div>
<p>Be wary that this command destroys all data associated with the users,
including their work. Use with care!</p>
<h2 id="data-persistence">Data persistence</h2>
<p>With this configuration, our JupyterHub server is not persistent: all
user data is irremediably lost when the containers are destroyed.
This may be acceptable for some use cases, however it is not what is
typically wanted.</p>
<p>Using <a href="https://docs.docker.com/storage/volumes/">Docker volumes</a>, we
can make storage permanent. Two kinds of data need to be stored: the
Hub data, containing information on administrators and existing users,
and the user’s data for the single-user servers.</p>
<p>To persist the Hub data, simply modify <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> by adding
a volume and mounting it inside the <code class="language-plaintext highlighter-rouge">jupyterhub</code> service:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">services</span><span class="pi">:</span>
<span class="na">jupyterhub</span><span class="pi">:</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">jupyterhub_data:/srv/jupyterhub</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="na">jupyterhub_data</span><span class="pi">:</span>
</code></pre></div></div>
<p>Be careful with this configuration: if you update
<code class="language-plaintext highlighter-rouge">jupyterhub_config.py</code> and rebuild the <code class="language-plaintext highlighter-rouge">jupyterhub</code> service, the
contents will not be updated inside the volume, unless you destroy it
first with <code class="language-plaintext highlighter-rouge">docker volume rm</code>.</p>
<p>The Hub takes care of handling volumes for the single-user servers.
We can configure data persistence by adding these lines to
<code class="language-plaintext highlighter-rouge">jupyterhub-config.py</code>:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">os</span>
<span class="c1"># user data persistence
# see https://github.com/jupyterhub/dockerspawner#data-persistence-and-dockerspawner
</span><span class="n">notebook_dir</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">environ</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'DOCKER_NOTEBOOK_DIR'</span><span class="p">)</span> <span class="ow">or</span> <span class="s">'/home/jovyan/work'</span>
<span class="n">c</span><span class="p">.</span><span class="n">DockerSpawner</span><span class="p">.</span><span class="n">notebook_dir</span> <span class="o">=</span> <span class="n">notebook_dir</span>
<span class="n">c</span><span class="p">.</span><span class="n">DockerSpawner</span><span class="p">.</span><span class="n">volumes</span> <span class="o">=</span> <span class="p">{</span> <span class="s">'jupyterhub-user-{username}'</span><span class="p">:</span> <span class="n">notebook_dir</span> <span class="p">}</span>
</code></pre></div></div>
<p>This way, even if we update the <code class="language-plaintext highlighter-rouge">jupyterlab</code> image and destroy the
associated containers, the user data will not be lost.</p>
<p>Note that the level of control on the user data is rather minimal in
this configuration. For example it is not possible to enforce
per-user disk quotas. It would be interesting to explore more
advanced uses of Docker volumes enabling complex usage patterns.</p>
<h2 id="block-networking">Block networking</h2>
<p>In contexts where you do not trust your users, it may be useful to
restrict networking from inside single-user containers. Indeed,
single-user servers are full fledged Unix servers, and can potentially
be used as relay nodes in network attacks.</p>
<p>A simple way to restrict networking is to ban any connection to the
outside world from single-user servers, letting them only communicate
with the Hub. This is easily accomplished using the <a href="https://docs.docker.com/v17.09/engine/userguide/networking/work-with-networks/">internal
flag</a>
for Docker networks. Edit <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> like this:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">services</span><span class="pi">:</span>
<span class="na">jupyterhub</span><span class="pi">:</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="na">DOCKER_NETWORK_NAME</span><span class="pi">:</span> <span class="s">${COMPOSE_PROJECT_NAME}_jupyter</span>
<span class="na">HUB_IP</span><span class="pi">:</span> <span class="s">jupyterhub_hub</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="na">default</span><span class="pi">:</span>
<span class="na">jupyter</span><span class="pi">:</span>
<span class="na">aliases</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">jupyterhub_hub</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.docker.network=${COMPOSE_PROJECT_NAME}_default"</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="na">jupyter</span><span class="pi">:</span>
<span class="na">internal</span><span class="pi">:</span> <span class="no">true</span>
</code></pre></div></div>
<p>More complex filtering can be achieved with iptables.</p>
<h2 id="conclusion">Conclusion</h2>
<p>We have described a full setup for a JupyterHub + Jupyter/JupyterLab
server targeted to a medium sized institution, with external
authentication and containerized components.</p>
<p>This setup can reasonably run on a personal computer, but is really
ideal for powerful servers, such as those found in research labs at
many universities.</p>
<p>Thanks to Docker compose, the whole configuration fits in a few
hundred lines, and is very easy to maintain and replicate.</p>
<p>We have been running an experiment for one year already, using a
similar setup to teach a class in University of Versailles. Since
September 2018 we are now running a larger scale deployment, used in
several classes. You can find the complete current configuration at
<a href="https://github.com/defeo/jupyterhub-docker">https://github.com/defeo/jupyterhub-docker</a>.</p>
<p>For comments, questions and suggestions, feel free to <a href="https://defeo.lu/">contact
me</a> directly.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:2" role="doc-endnote">
<p>See also
<a href="https://blog.jupyter.org/how-to-deploy-jupyterhub-with-kubernetes-on-openstack-f8f6120d4b1">https://blog.jupyter.org/how-to-deploy-jupyterhub-with-kubernetes-on-openstack-f8f6120d4b1</a>
for a Kubernetes deployment of JupyterHub on OpenStack. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Keynote at PyconFr Lille (France), Oct. 6, 20182018-10-06T00:00:00+00:00http://opendreamkit.org/2018/10/06/PyconFR<h2 id="main-goals">Main goals</h2>
<p>PyConFr is the annual gathering of the French Python community.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Viviane Pons was invited to be the opening keynote of the conference.</p>
<h2 id="event-summary">Event summary</h2>
<p>Viviane gave a 30 minute talk titled ``Science and Open-Source: what do we learn from each other’’. This was an occasion to discuss the many interactions between research and open-source development and highlight the role of projects such as OpenDreamKit.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The talk was well-received by the Python community and received much positive feedback.</p>
Software Tools for Mathematics Workshop Koper, Slovenia, 2018-09-24--2018-09-282018-09-24T00:00:00+00:00http://opendreamkit.org/2018/09/24/STM-Koper<h2 id="main-goals">Main goals</h2>
<p>The goal of the event was for mathematicians
to improve their coding skills and knowledge of mathematical software.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Samuel Lelièvre (OpenDreamKit member from UPSud) was one of
the organisers. The OpenDreamKit funds at Paris-Sud were used to fund travel
and stays of speakers.
The Faculty of Mathematics, Natural Sciences and Information Technologies at the University of Primorska
and Andrej Marušič Institute at the University of Primorska
provided coffee and snacks at coffee breaks as well as equipped lecture rooms.
The Slovenian Discrete and Applied Mathematics Society provided gifts for all participants.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event consisted in a two-day Software Carpentry
workshop (teaching participants the Unix shell, version control with Git,
and programming with Python) followed by three days on mathematical software
with mini-courses on CoCalc, GAP, SageMath and Jupyter, LaTeX and TikZ, as well
as talks on other mathematical software and databases, and on mathematical
research using software. A problem session allowed participants to submit
mathematical problems they cared about and thought software might help with,
a few of which were solved in the following days by other participants.</p>
<h2 id="demographics">Demographics</h2>
<p>The participants included a small number of bachelor
students, several PhD students and postdocs, professors and
researchers.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>At the Software Carpentry workshop,
the Unix shell was taught by Peter Palfrader and Jan Ber\v{c}i\v{c},
Git was taught by Olexandr Konovalov and Peter Palfrader,
and Python was taught by Julian R"{u}th and Nino Ba\v{s}i'{c}.</p>
<p>During the mathematical software part,
Olexandr Konovalov taught GAP,
Nino Ba\v{s}i'{c} taught LaTeX and TikZ,
Samuel Leli`{e}vre taught SageMath and CoCalc,
and Julian R"{u}th taught SageMath.</p>
<p>Many participants told the organisers, orally or by email, that this workshop
was transformative for them; often they felt they had passed some confidence
threshold: whereas before the conference they were interested in mathematical
software but unsure how to install and use them, they were now confident how
to do that, and felt they had the necessary resources to learn more.
One of the participants even commented that they were able to use what they learned
immediately in their research.
Some of the participants are involved in the organisation of the upcoming
European Congress of Mathematics (ECM 2020 Portoro\v{z}, Slovenia),
and expressed interest in highlighting open software for mathematics at the ECM.</p>
<p>Furthermore, a CoCalc instance was installed on University of Primorska servers
due largely to this event.</p>
<p>This event was conceived due to the success of the Software Tools for Mathematics
in Morelia earlier in 2018.
Based on the success of both workshops,
Samuel Leli`{e}vre and Katja Ber\v{c}i\v{c} (who participated in the organisation of both)
hope that the series will continue and are considering options for future installments.</p>
<p><img src="/public/images/events/stm-koper2019.jpeg =75%" alt="" />
Software Tools for Mathematics in Koper, Slovenia</p>
GAP Days Fall 2018 Siegen (Germany), 17--21 Sep 20182018-09-17T00:00:00+00:00http://opendreamkit.org/2018/09/17/GAPDaysFall2018<h2 id="main-goals">Main goals</h2>
<p>This was a meeting for experienced GAP developers and users
to discuss and contribute to the GAP project.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Michael Torpey and Markus Pfeiffer attended, and
worked on MathInTheMiddle and PackageManager.</p>
<p>OpenDreamKit participants: M.~Torpey, M.~Pfeiffer and D.~Pasechnik</p>
<h2 id="event-summary">Event summary</h2>
<p>This was a meeting for experienced developers and users
of GAP to discuss and influence the future development of GAP. There were
discussions and short talks on topics such as libGAP, Julia, and metapackages.
There were also various coding sprints for the GAP system and various GAP
packages.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>M.~Torpey and M.~Pfeiffer made considerable
progress on GAP’s MathInTheMiddle package, and after a discussion initiated at
the meeting, M.~Torpey started and made an initial release of PackageManager, a
new package for GAP.</p>
AMRS 2018: Conference on Applied Mathematics and Random Structures Birzeit, Palestine, 27th-30th of August 20192018-08-27T00:00:00+00:00http://opendreamkit.org/2018/08/27/AMRS<h2 id="main-goals">Main goals</h2>
<p>AMRS is a conference intended to foster collaborations between Palestinian and French universities or research centres;
to sustain the critical efforts of the mathematics community in Palestine to set up a doctoral school. AMRS provided an opportunity for dissemination of important and recent results in the field of Applied Mathematics and Statistics with Applications to Economy, Industry, and Science.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit was coorganizer for the conference and funded the travel costs
for Sage developer Thierry Monteil to deliver a Sage tutorial.</p>
<h2 id="event-summary">Event summary</h2>
<p>AMRS conference gathered 41 attendees around various fields of Applied mathematics: Statistics, Game Theory, Analytic Combinatorics,
Probability, Differ-ential or Difference Equations, Matrices,
Stochastic Optimal Control Problems, Partial Differential Equations,
Binary Search Trees, Ideals on Skew Matrices.
There were 13 talks given by 7 Palestinian professors, and 6 French); and 10 talks by students:</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>Thierry Monteil introduced the basic concepts of Sage and presented
many examples based on graphs algorithms to 24 students: 19 from
Birzeit– 3 from Al Quds– 1 from Polytechnic Palestine University
Hebron (PPU)– 1 from Malaya. 19 of the 24 students were female.</p>
<p><img src="/public/images/events/AMRS.png" alt="" />
AMRS2018</p>
JupyterCon 2018 New-York, August 21-252018-08-21T00:00:00+00:00http://opendreamkit.org/2018/08/21/jupytercon<p>## Main goals</p>
<p>JupyterCon is the major yearly conference
around the Jupyter technology.</p>
<p>## OpenDreamKit implication</p>
<p>Several OpenDreamKit members participated,
organized tracks, and disseminated OpenDreamKit’s outcome.</p>
Using Virtual Reality to Explore Mathematical Knowledge2018-08-20T00:00:00+00:00http://opendreamkit.org/2018/08/20/tgview3d<p>The Math-in-the-Middle (MitM) ontology and the system API theories in the MitM paradigm
are big theory graphs with thousands of nodes and edges. Understanding and interacting
with such large and complex objects is very difficult for humans.</p>
<p>The FAU group has conducted research into whether virtual reality technologies are helpful
for this task. We have presented a first working prototype at the
<a href="https://cicm-conference.org/2018/">Conference on Intelligent Computer Mathematics CICM 2018</a>
and the author: Richard Marcus - a master’s student at FAU has received a prize for best
presentation.</p>
<p>For sources, and videos video see the
<a href="https://github.com/UniFormal/TGView3D">GitHub Repository</a>
(<a href="https://github.com/UniFormal/TGView3D/releases/download/v1.0-beta.1/longVersion.mp4">long video</a>,
<a href="https://github.com/UniFormal/TGView3D/releases/download/v1.0-beta.1/shortVersion.mp4">short video</a></p>
CICM2018 -- Workshop Computer Algebra in the Age of Types RISC Linz, Hagenberg, Austria, 17th of August 20182018-08-17T00:00:00+00:00http://opendreamkit.org/2018/08/17/cicm-caat<h2 id="main-goals">Main goals</h2>
<p>We want to promote the use of types to compose existing
constructive and formal systems, enable formal checkability and correctness,
enable development of domain specific tools, provide access to machine verified
proofs and efficient computations, enable effective automated testing.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The workshop was organised by Markus Pfeiffer (UStan), his
travel costs and conference fees were paid by OpenDreamKit. The keynote lecture at the
workshop was given by Michael Kohlhase (FAU).</p>
<h2 id="event-summary">Event summary</h2>
<p>Following an excellent keynote lecture given by Michael
Kohlhase about the Math-in-the-Middle approach advocated by OpenDreamKit, we saw an
excellent usecase of types in Sebastian Posur’s talk about categorical
computation. Two tutorials, one on the programming language Idris by its creator
Edwin Brady (University of St Andrews), and one by Dennis Müller (FAU) on the
formal system MMT showed off what types can do for programming and mathematics
today.
Some of the workshop participants went to dinner together after the event.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>A lively discussion about the vision of using type
theory more in day-to-day mathematical programming, and for which purposes took
place during the event and also in the breaks and at dinner.
Some plans were discussed to use insights from the formal system MMT to drive
executable code in Idris.</p>
CICM2018 --- Conference on Intelligent Computer Mathematics RISC Linz, Hagenberg, Austria, 13th-17th of August 20182018-08-13T00:00:00+00:00http://opendreamkit.org/2018/08/13/cicm<h2 id="main-goals">Main goals</h2>
<p>Digital and computational solutions are becoming the prevalent means for the generation, communication, processing, storage and curation of mathematical information.
Separate communities have developed to investigate and build computer based systems for computer algebra, automated deduction, and mathematical publishing as well as novel user interfaces.
While all of these systems excel in their own right, their integration can lead to synergies offering significant added value.
The Conference on Intelligent Computer Mathematics (CICM) offers a venue for discussing and developing solutions to the great challenges posed by the integration of these diverse areas.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The program chair was Florian Rabe (FAU).
Workshops were co-organized by Michael Kohlhase (FAU) and Markus Pfeiffer (SA).
There were no costs to OpenDreamKit other than travel costs for OpenDreamKit members.</p>
<h2 id="event-summary">Event summary</h2>
<p>CICM featured 3 invited talks, 3 days of presentations (including a system demo session), a doctoral program, and 6 workshops.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The OpenDreamKit participants presented several papers that came out of OpenDreamKit.
This triggered a number of discussions with researchers from adjacent fields in computer science as well as a few mathematicians.
Richard Markus was awarded the Best Demo Award for his 3-dimensional theory graph viewer — a tool developed in OpenDreamKit.</p>
<p>Bruno Buchberger gave an invited talk on the future of a global digital mathematical library, which prompted discussions on future grant proposals in this direction.</p>
<p>Bruce Miller and Adri Olde Daalhuis presented the DLMF, a library of special mathematical functions.
With them, Rabe and Kohlhase (FAU) designed an integration of the DLMF with the MitM ontology developed in OpenDreamKit WP 6.</p>
ICMS 2018 -- Session 11: Backtrack search techniques in groups and combinatorics South Bend, USA, 25th July 20182018-07-24T00:00:00+00:00http://opendreamkit.org/2018/07/24/icms-partbacktrack<h2 id="main-goals">Main goals</h2>
<p>We invited experts from AI, combinatorics, computational group theory and
related areas with the aim of sharing and exchanging ideas, problems,
results and implementations.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Organised by Markus Pfeiffer (UStan), and his
colleague Christopher Jefferson. Some of Markus Pfeiffers work improves the
software tools used in backtrack search in GAP.</p>
<p>OpenDreamKit funded Markus Pfeiffer’s travel to ICMS and the conference fee, and the
accommodation for the session participants Paula Hähndel and Wilf Wilson (both
from Halle, Germany).</p>
<h2 id="event-summary">Event summary</h2>
<p>We saw a great selection of talks from experts around
the world on the current state of the art in backtrack search: Chris Jefferson
on partition backtrack, Robin Candy (PhD student of Brendan McKay) on canonical
labelling subgroups under conjugacy, Paula Hähndel on using Orbital Graphs
(which are used in partition backtrack and canonical labelling), Anton Betten
and Matthias Koch on ``Snakes and Ladders’’, Abdullah Al-Azemi (Kuwait
University) on HPC-Techniques in backtrack search, and Markus Pfeiffer on
backtrack search in the free group.</p>
<p>In particular, two popular techniques <code class="language-plaintext highlighter-rouge">Partition Backtrack'' and</code>Snakes and
Ladders’’ were discussed, and for the first time in recent decades experts in
this are were able to sit together and learn from each other how to combine the
two approaches to an even more powerful one.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>New collaborations were setup, Robin Candy is going
to visit Halle (Germany) and St Andrews later this year to work on improving
backtrack search for canonical labels under subgroup conjugacy. Wilf Wilson from
Halle who is now working on a joint project with Chris Jefferson, Rebecca
Waldecker and Markus Pfeiffer was able to meet experts and get an introduction
to the field.</p>
ICMS 2018 -- Session 14: Towards Composable Mathematical Software South Bend, Notre Dame, 25th July 20182018-07-24T00:00:00+00:00http://opendreamkit.org/2018/07/24/icms-mitm<h2 id="main-goals">Main goals</h2>
<p>The aim of this session is to provide a forum for
developers and users of mathematical software with an interest in composablity
and interoperability of, and knowledge and data exchange between systems, to
share experiences, solutions, and a vision for the future.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The OpenDreamKit participants Markus Pfeiffer, Nicolas Thiery,
and Florian Rabe organised this session, and invited the speakers.</p>
<p>Travel costs for Markus Pfeiffer were covered by OpenDreamKit, as well as accommodation
costs for Sebastian Gutsche.</p>
<h2 id="event-summary">Event summary</h2>
<p>In a single session we saw talks by Sebastian Gutsche
about integrating GAP and Julia, William Stein about the challenges with
SageMath integrating a wealth of software, Michael Kohlhase about the
Math-In-the-Middle approach advocated by OpenDreamKit and Tim Daly about proving the
computer algebra system axiom sane.</p>
<p>The session was very well attended which points at the demand for this
technology. The ratio of talk submissions to attendees also hints at the fact
that not many people have the resources to address the challenges brought up by
the session topic.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The session enabled a lively discussion about the
challenges of interfacing mathematical software correctly and so achieved one of
its goals.</p>
Live online slides with SageMath, Jupyter notebooks, RISE and Binder2018-07-23T00:00:00+00:00http://opendreamkit.org/2018/07/23/live-online-slides-with-sagemath-jupyter-rise-binder<h2 id="scenario">Scenario</h2>
<p>Viviane is writing slides for a talk; she would like to include live
SageMath computations in her slides. She also would like to publish
her slides online, letting anyone – during and after her talk –
browse the slides and play around with the provided examples.</p>
<p>Note: most of this blog can be adapted for any other computational
system that can be run in Jupyter like, for mathematics, GAP, Pari/GP,
or Singular.</p>
<h2 id="some-instances">Some instances</h2>
<ul>
<li>
<p><a href="https://github.com/VivianePons/public-notebooks/blob/master/TamariIntervalPosets/2018_ICERM_Presentation.ipynb">The Rise contact involution on Tamari Intervals</a>,
Viviane Pons
(<a href="https://github.com/VivianePons/public-notebooks/">GitHub repository</a>,
<a href="https://mybinder.org/v2/gh/VivianePons/public-notebooks/master?filepath=TamariIntervalPosets/2018_ICERM_Presentation.ipynb">Live slides</a>)</p>
<p><a href="/public/images/use-cases-sage-jupyter-binder-rise2.png"> <img src="/public/images/use-cases-sage-jupyter-binder-rise2.png" alt="Live slides with SageMath, Jupyter, and RISE" class="small-screenshot" /> </a></p>
</li>
<li>
<p><a href="https://github.com/nthiery/harmonic-modules/blob/master/talk.ipynb">Computing huge subspaces of polynomials: Symmetries to the rescue!</a>,
Nicolas M. Thiéry
(<a href="https://github.com/nthiery/harmonic-modules">GitHub repository</a>,
<a href="https://mybinder.org/v2/gh/nthiery/harmonic-modules/master?filepath=talk.ipynb">Live slides</a>)</p>
<p><a href="/public/images/use-cases-sage-jupyter-binder-rise.png"> <img src="/public/images/use-cases-sage-jupyter-binder-rise.png" alt="Live slides with SageMath, Jupyter, and RISE" class="small-screenshot" /> </a></p>
</li>
</ul>
<h2 id="suggestion-of-solution">Suggestion of solution</h2>
<p>We will use the above slides from the repository
https://github.com/nthiery/harmonic-modules as running example.</p>
<h3 id="components-involved">Components involved</h3>
<ul>
<li>
<p><a href="https://sagemath.org">SageMath</a> (Sage for short) is a Python based
general purpose system for computational mathematics.</p>
</li>
<li>
<p><a href="https://jupyter.org">Jupyter</a> is a web application for authoring and
sharing documents that contain live code, equations, visualizations
and narrative text.</p>
</li>
<li>
<p><a href="http://rise.readthedocs.io/">RISE</a> is a Jupyter extension
to render Jupyter notebooks as slides, based on the popular HTML
presentations framework <a href="https://revealjs.com/">reveal.js</a>.</p>
</li>
<li>
<p><a href="https://mybinder.org">Binder</a> is a service that lets any user on
the internet define, run, and share temporary virtual environments
equipped with an arbitrary software stack
(<a href="https://jupyter.org/try">examples</a>).</p>
</li>
</ul>
<h3 id="installation">Installation</h3>
<p>Install Jupyter’s <a href="http://rise.readthedocs.io/">RISE</a> extension in Sage.
From the terminal:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sage -pip install RISE
</code></pre></div></div>
<p>Notes:</p>
<ul>
<li>
<p>The instructions above assume that the Sage command is available as
<code class="language-plaintext highlighter-rouge">sage</code>; otherwise adapt the command accordingly.</p>
</li>
<li>
<p>The instructions above assume that you have write access to the Sage
installation.</p>
</li>
<li>
<p>The instructions above have a bit of additional complexity because
they require accessing <code class="language-plaintext highlighter-rouge">pip</code> and <code class="language-plaintext highlighter-rouge">jupyter</code> from the Sage
distribution; in general, one would just do:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> pip install RISE
</code></pre></div> </div>
</li>
<li>
<p>The first instruction may fail complaining about OpenSSL. This will
happen when Sage has not been compiled with SSL support which is
needed for the pip package manager which, alas, happens too often.
The <a href="https://doc.sagemath.org/html/en/installation/source.html#section-notebook-ssl">installation instructions</a>
may help fix that.</p>
</li>
</ul>
<h3 id="authoring-and-showing-slides">Authoring and showing slides</h3>
<p>Open the Jupyter notebook that you want to turn into slides – or
create a new one – ; we will assume it’s called <code class="language-plaintext highlighter-rouge">talk.ipynb</code>.
Activate the <code class="language-plaintext highlighter-rouge">Slideshow</code> metadata:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>View -> Cell Toolbar -> Slideshow
</code></pre></div></div>
<p>Choose the role each cell should play: for example start a new
slide (Slide), add a fragment to the current slide (Fragment), or
don’t show up (Note).</p>
<p>Activate the slideshow by clicking in the corresponding button in the
toolbar.</p>
<p>For further details on authoring, showing, and configuring the
slideshow, see the <a href="http://rise.readthedocs.io/">RISE documentation</a>.</p>
<h3 id="publishing-the-slides-on-a-web-page">Publishing the slides on a web page</h3>
<p>This assumes that you have access on a web server to publish HTML
files that embed javascript.</p>
<p>The slides can be converted to static HTML pages with embedded
javascript with:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>jupyter nbconvert talk.ipynb --to slides
</code></pre></div></div>
<p>The produced file <code class="language-plaintext highlighter-rouge">talk.slides.html</code> can then be copied to the web
server. Readers will be able to see and navigate the slides; however
they won’t be able to modify and run the examples.</p>
<h3 id="publishing-live-slides-on-binder">Publishing live slides on Binder</h3>
<p>Add the notebook to a public git repository, for example on github.</p>
<h4 id="specifying-software-dependencies">Specifying software dependencies</h4>
<p>Add metadata to that git repository to specify the software required
to run the computations. For SageMath, you can add the following
<code class="language-plaintext highlighter-rouge">Dockerfile</code> at the root of the repository:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>FROM sagemath/sagemath:8.7
# Make sure the contents of the repository is in ${HOME}
COPY --chown=sage:sage . ${HOME}
</code></pre></div></div>
<h4 id="specifying-additional-software-dependencies-optional">Specifying additional software dependencies (optional)</h4>
<p>The sagemath base docker image contains the required software needed
to be compatible with binder. If the notebook uses additional software
in addition to SageMath, they need to be specified as well in the
Dockerfile. For example, if you are using RISE slides, add this line:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>RUN sage -pip install RISE
</code></pre></div></div>
<p>In general a Python package available from <a href="https://pypi.org/">pypi</a>
can be installed with:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>RUN sage -pip install <package name>
</code></pre></div></div>
<p>Sage packages can be installed with:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>RUN sage -i <package name>
</code></pre></div></div>
<p>Debian/Ubuntu packages can be installed with:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>RUN apt-get update && apt-get install -y <package name>
</code></pre></div></div>
<p>Hence, to have dot2tex graphics, you would use:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>RUN apt-get update && apt-get install -y graphviz
RUN sage -pip install dot2tex
</code></pre></div></div>
<p>See also the section
<a href="https://mybinder.readthedocs.io/en/latest/tutorials/dockerfile.html#preparing-your-dockerfile">Preparing your docker file</a>
from the <a href="https://mybinder.readthedocs.io/">binder documentation</a>.</p>
<h4 id="accessing-the-notebook-in-binder">Accessing the notebook in Binder</h4>
<p>Go to <a href="https://mybinder.org">mybinder.org</a><code class="language-plaintext highlighter-rouge">, specify the address of
the git repository (here https://github.com/nthiery/harmonic-modules/
) and the name of the file (here </code>talk.ipynb`). Click launch, and
voilà!</p>
<p>Note: the SageMath docker image is relatively big. Therefore the first
time the slides are accessed, or if it has not been accessed in a
while, Binder will typically take a couple minutes to launch: it
indeed needs to fetch the base SageMath image, build the new image
containing the Jupyter extensions and the notebook, and then copy this
image around between it’s various machines (that building the image,
that storing them, and those running them).</p>
<h4 id="publishing-the-binder-link">Publishing the Binder link</h4>
<p>From Binder’s main page, you can get a URL to access directly the live
notebook, or even a badge to copy-paste in your web page.</p>
<h2 id="alternatives">Alternatives</h2>
<p>Instead of publishing on Binder, one may author the slides on
<a href="http://cocalc.com">cocalc</a>, and use its publish feature.
(TODO: double check that RISE works on cocalc).</p>
<p>Instead of using a Jupyter notebook, one may author in HTML (or some
format that can be converted to HTML), and use
<a href="https://github.com/minrk/thebelab">ThebeLab</a> or
[SageCell][http://sagecell.sagemath.org/]to enable live computations.</p>
<h2 id="advantages-and-drawbacks">Advantages and drawbacks</h2>
<p>Traditionally, most mathematicians author their slides using
LaTeX+Beamer, and display and publish them in PDF format.</p>
<ul>
<li>
<p>Authoring slides with Jupyter+RISE tends to be more lightweight</p>
</li>
<li>
<p>LaTeX+Beamer gives fine control on the layout for authoring complex
slides; some fine control can be achieved with Jupyter+RISE, using
explicit html+css, but this is not as convenient.</p>
</li>
<li>
<p>One may include live computations</p>
</li>
<li>
<p>One may easily include multimedia documents</p>
</li>
<li>
<p>One may edit the slides on the fly</p>
</li>
<li>
<p>The semantic structure tends to get lost in PDF files, which goes in
the way of accessibility. On the other hand, HTML based documents
tend to support well web-based accessibility tools like screen
readers.</p>
</li>
</ul>
<h2 id="to-do">To do</h2>
<ul>
<li>Estimate the number of instances;</li>
<li>Provide a template.</li>
</ul>
<h2 id="time-and-expertise-required">Time and expertise required</h2>
<p>Assuming Viviane is familiar with Jupyter, writing her first slides
using RISE will take a few minutes and mastering it a few hours.</p>
<p>Assuming Viviane is familiar with version control and Jupyter (basic
lab skills taught at <a href="http://software-carpentry.org/">Software
Carpentry</a>, and the required software
required is packaged (conda, debian, docker container, …), the
publishing part could take two hours the first time, and minutes later
on (typically by having a preconfigured repository where one stores
all of one’s presentations)..</p>
<h2 id="whats-new-since-opendreamkit-started">What’s new since OpenDreamKit started</h2>
<ul>
<li>Apparition of <a href="http://mybinder.org">Binder</a>;</li>
<li>Expansion of the <a href="http://jupyter.org">Jupyter</a> technology;</li>
<li>Apparition of <a href="http://rise.readthedocs.io/">RISE</a></li>
<li>Better packaging and interfacing of math software.</li>
</ul>
<h2 id="opendreamkit-contribution">OpenDreamKit contribution</h2>
<ul>
<li>Development and contributions to Jupyter interfaces (kernels) for
math software (GAP, Pari/GP, SageMath, Singular) and C++;
see <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/96">D.47</a>.</li>
<li>Contributions to the packaging of math software (GAP, Pari/GP,
SageMath, Singular, …); see
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/58">D3.1</a>
and
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/59">D3.10</a>;</li>
<li>Early adoption of Binder;</li>
<li>Contributions to the
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/238">deployment of new Binder instances</a>;</li>
<li>Advertising, training, providing a template (TODO!), …</li>
</ul>
GAP Tutorial at the 20th Postrgaduate Group Theory Conference St Andrews, Jul. 16 -- 20, 20182018-07-23T00:00:00+00:00http://opendreamkit.org/2018/07/23/PGTC<h2 id="main-goals">Main goals</h2>
<p>The Postrgaduate Group Theory Conference (PGTC) is an annual meeting of
young researchers in group theory. The GAP Tutorial was organised as an optional satellite event.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Although OpenDreamKit was not formally involved in this event, it was
organised by Olexandr Konovalov, and was used to promote the project and its outcomes.</p>
<h2 id="event-summary">Event summary</h2>
<p>The GAP Tutorial was intended for beginners without requiring
any prior knowledge of the GAP system. The first part on Monday July 16th covered
the Software Carpentry lesson <code class="language-plaintext highlighter-rouge">Programming with GAP'' and was delivered in the live
coding style. The second part on Friday July 18th included a talk on debugging and
profiling in GAP by Christopher Jefferson (St Andrews) and a talk
</code>Using GAP Effectively: Some Tips and Pitfalls’’ by Olexandr Konovalov. The latter
included the demonstration of the GAP Jupyter interface as included in the GAP 4.9.2
release (June 2018). Using Jupyter notebooks for this talk contained in the repository
at https://github.com/alex-konovalov/gap-teaching, attendees were able to launch
them on Binder and try to work with GAP in Jupyter themselves. Several participants
received help with installing the latest release of GAP and configuring its Jupyter
kernel after the tutorial and left the event with fully usable working environment.</p>
ICM 2018 - Computational micromagnetics with JOOMMF workshop San Francisco, CA, USA, 15-20 July 20182018-07-15T00:00:00+00:00http://opendreamkit.org/2018/07/15/ICM<h2 id="main-goals">Main goals</h2>
<p>In this workshop we taught the participants to run micromagnetic simulations using JOOMMF.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>JOOMMF was developed as a part of the OpenDreamKit project and two participants from the OpenDreamKit were present to deliver the workshop (Marijan Beg, and Ryan A. Pepper). The workshop was fully funded from the OpenDreamKit project and the total costs were 7259.58 Euros.</p>
<h2 id="event-summary">Event summary</h2>
<p>We started this 3.5 hours workshop by helping the participants to install all required software on their laptops. After that we introduced some physics basics of micromagnetics as well as theoretically explained how does a micromagnetic simulation work. Then we went through the tutorials and explained all individual commands of JOOMMF required for participants to complete the exercises. Finally, participants were working on an exercise and managed to reproduce the results from already published work. We had a lot of opportunity to talk to the existing and potential users and get feedback from them.</p>
<h2 id="demographic">Demographic</h2>
<p>We had around 150 participants, but due to the data protection we could not obtain any demographics data from them.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>During the workshop we received the feedback from the participants about our Python interface to OOMMF as well as gained experience which helped us to structure future workshops.</p>
<p><img src="/public/images/events/ICMPhoto1.jpg" alt="" /></p>
<p><img src="/public/images/events/ICMPhoto2.jpg" alt="" /></p>
<p><img src="/public/images/events/ICMPhoto3.jpg" alt="" />
JOOMMF workshop at ICM2018 conference in San Francisco, CA, USA</p>
Sage Days at Icerm ICERM Providence (United States), Jul. 23 -- 27, 20182018-07-14T00:00:00+00:00http://opendreamkit.org/2018/07/14/ICERM<h2 id="main-goals">Main goals</h2>
<p>These Sage Days were focused on Combinatorics and Representation Theory. They were planned one week after the annual gathering of the algebraic combinatorics community at FPSAC 2018 (Hanover, New Hampshire).</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The event was not organized nor funded by OpenDreamKit. Nevertheless, it featured two talks by OpenDreamKit members (Nicolas Thiéry and Viviane Pons) and OpenDreamKit paid the travel expenses of some of the French participants.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event featured many presentations focusing on the link between Sage development and research. It also allowed for some free coding time and discussions. Nicolas Thiéry and Viviane Pons both gave research presentations including some Sage demo. It was an occasion to promote the Sage docker and Jupyter live slides which is one of the use cases presented on OpenDreamKit website.</p>
<h2 id="demographic">Demographic</h2>
<p>Over 18 planed presentations, 9 were given by women.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This event was a good occasion for OpenDreamKit to connect the recent developments to the actual community and research. Beside Sage development, many participants discovered docker during the event and showed lots of interest.</p>
Modular Knowledge Oxford, 2018-07-132018-07-13T00:00:00+00:00http://opendreamkit.org/2018/07/13/tetrapod<h2 id="main-goals">Main goals</h2>
<p>Expanding on the Tetrapod workshop at the conference on intelligent computer mathematics (CICM) 2016, this workshop brings together researchers from a diverse set of research areas in order to create a universal understanding of the challenges and solutions regarding highly structured knowledge bases.
Of particular interest are foundational principles such as theory graphs and colimits, interchange languages and module systems, languages and tools for representing, reasoning, computing, managing, and documenting modular knowledge bases.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>This was co-organized by OpenDreamKit participants together with Jacques Carette, in order to connect with researchers in adjacent fields.
Cost to OpenDreamKit was limited to travel cost.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event had 6 invited speakers, each of which introduced a specific topic.
This consisted of a 15-minute presentation, which was followed by a 30-minute free discussion session.</p>
<h2 id="demographic">Demographic</h2>
<p>Researchers in logic, mostly male, mostly senior.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>No direct impact on OpenDreamKit tasks and deliverables but it informed \WPref{dksbases} in general.</p>
Modularity and composability in Mathematical Computation2018-07-13T00:00:00+00:00http://opendreamkit.org/2018/07/13/ModularityInMathematicalComputation<h1 id="modularity-and-composability-in-mathematical-computation">Modularity and composability in Mathematical Computation</h1>
<p>Nicolas M. Thiéry et al.</p>
<p>Tetrapod: Modular Knowledge workshop</p>
<p>FLoC 2018, Oxford, July 13th of 2018</p>
<hr />
<h2 id="abstract">Abstract</h2>
<p>Over the last decades, a huge amount of computational software was
developed for pure mathematics, in particular to support research and
education. As for any complex ecosystem of software components, the
ability to compose them has a multiplier effect on the expressive
power, flexibility, and range of applications.</p>
<p>The purpose of this session is to share experience on the many
barriers to composability in computational mathematics, and how they
are being tackled. Of particular interest will
be the exploitation of knowledge to leverage some of the barriers.
feedback from neighbor fields (proofs, data, knowledge) will be most
welcome.</p>
<hr />
<h2 id="a-collaborative-document">A collaborative document</h2>
<p>This document, hosted on: https://hackmd.io/IZESWGzKSOCKJiC_tIP3pw,
benefited from edits and comments from the participants of the
workshop, including:</p>
<ul>
<li>Georges Gonthier (Inria Saclay)</li>
<li>Florian Rabe (FAU Erlangen-Nürnberg, LRI, Univ. Paris Sud)</li>
<li>Dennis Müller (FAU Erlangen-Nürnberg)</li>
<li>Jeremy Gibbons (University of Oxford), local observer and gate-crasher</li>
<li>Jacques Carette (McMaster University) System designer and developper</li>
</ul>
<hr />
<h2 id="composability-for-math-software">Composability for Math software?</h2>
<p>For the sake of simplicity, we will mostly focus on two aspects of
composability: the ability to transfer data and run procedures across
“systems” (two different software, two instances of the same software
on different machines, two libraries within a system, …).</p>
<hr />
<h3 id="data-transfers">Data transfers</h3>
<ul>
<li>
<p><strong>Goal</strong>: Transfer an object O from system <code class="language-plaintext highlighter-rouge">A</code> to system <code class="language-plaintext highlighter-rouge">B</code></p>
</li>
<li>
<p><strong>Requirements</strong>:</p>
<ul>
<li>
<p><strong>A communication channel</strong>: shared memory, disk, pipe, (web)socket, …</p>
</li>
<li>
<p><strong>A communication protocol</strong></p>
</li>
<li>
<p><strong>Serialization / Deserialization</strong>: conversion to/from a string of bytes</p>
</li>
<li>
<p><strong>A format specification</strong>; e.g. XML (syntax) + OpenMath content dictionary (semantic)</p>
</li>
<li>
<p><strong>Format conversion</strong></p>
<ul>
<li><strong>syntax</strong>: e.g.: JSON <-> XML</li>
<li><strong>adaptation</strong>: <code class="language-plaintext highlighter-rouge">DihedralGroup(4)</code> <-> <code class="language-plaintext highlighter-rouge">DihedralGroup(8)</code></li>
<li><strong>change of representation (codec)</strong>: recursive <-> sparse polynomial</li>
</ul>
<p>Idealy: applying a <em>theory morphism</em> to guarantee semantic preservation</p>
</li>
</ul>
</li>
</ul>
<hr />
<h3 id="procedure-calls">Procedure calls</h3>
<ul>
<li><strong>Goal</strong>: From <code class="language-plaintext highlighter-rouge">B</code> request the computation <code class="language-plaintext highlighter-rouge">f(a,b,c)</code> in <code class="language-plaintext highlighter-rouge">A</code></li>
<li><strong>Requirements</strong>:
<ul>
<li><strong>Data transfers</strong>: send <code class="language-plaintext highlighter-rouge">a</code>,<code class="language-plaintext highlighter-rouge">b</code>,<code class="language-plaintext highlighter-rouge">c</code> to <code class="language-plaintext highlighter-rouge">A</code>, receive back the result</li>
<li><strong>Procedure call</strong>: bindings, remote procedure calls, …</li>
<li><strong>An API specification</strong>: syntax and semantic of <code class="language-plaintext highlighter-rouge">f</code> in <code class="language-plaintext highlighter-rouge">A</code>?</li>
<li>
<p><strong>Adapting the API in B to the API in A</strong>: <code class="language-plaintext highlighter-rouge">Size(A)</code> <-> <code class="language-plaintext highlighter-rouge">A.cardinality()</code></p>
<p>Idealy: applying a <em>theory morphism</em> to guarantee semantic preservation</p>
</li>
</ul>
</li>
</ul>
<hr />
<h2 id="barriers-faced-by-the-computational-pure-maths-community">Barriers faced by the computational pure maths community?</h2>
<hr />
<h3 id="semantic-barriers">Semantic barriers</h3>
<ul>
<li>
<p><strong>Many kinds of objects</strong>:</p>
<p>E.g. thousands in Sage, compared to “matrices of floats” in Matlab</p>
</li>
<li>
<p><strong>Many data representations</strong>, with very different algorithmic complexity</p>
<p>E.g. for polynomials: sparse, dense, recursive, straight line
program, evaluation, …</p>
</li>
<li>
<p>Objects at different levels of abstraction</p>
</li>
<li>
<p><strong>Few core concepts</strong></p>
<p>addition, multiplication commutativity</p>
</li>
<li>
<p>… but <strong>many interesting ways to combine them</strong>:</p>
<p>Groups, Fields, graded commutative algebras, …</p>
</li>
<li>
<p><strong>Many interactions across different areas of mathematics</strong></p>
<p>Hey, that’s the point of maths!</p>
</li>
<li>
<p>A <strong>large variety of use cases</strong></p>
<p>speed maniacs, flexibility fans, composability devouts</p>
</li>
</ul>
<hr />
<h4 id="the-good-news">The good news</h4>
<p>We can afford not to be completely formal; <strong>best effort is usually ok</strong></p>
<hr />
<h3 id="social-barriers">Social barriers</h3>
<ul>
<li><strong>Closed science</strong> and misc <strong>Montaigu-Capulet fights</strong>
<ul>
<li>“don’t want/dare to share my code”</li>
<li>“won’t use code from XXX”</li>
<li>“won’t fund contributions to a US-born project”</li>
</ul>
<p>Come on, we are in 2018!</p>
</li>
<li>
<p><strong>Tight man power resources</strong></p>
<ul>
<li>Hundreds of thousands use computer algebra</li>
<li>hundreds implement it;</li>
<li>a handful are paid full time for it (+ a hundred in commercial systems)</li>
</ul>
</li>
<li>
<p><strong>Mastering math and computer science</strong></p>
</li>
<li><strong>High level of math specialization required when writing code</strong>
<ul>
<li>thousands of people use Gröbner bases</li>
<li>a handful know how to implement them right</li>
</ul>
</li>
<li>Pieces of software that took decades to develop
<ul>
<li>We can’t afford to reimplement them</li>
<li>
<p>We can’t afford to not reimplement them:</p>
<p>learn from one’s mistakes, benefit from technology advances</p>
</li>
</ul>
</li>
<li>A fragmented community</li>
</ul>
<hr />
<h3 id="technical-barriers">Technical barriers</h3>
<ul>
<li>
<p><strong>Systems written in different languages / idioms</strong></p>
<p>C, C++, Python 2/3, Java, Javascript, Julia, GAP, Singular,
Macaulay, Maple, Mathematica, MuPAD, Magma, Axiom/Aldor, Haskell, ML, Agda, …</p>
</li>
<li>
<p><strong>Systems written using different frameworks / API</strong></p>
</li>
<li>
<p><strong>Packaging barriers</strong></p>
</li>
<li>
<p>Tendency for <strong>large old systems</strong>: 10-30 years, $10^6$ lines of code</p>
<ul>
<li>lots of <strong>technical debt</strong>; <strong>slow evolution</strong></li>
<li><strong>Lack of modularity</strong>, <strong>large dependencies</strong>
E.g. having to install SageMath to use just a few of its lines</li>
</ul>
</li>
</ul>
<hr />
<h2 id="tendencies">Tendencies</h2>
<ul>
<li>
<p>Specialized projects – often with a dedication to speed – make
efforts to be developed as low-level libraries (e.g. C, C++).</p>
</li>
<li>
<p>Large systems make efforts to be usable as <strong>libraries</strong> and not
just <strong>standalone systems</strong>. Typically enabling low-level calls at
the C level for fine-grained interactions.</p>
</li>
<li>
<p>Modern languages make it easier to call libraries written in other
languages (e.g. C, C++).</p>
</li>
<li>Everybody makes efforts for better software development practices:
<ul>
<li><strong>development workflows</strong>, <strong>publication</strong>, <strong>packaging</strong>,
<strong>distribution</strong>, …</li>
<li>There even is some funding for it (OpenDreamKit, OSCAR)</li>
</ul>
</li>
<li><strong>Examples</strong>:
<ul>
<li>Integration of GAP, Singular, … in SageMath</li>
<li>Symmetric integration Julia-GAP (OSCAR project)</li>
<li>Integration of linbox, libsemigroups, gmp, MPIR in GAP, SageMath, Singular</li>
</ul>
</li>
<li><strong>Caveat</strong>: takes care of <strong>binding</strong> not yet of <strong>adapting</strong></li>
</ul>
<hr />
<h3 id="knowledge-for-in-system-modularity">Knowledge for in-system modularity</h3>
<ul>
<li>Many systems (e.g. Axiom, Sage, GAP, …) embed mathematical
knowledge (partial formalization) to:
<ul>
<li>structure their library</li>
<li>model closely the mathematics</li>
<li>define modular API’s</li>
<li>enable composing its building blocks</li>
</ul>
</li>
<li>The few-concepts-many-combinations feature goes in the way of
traditional computer-science best practices: <strong>decoupling</strong>,
<strong>separation of concerns</strong>. This led to some <strong>innovative designs</strong>:
<ul>
<li>Bespoke method-resolution mechanism (GAP)</li>
<li>Object orientation using categories (Axiom, MuPAD)</li>
<li>Standard Python Object Orientation + mechanization of the
generation of the large class hierarchy (SageMath)</li>
</ul>
</li>
</ul>
<hr />
<h3 id="knowledge-for-cross-system-modularity">Knowledge for cross-system modularity</h3>
<ul>
<li>
<p>A not so successful attempt: OpenMath</p>
</li>
<li>
<p>A tentative reboot with OpenDreamKit’s Math-in-the-Middle approach</p>
<ul>
<li>Uncovering the knowledge already embedded in each systems</li>
<li>Aligning it through a central ontology</li>
<li>Exploiting this for automatic generation of adapters</li>
</ul>
</li>
</ul>
<hr />
<h4 id="a-tension-when-attempting-to-modularize-large-systems">A tension when (attempting to) modularize large systems</h4>
<ul>
<li>The “library of books” approach (e.g. GAP)
<ul>
<li>Fosters small independent packages</li>
<li>Credits package authors</li>
<li>Leads to duplication (analogy: overlapping books written by
different authors, with different notations)</li>
</ul>
</li>
<li>The “wikipedia approach” (e.g. SageMath)
<ul>
<li>Fosters joint ownership, harmonization of API’s</li>
<li>Fosters cross-subject interrelations; hey, it’s math!</li>
<li>Leads to one huge tightly coupled blob</li>
</ul>
</li>
</ul>
<hr />
<h2 id="open-discussion">Open discussion</h2>
<ul>
<li>
<p>How to foster a sustainable and agile ecosystem where (sub)components have their own life cycle, explore new approaches, compete, and happily die when no more relevant.</p>
</li>
<li>
<p>What other barriers do you identify in Computational Math?</p>
</li>
<li>
<p>How do the barriers differ from other areas</p>
</li>
<li>
<p>Other case studies? solutions? approaches?</p>
</li>
</ul>
<hr />
<h2 id="appendix-some-case-studies-and-tentative-solutions">Appendix: some case studies and tentative solutions</h2>
<hr />
<h3 id="openmath">OpenMath</h3>
<ul>
<li><strong>Goal</strong>: standardized serialization format to exchange data across systems</li>
<li><strong>Syntax</strong>: XML, json, binary</li>
<li><strong>Semantic</strong>: defined by content dictionaries (CD)</li>
<li><strong>Hard problem</strong>: agreeing on CD’s for polynomials was already hard; scaling?</li>
</ul>
<p>After twenty years, little adoption</p>
<hr />
<h3 id="the-math-in-the-middle-approach">The Math-in-the-Middle approach</h3>
<ul>
<li><strong>Goals</strong>:
<ul>
<li>Separate <strong>serialization</strong> from <strong>adaptation</strong></li>
<li>Separate <strong>binding</strong> from <strong>adaptation</strong></li>
<li>Separate the treatment of each system</li>
</ul>
</li>
<li><strong>Approach</strong>:
<ul>
<li>
<p>A central math ontology</p>
</li>
<li>
<p>Formalize each systems by aligning to the central ontology</p>
</li>
</ul>
</li>
</ul>
<hr />
<h3 id="categories-in-gap-axiom-mupad-sagemath">Categories in GAP, Axiom, MuPAD, SageMath</h3>
<ul>
<li>
<p><strong>Barrier</strong>: compose building blocks <code class="language-plaintext highlighter-rouge">A1</code> and <code class="language-plaintext highlighter-rouge">A2</code> into modular generic code <code class="language-plaintext highlighter-rouge">B</code>: <code class="language-plaintext highlighter-rouge">f(a1, a2)</code></p>
</li>
<li>
<p><strong>Requirements</strong>: <strong>uniform modular API</strong> + <strong>method resolution mechanism</strong></p>
<p>Remember: few core concepts, hundreds of interesting combinations</p>
</li>
<li>
<p><strong>Approaches</strong>:</p>
<ul>
<li>
<p>Axiom, MuPAD: OOP with large hierarchies of abstract classes (categories)</p>
</li>
<li>SageMath:
<ul>
<li>same, but even larger</li>
<li>based on standard OOP (Python) + infrastructure to scale</li>
<li>embed semantic knowledge at single point of truth;
exploit it to automatically generate the class hierarchy</li>
</ul>
</li>
<li>GAP: bespoke method resolution mechanism</li>
</ul>
</li>
</ul>
<hr />
<h3 id="case-study-libsemigroups-a-library-for-computing-with-semigroups">Case Study: libsemigroups:, a library for computing with semigroups</h3>
<p>Author: James B. Mitchel et al.</p>
<ul>
<li><strong>Problems</strong>
<ul>
<li>Speeding GAP’s semigroup package while increasing its availability</li>
<li>Use a low-level library <code class="language-plaintext highlighter-rouge">A</code> from a system <code class="language-plaintext highlighter-rouge">B</code> written in a different language</li>
</ul>
</li>
<li><strong>Approach</strong>
<ul>
<li>Rewrite as a standalone templated C++ library</li>
<li>API: many iterations until converging to a natural API (no memory management, …)</li>
<li>Bindings: from Python: on-the-fly binding using cppyy (also tried Cython, Pybind11, …)</li>
<li>Adaptation: a thin layer for now; could use alignments</li>
</ul>
</li>
</ul>
<hr />
<h3 id="case-study-the-sage--gap-interface">Case Study: the Sage / GAP interface</h3>
<ul>
<li>
<p><strong>Problem</strong></p>
<p>Use funtionalities from a system <code class="language-plaintext highlighter-rouge">A</code> in a system <code class="language-plaintext highlighter-rouge">B</code> written in a different language</p>
</li>
<li><strong>Data exchange</strong>
<ul>
<li>Objects returned as references (handles):
<ul>
<li>Few conversions</li>
<li>Reduced overhead</li>
<li>Manipulate from B objects of A with no native representation in B</li>
</ul>
</li>
</ul>
</li>
<li><strong>Binding</strong>
<ul>
<li>Originally: a pexpect interface</li>
<li>Now: ABI (direct calls at the C level)</li>
</ul>
</li>
</ul>
<hr />
<ul>
<li><strong>Adapting</strong>
<ul>
<li>Currently: monolithic adapters for some cases (groups, …)</li>
<li>In progress:
<ul>
<li>modular adapters, exploiting the category hierarchy and semantic alignments</li>
<li>objects returned as reference + semantic</li>
<li>alignment <code class="language-plaintext highlighter-rouge">B.cardinality() -> Size(A)</code> attached to the category of sets</li>
<li>alignment <code class="language-plaintext highlighter-rouge">B.conjugacy_classes() -> ConjugacyClasses(A)</code> attached to the category of groups</li>
</ul>
</li>
</ul>
</li>
</ul>
New OMDoc/MTT tutorial for mathematicians2018-07-12T00:00:00+00:00http://opendreamkit.org/2018/07/12/MMT-math-tutorial<p>The FAU group completed the tutorial <strong><a href="https://gl.mathhub.info/Tutorials/Mathematicians/blob/master/tutorial/mmt-math-tutorial.pdf">OMDoc/MMT for Mathematicians</a></strong> on working with MMT content targeted at Mathematicians. It covers creating new <a href="http://uniformal.github.io/doc/applications/archives.html">archives</a>, the <a href="http://uniformal.github.io/doc/language/">MMT language</a>, using LF, working with <a href="http://uniformal.github.io/doc/applications/jedit.html">jEdit</a>, <a href="http://uniformal.github.io/doc/applications/building.html">building</a> archives, browsing the MMT <a href="http://uniformal.github.io/doc/applications/server.html">server</a>.</p>
<p>This is a good way to get into formalizing mathematics in the Math-in-the-Middle ontology which is central to WP6 in OpenDreamKit.</p>
Work group on Free and Open Source software, ``Open Science Committee'', French Ministry for Research 2018-082018-07-01T00:00:00+00:00http://opendreamkit.org/2018/07/01/AMI-CoSO<h2 id="main-goals">Main goals</h2>
<p>The French Ministry for Research as recently
spawned a new committee to foster and coordinate Open Science take up
in France. One of its four work groups is on Free and Open Source
Software, led by Roberto Dicosmo, long time Open Source advocate and
initiator of the Software Heritage project.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Nicolas M. Thiéry, OpenDreamKit coordinator, was
invited to join the work group.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This work group will give long time Open
Source and Open Science advocates leverage to disseminate best
practices and supporting technologies far beyond their original
communities.</p>
June 2018 Steering committee meeting: Progress reports for the Period January-June 20182018-06-21T00:00:00+00:00http://opendreamkit.org/2018/06/21/XFELSteeringCommittee<p>Brief yearly progress reports for all sites.</p>
<p>Includes job openings, hires, and associated persons to the project,
main activities, achieved and ongoing deliverables …</p>
<p>The slides below are to be filled in:</p>
<ul>
<li><a href="/meetings/2018-06-20-EuropeanXFEL/ProgressReports/ParisSud">Université Paris Sud</a></li>
<li><a href="CNRS">CNRS</a></li>
<li><a href="JacU-FAU">Jacobs University Bremen with FAU Erlangen-Nürnberg</a></li>
<li><a href="UGA">Université Grenoble-Alpes</a></li>
<li><a href="Kaiserslautern">University of Kaiserslautern</a></li>
<li><a href="Oxford">Oxford</a></li>
<li><a href="Silesia">University of Silesia</a></li>
<li><a href="Sheffield">University of Sheffield</a></li>
<li><a href="Southampton-XFEL">University of Southampton with XFEL</a></li>
<li><a href="StAndrews">University of St Andrews</a></li>
<li><a href="UVersailles">Université Versailles Saint-Quentin</a></li>
<li><a href="Warwick">University of Warwick</a></li>
<li><a href="Zurich">Universität Zürich</a></li>
<li><a href="Logilab">Logilab</a></li>
<li><a href="Simula">Simula Research Laboratory</a></li>
<li><a href="UGent">Universiteit Gent</a></li>
</ul>
Project meeting at European XFEL GmbH2018-06-20T00:00:00+00:00http://opendreamkit.org/2018/06/20/Hamburg-DisseminationWorkshop-SteeringMeeting<p>OpenDreamKit’s project meeting will take place at European XFEL GmbH
in Schenefeld near Hamburg (Germany), from Wednesday 20 - Friday 22
June 2018. We expect about 15 participants at this event.</p>
<p><em>!! Note that during Wednesday all day as well as on Thursday morning, a professionnal videographer will be running 6 interviews (or more if relevant). Viviane Pons will directly contact the participants who are likely to be required to share their points of views and insights (on ODK as a project, Jupyter, dissemination activities…). The content will eventually be added on the next version of the website once the videographer is done editing.</em></p>
<h1 id="the-3-day-long-meeting-will-be-devoted-to-wp2-dissemination-and-wp1-steering-committee-and-preparation-of-deliverables-with-a-side-workshop-about-jupyter-activities">The 3-day long meeting will be devoted to WP2 (dissemination) and WP1 (steering committee and preparation of deliverables), with a side-workshop about Jupyter activities</h1>
<p>=> <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/270">Open GitHub issue to decide upon the content of the three days</a></p>
<h1 id="agenda">Agenda</h1>
<h2 id="wednesday-all-in-xtob101">Wednesday (all in XTOB1.01)</h2>
<p><em>10:30 Coffee for those already present</em></p>
<p>11:00 Session 1</p>
<ul>
<li>Viviane will begin the day by introducing the current dissemination/communication situation and our strategy for the last year of OpenDreamKit</li>
<li>Schools/trainings/workshops by Adrien Boussicault (Viviane)</li>
</ul>
<p><em>12:30 lunch</em></p>
<p>14:00 Session 2</p>
<ul>
<li>Presentations of participants eager to share their activities or ideas on dissem. and comm. content</li>
<li>Brainstorming if needed and/or writing of content (use cases, etc)</li>
</ul>
<p>15:00 Session 3: Introduction to XFEL and visit to experimental hall Coffee</p>
<p><em>16:00 Coffee break</em></p>
<p>16:30 Session 4</p>
<ul>
<li>Presentations of participants eager to share their activities or ideas on dissem. and comm. content</li>
<li>Brainstorming if needed and/or writing of content (use cases, etc)</li>
</ul>
<p>18:00 Close</p>
<h2 id="thursday-all-in-xtob101">Thursday (all in XTOB1.01)</h2>
<p>09:00 XTOB1.01 Session 1</p>
<ul>
<li>Presentations of participants eager to share their activities or ideas on dissem. and comm. content</li>
<li>Brainstorming if needed and/or writing of content (use cases, etc)</li>
</ul>
<p><em>10:30 Coffee break</em></p>
<p>11:00 Session 2</p>
<ul>
<li>XFEL presentation on Jupyter OOMMF and dissemination workshops</li>
<li><a href="/meetings/2018-06-20-EuropeanXFEL/EOSC-June2018.pdf">EOSC overview</a> (Benoît)</li>
<li>EOSC pilot (<a href="https://confluence.desy.de/pages/viewpage.action?pageId=13795644">Frank Schluenzen</a>)</li>
<li>NumFOCUS Europe (Nicolas)</li>
</ul>
<p><em>12:30 lunch</em></p>
<p>14:00 Session 3-a:
<a href="https://opendreamkit.org/meetings/2018-06-20-EuropeanXFEL/ProgressReports">Steering committee meeting</a> and preparation/writing of deliverables due for end August 2018</p>
<p><em>16:15 Coffee</em></p>
<p>17:00 Session 3-b: Steering committee meeting (IF NEEDED)</p>
<p>18:00 Close, and walk to restaurant</p>
<p><em>18:30 Dinner at <a href="http://www.hotel-kloevensteen.de/restaurant">Peter’s Bistro</a>)</em></p>
<h2 id="friday">Friday</h2>
<h3 id="jupyter-workshop-on-3d-graphics-in-xtob101">Jupyter workshop on 3d graphics in XTOB1.01</h3>
<p>09:00 Session 1 (XTOB1.01)</p>
<ul>
<li>
<p>Review of existing technology (Vidar) (something like <a href="https://pyvideo.org/euroscipy-2017/interactive-3d-visualization-in-jupyter-notebooks.html">EuroSciPy talk</a>)?</p>
</li>
<li>
<p>Objective of 3d visualisation for Jupyter OOMMF (Marijan)</p>
</li>
</ul>
<p><em>10:30 Coffee break (XTOB1.01)</em></p>
<p>11:00 Session 2 (XTOB1.01)</p>
<p><em>12:30 lunch (XTOB1.01)</em></p>
<p>14:00 Session 3 (XTOB1.01)</p>
<p>15:30 Coffee (XTOB1.01)</p>
<p><em>16:00 Closing (XTOB1.01)</em></p>
<h3 id="dissemination-workshop-in-xhq2097">Dissemination workshop in XHQ2.097</h3>
<p>(Can also use the smaller room XHQ1.007 instead of XHQ2.097 if preferred.)</p>
<p>09:00 Session 1 (XHQ2.097)</p>
<p><em>10:30 Coffee break (XTOB1.01)</em></p>
<p>11:00 Session 2 (XHQ2.097)</p>
<p><em>12:30 lunch (XTOB1.01)</em></p>
<p>14:00 Session 3 (XHQ2.097)</p>
<p>15:30 Coffee (XTOB1.01)</p>
<p><em>16:00 Closing (XTOB1.01)</em></p>
<h1 id="booked-rooms-at-european-xfel">Booked rooms at European XFEL</h1>
<ul>
<li>
<p>Main room</p>
<p>The main room for the entire meeting is XTOB1.01, which accommodates about 50 people. Coffee breaks (daily at 10:30 and 15:45) and lunch (daily 12:30-13:30) will be served there. This room is available to all participants at all times.</p>
</li>
<li>Breakout rooms
<ul>
<li>Wednesday 20 June
<ul>
<li>XHQ E1.150, 12:00-18:00, max. 8 people</li>
</ul>
</li>
<li>Thursday 21 June
<ul>
<li>XHQ E2.042 ~ 15 people</li>
<li>XHQ E1.150 ~ 8 people (except 12:30-14:00)</li>
</ul>
</li>
<li>Friday 22 June
<ul>
<li>XHQ E1.007 ~ 8 people</li>
</ul>
</li>
</ul>
</li>
<li>Interview rooms
<ul>
<li>Wednesday 20 June
<ul>
<li>XHQ E2.097 15:00-20:00</li>
</ul>
</li>
<li>Thursday 21 June
<ul>
<li>XHQ E2.097 12:00-18:45</li>
</ul>
</li>
<li>Friday 22 June
<ul>
<li>XHQ E2.097 all day</li>
</ul>
</li>
</ul>
</li>
</ul>
<h1 id="participants">Participants</h1>
<p><a href="https://framadate.org/XQuv3rtgcC8jtl7c">Registration page</a></p>
<ul>
<li>European XFEL GmbH: Hans Fangohr, Marijan Beg</li>
<li>Université Paris-Sud: Benoît Pilorget, Nicolas Thiéry, Viviane Pons, Erik Bray, (+ Romain Turminel?)</li>
<li>Logilab: Olivier Cayrol</li>
<li>UWarwick: John Cremona</li>
<li>University of Leeds: Mike Croucher</li>
<li>Simula: Min RK, Vidar T. Fauske</li>
<li>University of Silesia: Marcin Kostur</li>
<li>TU Kaiserslautern: Wolfram Decker</li>
<li>FAU: Michael Kohlhase</li>
</ul>
<h1 id="how-to-reach-the-european-xfel">How to reach the European XFEL</h1>
<p>For the details on how to reach the venue, please check <a href="https://www.xfel.eu/contacts/index_eng.html">here</a>.</p>
<h1 id="accommodation">Accommodation</h1>
<p>Participants need to book and pay themselves for the rooms.</p>
<p>You can book any hotel you want, but three probably the closest hotels to the venue are:</p>
<ul>
<li><a href="http://www.hotel-kloevensteen.de">Hotel Klövensteen</a> (1.0 km)</li>
<li><a href="http://www.wulff-pension.de">Pension Wulff</a> (0.5 km)</li>
<li><a href="http://www.hotel-blankenese.de/">Hotel Blankenese</a> (2.0 km)</li>
</ul>
<p>For further information about the possible hotels recommended by European XFEL, please check this <a href="#">pdf</a>. Please note that prices in the document can be outdated.</p>
<h1 id="public-transport">Public transport</h1>
<p>European XFEL is in Schenefeld, but it is very easy to reach it both from the Hamburg Airport, Main Train Station (Hauptbahnhof), and central Hamburg. To plan your local journey, you can visit <a href="http://www.hvv.de/en/">HVV</a>, enter your starting station and use “Schenefeld, Aneken” as your final stop.</p>
<p>Tickets can be bought at all S-Bahn and U-Bahn stations (cards and cash accepted) as well as from the bus driver (cash only, change given). In addition, you can also get a ticket using <a href="http://www.hvv.de/fahrplaene/hvv-app/">HVV-App</a>, but you will need to create an account.</p>
<h1 id="other">Other</h1>
<p>If you have any questions or requests regarding the meeting (accommodation, transport, food), please email either Marijan Beg (marijan.beg (at) xfel.eu) or Hans Fangohr (hans.fangohr (at) xfel.eu).</p>
Leveraging literate computing technology for teaching: a template web site for jupyter-based courses2018-06-18T11:07:00+00:00http://opendreamkit.org/2018/06/18/bridging-the-gap-between-literate-programming-and-teaching-modules<h2 id="background">Background</h2>
<p>The <a href="http://jupyter.org/">Jupyter notebook</a> is a popular tool used by
thousands of individuals and organisations all over the world. The
notebook popularity is associated with the paradigm of <strong>literate
computing</strong>; this is a variant of the paradigm of <strong>literate
programming</strong> introduced by <strong>Donald E. Knuth</strong>; under the latter the
(computer) program is treated like a piece of literature addressed to
humans rather than to computers. As a consequence, such programs
focus on the logic and flow of human thought. Similarly, a literate
computing document integrates both the data analysis and textual
documentation, making the programs more robust, more portable, and
easier to maintain than programs that are solely written in high-level
languages.</p>
<p>The notebooks, or rather the Jupyter Ecosystem have found their place
within research, education and data science, among many other areas,
particularly promoting knowledge transfer. I first started using the
notebooks when I realized that they allowed me not to only write code
in a number of programming languages but also allowed me to build a
compelling narrative in which analysis, documentation, and data are
presented together transparently. Since then I have used the notebooks
for dissemination, education, prototyping, have developed ad-hoc
extensions/widgets and have helped others to use the notebooks better.</p>
<p>However, the emergence of novel technologies also brings a new set of
challenges to overcome during their implementation and usage. This
case study presents some of the work the members of ODK have done in
order to bring literate programming to the classroom.</p>
<h2 id="scenario">Scenario</h2>
<p>Most universities in the UK have an institutional learning platform
commonly used by lecturers to share materials with students. Such a
platform only allows for upload and rendering of static documents in
.pdf, .txt, and .docx formats (among others). As science becomes more
computationally driven, there is a greater need to increase the data
and computational literacy of our students. Several lecturers and
professors at the University of Sheffield wanted to migrate from using
paid-license software, such as MATLAB, to open source software such as
Python and at the same time, use the notebooks in their class. In
addition to the notebooks lecturers need a permanent site to upload
the materials, class notes, and other resources so that students can
revise at any point and download the materials.</p>
<h2 id="possible-solutions">Possible solutions</h2>
<ol>
<li>Use of the institutional learning platform and separate
distribution of the notebooks.</li>
<li>Use of cloud solutions such as CoCalc to manage their course
content.</li>
<li>Leverage the use of the Jupyter notebooks along with binder,
Microsoft Azure notebooks and GitHub pages.</li>
</ol>
<p>The main problem with solution 1. is the duplicate point of access for
both students and the people running the class increasing the
maintenance burden. We have used CoCalc in Sheffield for a number of
classes (e.g. Biomedical sciences, scalable machine learning) with
great success in the past. But in that case only those enrolled in the
course can access the materials so unless the materials are in a
service like GitHub or GitLab they cannot be publicly distributed.</p>
<p>We finally decided to go for solution #3: leverage the use of other
open source solutions and develop a tool that could equally be used by
highly literate developers and lecturers with minimal GitHub
knowledge.</p>
<h2 id="time-and-expertise-required">Time and expertise required</h2>
<p>Assuming that the researcher is familiar with the Jupyter notebook and
with GitHub, installing the Python package is as simple as doing <code class="language-plaintext highlighter-rouge">pip
install nbjekyll</code> and the template can be readily forked or imported
from GitHub
()<a href="https://github.com/trallard/Modules-template">https://github.com/trallard/Modules-template</a>). Once
this is done the publishing part should take only a few minutes. The
notebooks can then be added or modified as <code class="language-plaintext highlighter-rouge">nbjekyll</code> will take care
of the testing, converting and appropriate formatting of the books
inside the repository.</p>
<p>A detailed guide on how to get started and how to convert the
notebooks into Jekyll ready documents can be found on the Modules
template documentation website:
<a href="http://bitsandchips.me/Modules-template-docs/">http://bitsandchips.me/Modules-template-docs/</a></p>
<p>The template enables the use of more advanced features such as
embedding of reveal.js slides and linking to Microsoft Azure
notebooks. All of which are also detailed in
<a href="http://bitsandchips.me/Modules-template-docs/advanced">http://bitsandchips.me/Modules-template-docs/advanced</a></p>
<h2 id="the-implementation-technical-bits">The implementation: technical bits</h2>
<p>Thanks to <code class="language-plaintext highlighter-rouge">nbformat</code> it is possible to convert the notebooks into
other formats, such as .pdf, .tex, .py, .html, .md and even reveal.js
slides. On the other hand, it is possible to now perform regression
tests on the notebooks thanks to the ODK developed tool
<a href="https://github.com/computationalmodelling/nbval">nbval</a>, and finally,
GitHub supports and serves static websites generated using Jekyll.
Thus we worked on developing a readily usable Jekyll template
specifically designed for university course modules and with Jupyter
notebooks as the main source of content.</p>
<p><br /> <br />
<img src="http://opendreamkit.org/assets/Jekyll.PNG" style="margin:10px 0; border:1px solid #b5b5b5;" />
<br /><br /></p>
<p>It would then be possible to convert the notebooks to <code class="language-plaintext highlighter-rouge">.md</code> or <code class="language-plaintext highlighter-rouge">.html</code>
and then add the <code class="language-plaintext highlighter-rouge">.yaml</code> header needed by Jekyll to render it within
the posts. In order to remove the manual burden we developed a python
package <a href="https://github.com/trallard/nbjekyll">nbjekyll</a> that uses
nbformat to convert the notebook to a Jekyll post. By calling
nbjekyll within the filesystem containing a Jekyll blog it will find
the notebooks and convert them into Jekyll ready .md files with the
appropriate front-matter needed, export the images (if any) within the
notebook and correctly update the paths in the .md files so that they
point to the images in the site.</p>
<p><img src="http://opendreamkit.org/assets/Jekyll-2.PNG" style="margin: 10px 0; border: 1px solid #b5b5b5;" />
<br /></p>
<p>In addition, when doing the conversion, <code class="language-plaintext highlighter-rouge">nbjekyll</code> uses nbval under
the hood to perform a sanity check on the notebook. By doing so a
badge is added to indicate if the tests on the notebook were
successful of if there were any problems encountered during the
testing phase.</p>
<p>Finally, as the whole site is intended to be used in conjunction with
GitHub (or GitLab) the package leverages the use of version control to
identify the last commit, the author and the date of the last commit
and this information is added to the top of the yaml header so that it
is rendered as shown in the image.</p>
<p>Finally, if the notebooks are hosted in Microsoft Azure, the author
can indicate this in the <code class="language-plaintext highlighter-rouge">_config.yml</code> file and a cloud button will be
added to the site so that the repository is imported as an Azure
library and the notebooks can be accessed directly from the course
site. This gives the advantage that the students can then clone the
library for the course to their own Azure account and can start
working on the notebooks without the need to install any packages
locally.</p>
<h2 id="to-do">To Do</h2>
<ul>
<li>Support for the user to use <a href="http://mybinder.org/">binder</a> shall the
materials not be deposited in a Microsoft Azure library</li>
<li>Jupyter Notebook extension to directly convert the notebook from the
Jupyter server instead of the command line</li>
</ul>
<h2 id="opendreamkit-contribution">OpenDreamKit contribution</h2>
<ul>
<li>Development of the Modules-template for Jekyll</li>
<li>Support to ongoing courses at the University of Sheffield and case
study on the technology</li>
<li>Creation of nbjekyll (available from <a href="https://pypi.org/project/nbjekyll/">PyPI</a>)</li>
</ul>
Inauguration of UK Research Software Engineering Association United Kingdom, 15th of June 20182018-06-15T00:00:00+00:00http://opendreamkit.org/2018/06/15/leeds-rse<h2 id="main-goals">Main goals</h2>
<p>The Research Software Engineering movement began in the UK in 2012 (See https://doi.org/10.5281/zenodo.495360} for background and history) and aimed to support the people in academia who’s careers focus on the computing that underpins research rather than the research itself. OpenDreamKit was an example of an EU-funded project that employed a large number of Research Software Engineers which served as a useful demonstrator for this emerging profession. In 2018, The UK Association of Research Software Engineers \url{https://rse.ac.uk/about/the-association/ became a formal society as recognised in UK Law. This was supported, in part, by the work of OpenDreamKit-funded Tania Allard who was elected to the association committee in late 2017.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit was involved in this work via OpenDreamKit-funded team members Tania Allard and Mike Croucher of University of Leeds.</p>
<h2 id="event-summary">Event summary</h2>
<p>Following months of work, the committee signed the legal documents on 15th June 2018. The event was summarized on twitter at https://twitter.com/sjh5000/status/1007639883165454336</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>OpenDreamKit is part of the history of the development of this new profession. Initially as proof that Research Software Engineers could win funding in their own right to perform vital work for the research community and later as an integral part of the formation of the legal entity that supports the RSE community. Research Software Engineers are changing the way that research is being performed for the better. Initially based only in the UK, the movement has since gone international https://rse.ac.uk/community/international-rse-groups/.</p>
LinBox Days June 2018 Grignan (FR),2018-06-20 to 2018-06-222018-06-08T00:00:00+00:00http://opendreamkit.org/2018/06/08/linbox<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: C. Pernet, J.-G. Dumas, H. Zhu.</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and travel expenses), for about 2.6k Euro.</p>
<h2 id="event-summary">Event summary</h2>
<p>The Grignan LinBox developer meeting took place in Grignan (France), from June
20th to 22nd.</p>
<p>There were 9 participants from 3 different institutions (UGA, Université
Montpellier 2, US Naval Academy).
No registration fees were applied.</p>
<p>After a round table brainstorming on development projects for the meeting,
developers gathered in groups each addressing a given project.
The main achievements of the meeting include</p>
<ul>
<li>Progress on the distributed parallel ration solver (\delivref{hpc}{LinBox-distributed});</li>
<li>Design and refactorization of the code for randomized algorithms in LinBox;</li>
<li>Completion of the rewrite of the prime fields in Givaro;</li>
<li>Improvement of code robustness and release preparation (\delivref{hpc}{LinBox-algo}).</li>
</ul>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop was very productive and particularly beneficial to WP5 (\delivref{hpc}{LinBox-algo}).</p>
Ecole Jeunes Chercheurs en Theorie des Nombres Besancon (FR), 2018-06-05 to 2018-06-082018-06-08T00:00:00+00:00http://opendreamkit.org/2018/06/08/EJC-TDN<h2 id="main-goals">Main goals</h2>
<p>This is a biennial French national meeting for young researchers in number
theory (PhD students and post-docs), as a CNRS Thematic School.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Bill Allombert was invited to give a 3-hours introduction to PARI/GP on
the conference final day.</p>
<p>OpenDreamKit participants: B. Allombert from Bordeaux.</p>
<p>OpenDreamKit did not fund the event.</p>
<h2 id="event-summary">Event summary</h2>
<p>This conference took in Besan\c{c}on (France) from june 5th to
8th. There were about 50 participants
from all over France (no registration fees). The event featured three
mini-courses, a presentation of PARI/GP with special focus on Algebraic
Number Theory; 12 students gave short presentation of their research.</p>
<p>Slides for the PARI/GP presentation are available at
http://pari.math.u-bordeaux.fr/Events/EJC2018/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a successful teaching and dissemination event.</p>
Summer School Zeta functions, polyzeta functions, arithmetical series Chamb\'ery (FR), 2018-06-18 to 2018-06-292018-06-06T00:00:00+00:00http://opendreamkit.org/2018/06/06/ZETAS<h2 id="main-goals">Main goals</h2>
<p>This was a summer school (CNRS Thematic School) on the Theory of Motives and
the Theory of Numbers, targeted at young researchers (PhD students and
post-docs)</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Bill Allombert was invited to give a 3-hours introduction to PARI/GP during
the conference.</p>
<p>OpenDreamKit participants: B. Allombert from Bordeaux.</p>
<p>OpenDreamKit did not fund the event.</p>
<h2 id="event-summary">Event summary</h2>
<p>This conference took in Chamb'ery (France) from june 18th to
29th. There were about 30 participants (registration fees: 40 Euro for PhD
students, 90 Euro otherwise). The event
featured six mini-courses and a presentation of PARI/GP with special focus
on zeta functions.</p>
<p>Slides for the PARI/GP presentation are available at
http://pari.math.u-bordeaux.fr/Events/ZETAS2018/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a successful teaching and dissemination event.</p>
Project meeting in XFEL XFEL Hamburg, June 20 --22, 20182018-06-06T00:00:00+00:00http://opendreamkit.org/2018/06/06/PM<h2 id="main-goals">Main goals</h2>
<p>At the occasion of the project meeting, we also held a workshop on dissemination and a Jupyter workshop on 3d graphics.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The event was hosted by XFEL and each participant was funded by OpenDreamKit through their institutions.</p>
<h2 id="event-summary">Event summary</h2>
<p>The first part of the event was focused on dissemination and communication in OpenDreamKit. It was an occasion to expose what had been done in the previous years and start a reflexion on how to better promote it. We discussed the new website design and organization and we wrote some new content. We also had a professional interviewer on site who conducted interviews of the OpenDreamKit team to give an overview of the project goals, accomplishments, and philosophy. The second part of the event was dedicated to a Jupyter workshop on 3d graphics.</p>
<h2 id="demographic">Demographic</h2>
<p>All participants were OpenDreamKit members.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a very useful consultation. The new website was released in the following weeks. The interviews are still being edited but should be live soon.</p>
6th Encuentro Colombiano de Combinatoria Barranquilla (Colombia), June 5 --- 16, 20182018-06-06T00:00:00+00:00http://opendreamkit.org/2018/06/06/ECCO<h2 id="main-goals">Main goals</h2>
<p>ECCO (for <strong>Encuentro Colombiano de Combinatoria</strong>) is a combinatorics research school organized every other year in Colombia. It gathered students from many countries (from both south-America and elsewhere) on advanced mathematics classes in an inclusive and welcoming environment. For the second time, it offered a Sage class as well which goal is introduce Sage to the students and help them develop their coding skills in relations to mathematics.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The Sage class was given by Viviane Pons from OpenDreamKit. Her travel costs were covered by OpenDreamKit.</p>
<h2 id="event-summary">Event summary</h2>
<p>Two afternoon sessions were organized (one for each week of the workshop). We had approximately 80 students on the first session and 40 on the second. Both sessions were organized in the computer room of the university so that all students would work whether they had material or not. Focus was given to introduction to Sage in the context of combinatorics by offering many different tutorials and exercise worksheets for the students to choose from. We had created some specific worksheets in both English and Spanish directly related to the others courses of the conference so that the students could reproduce some of the results they had seen during classes and exercise sessions.</p>
<h2 id="demographic">Demographic</h2>
<p>We don’t have access to the demographic information of the conference. Nevertheless we can say that most participants were students (from undergrad to PhD). Beside, this conference is in general very careful in creating a diverse and welcoming space.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was the second time that we were at ECCO and we could see again that this was a great success. We had excellent feedbacks from both the students and the organizers. In 2016, the Sage sessions were added late in the planning as extra sessions. In 2018, this is now an official Sage course listed along the other mathematical classes of the conference. By being present twice in a row, we started a tradition of offering Sage classes during this conference and this will be continued even after the end of OpenDreamKit. For many students attending ECCO, the Sage class is their first encounter with Sage or even a mathematic software.</p>
<p><img src="/public/images/events/ECCO.jpg" alt="" /></p>
<p>Students working at ECCO</p>
SPLS Heriot-Watt University, Edinburg, Scottland, June 5, 20182018-06-05T00:00:00+00:00http://opendreamkit.org/2018/06/05/SPLS<h2 id="main-goals">Main goals</h2>
<p>The <strong>Scottish Programming Languages Seminar</strong> is a
forum for discussion of all aspects of programming languages. They meet for a
day or afternoon once every few months, at some congenial location in
Scotland.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Florent Hivert gave an invited talk. The cost for
OpenDreamKit was therefore null.</p>
<h2 id="event-summary">Event summary</h2>
<p>Florent Hivert was invited to give a keynote talk
presenting his work on WP5 T5.6. The talk was entitled \emph{ Multi-level
parallelism for high performance combinatorics}. Here is the abstract:</p>
<p>In this talk, I will report on several experiments around large scale
enumerations in enumerative and algebraic combinatorics. I’ll describe a
methodology used to achieve large speedups in several enumeration
problems. Indeed, in many combinatorial structures (permutations, partitions,
monomials, young tableaux), the data can be encoded as a small sequence of
small integers that can often efficiently be handled by a creative use of
processor vector instructions. Through the challenging example of numerical
monoids, I will then report on how Cilkplus allows for a extremely fast
parallelization of the enumeration. Indeed, we have been able to enumerate
sets with more that $10^15$ elements on a single multicore machine.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The talk was a chance to disseminate OpenDreamKit work in
a wider audience and to present the result on deliverable D5.1 and the ongoing
progress on the overall work package. The fact that it was an invited keynote
talk witnesses that the community is particularly interested and attentive on
the OpenDreamKit progress on this matters.</p>
GAP Days -- Jupyter in GAP and other CAS St Andrews, United Kingdom, 4th to 8th June 20182018-06-04T00:00:00+00:00http://opendreamkit.org/2018/06/04/jupyter-gap<p>## Main goals</p>
<p>The aim of the workshop was to bring together
developers and early adopters of Jupyter, Binder, Thebelab and other
web technology relevant for Virtual Environments for research and
teaching, with some focus on the GAP computational system.</p>
<p>Topics that were worked on included:</p>
<ul>
<li>The GAP JupyterKernel</li>
<li>Javascript-Visualizations using Jupyter</li>
<li>Using Thebe for interactive manuals</li>
<li>Developing teaching materials with Jupyter, and publishing them using MyBinder</li>
<li>Writing academic publications using MyBinder and Docker images, to make
all computational results and all examples fully and easily reproducible.</li>
<li>Discussing demands on software and hardware infrastructure in day to day
use.</li>
</ul>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Organised by Markus Pfeiffer at UStan. OpenDreamKit paid for
the workshop dinner, the accommodation for Nusa Zidaric, Pedro
Garcia-Sanchez, and Sergio Siccha, travel and accommodation for Nicolas Thiéry.</p>
<h2 id="event-summary">Event summary</h2>
<p>We met in a small group of developers and interested
clients of our Jupyter, ThebeLab, Binder and other web technology relevant for
virtual research environments, focused on GAP and the wider context of
OpenDreamKit. We explained how the GAP Jupyter Kernel works, developed some
prototypes with and improvements of the GAP Jupyter kernel.</p>
<p>We demonstrated the work of Manuel Martins (Markus Pfeiffer’s PhD student),
Sebastian Gutsche, and Pedro A. García-Sánchez who are using GAP’s Jupyter
kernel in research and teaching and to develop new ways of using them.</p>
<p>To structure programming work and discussions, we followed the well-established
practice of a stand-up meeting in the morning to coordinate work, and a meeting
in the afternoon to record achievements.</p>
<p>Discussions ranged from deepining the understanding of how Jupyter works, and
what needs the research and teaching community has to more technical topics
which reach into other workpackges such as improved integration between SageMath
and GAP.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>Teachers and researchers intending to use the GAP
Jupyter kernel and adjacent technologies in teaching and research, are now much
more comfortable using these technologies. The demonstration of the
possibilities left a good impression of the potential of this technology.</p>
Working with Web data in digital social sciences2018-06-01T12:58:00+00:00http://opendreamkit.org/2018/06/01/working-with-webdata-in-digital-social-sciences<p>OpenDreamkit is highly involved in dissemination activities and the introduction
of researchers and educators to the Jupyter ecosystem. This is not only limited
to researchers within the physical sciences and mathematics. Over the last couple
of months at Sheffield and now at Leeds, we have been working with some
digital scientists from the <a href="https://www.sheffield.ac.uk/smi">Research Methods Institute</a>
to develop some teaching materials around the collection and use of Web Data.</p>
<p>Both, Binder and the Jupyter notebook have played an integral part in the development
and delivery of this series of workshops. The use of these technologies not only
makes it easier to share research and educational materials but, if used appropriately,
they can serve as valuable pedagogical tools.</p>
<p>At the moment all of the materials are focused on Python as the programming language
for the acquisition, processing and visualisation of the data. The developed materials cover an introduction to regular expressions, dictionaries, json and XML data, web scraping and HTTP requests and APIs. In the upcoming months, we plan to add more content on streams and programmatic web scraping. Also, we have plans
to produce an R version of the materials using again, Jupyter notebooks and
Binder’s improved support for the R language.</p>
<p>The current version of the materials was developed as part of a series of workshops
for digitial scientists over the last two weeks of May 2018.</p>
<p>If you want to have a look at the materials or wish to suggest additional content
please visit the <a href="https://github.com/trallard/WebData_Python">GitHub repository</a>.</p>
PyCon 20182018-05-30T00:00:00+00:00http://opendreamkit.org/2018/05/30/pycon-2018<p>I recently attended PyCon 2018 in Cleveland, Ohio. Apart from attending many
exciting talks, BoF (bird of a feather session) and additional events I gave
a tutorial on Reproducible analysis workflows. I have to admit I was a bit
surprised when I realised that the 58 places available for the workshop had been
booked by attendees to the conference.</p>
<p>The workshop started with a presentation illustrating why reproducibility is
important and focusing on the definition of reproducibility, replicability, and
robustness. This introductory section was also an excellent opportunity to promote
discussions among the
attendees and to better understand what the main barriers to reproducibility are
they have encountered.</p>
<p>The next part of the workshop was focused around providing the attendees with practical
guidelines and tips to store better and manipulate their data, create filesystems,
develop analysis packages and test their code. One of the things I <em>really</em> wanted
to highlight was the importance of licensing your scientific code. When you have little
experience writing and sharing code, this might be a tricky subject to dive into.
Over the years I have encountered projects deposited in places like GitHub without a license,
which leaves the software in a weird usage limbo. On the one hand, the code is publicly
stored in a repository. On the other hand, it does not explicitly say whether others
can use, modify or distribute the software in any given way.
As a consequence, many people would opt not to use the software as they are not
confident they are allowed to.</p>
<p>Once again, with the popularity of Jupyter notebooks, it is essential to
facilitate the testing and version control as we’d do with any other pieces of
standalone scripts. Thus for these topics, I always go back to using nbdime and
nbval. The main reason? These packages bring all of the version control and
regression testing capabilities that Jupyter notebooks had been missing for a while.
What is more, now with <a href="https://github.com/jupyter/nbdime">nbdime 1.0</a> out you not only get better integration with
git but also get Jupyter notebook <strong>and</strong> JupyterLab extensions 🎊🎉.
So if you are using JupyterLab, you can now have the HTML rendered diffs served directly
in the app without even leaving it. When it comes to Jupyter Notebooks regression testing, we are often interested in verifying whether the results are consistent across different
runs of the analysis or if the results and analyses have not been affected by newer
releases of specific packages we use. In those cases <a href="https://github.com/computationalmodelling/nbval">nbval</a> does the trick for you.</p>
<p>Finally, the workshop covered how you can share your code and make it publicly
available <em>and</em> what you can do to get credit for it (generate a DOI and a citation
file).</p>
<p>Overall, the workshop was a success, and we will be doing a re-run in Leeds on the 14th of June 2018. For anyone willing to have a look at the materials they can be found in this <a href="https://github.com/trallard/ReproduciblePython">GitHub repository</a>.</p>
Atelier et \'Ecole MathExp 2018 Saint-Flour (FR), 2018-05-21 to 2018-06-012018-05-21T00:00:00+00:00http://opendreamkit.org/2018/05/21/MathExp<h2 id="main-goals">Main goals</h2>
<p>The MathExp school and atelier organized in Saint Flour was a unique
opportunity for young mathematicians to learn about computer science
and the tools developed in the framework of OpenDreamKit. The event
was divided in two weeks. The first one focused on 4 courses and
introductory tutorials with SageMath and Jupyter. During the second
week the participants were asked to develop programs related to their
own research projects.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit organizer: V. Delecroix</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and some of the travel expenses) for about 40K Euro. There were
also registration fees and the event was cofounded with the CNRS.</p>
<h2 id="event-summary">Event summary</h2>
<p>The school and atelier MathExp took place in Saint-Flour (France)
from May 21st to June 1st.</p>
<p>There were 22 registered participants.</p>
<p>A typical day of the school consisted of 2 courses and tutorials
on computers. During the atelier, participants worked on their
own research projects asking for help when needed.</p>
<p>The School featured 4 courses on computer science topics directly
related to mathematical computations: probability (Ana Bušić (Paris, FR)),
linear programming (Xavier Goaoc (Marne-la-Vallée, FR)), formal computation
(Bruno Salvy (Lyon, FR)) and backtracking techniques (Michaël Rao (Lyon, FR)).</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>We were delighted to achieve a gender equidistribution among the participants
(10 female and 12 males) which is not often in Mathematics and Computer
Science.</p>
<p>The school and workshop were very productive and beneficial to disseminate
the work achieved in the various work packages. In particular
all the work around SageMath and Jupyter.</p>
Full-time developer position opening for 20182018-05-18T00:00:00+00:00http://opendreamkit.org/2018/05/18/position-grenoble<p>This is an announcement for a full-time developer (<a href="http://fr.wikipedia.org/wiki/Ing%C3%A9nieur_de_recherche">Ingénieur de
Recherche</a>)
position opening at Université Grenoble-Alpes, working on open source
software engineering for computational mathematics.</p>
<p>contact: <a href="mailto:clement.pernet@univ-grenoble-alpes.fr,jean-guillaume.dumas@univ-grenoble-alpes.fr">clement.pernet@univ-grenoble-alpes.fr and jean-guillaume.dumas@univ-grenoble-alpes.fr</a></p>
<h1 id="duration">Duration</h1>
<p>For 1 year starting no later than September 2018.</p>
<h1 id="salary">Salary</h1>
<p>Approx. 2900 € of monthly <em>salaire brut</em> (before tax).</p>
<h1 id="location">Location</h1>
<p>The developer will work at the <a href="http://www-ljk.imag.fr">LJK</a> lab. on the campus of Saint-Martin d’Heres, 15 minutes
from Grenoble city center by public transportation.</p>
<h1 id="context">Context</h1>
<p>Exact linear algebra, computing with arbitrary precision integer or rationals, and over finite field, is a core component of computer algebra software and also plays a central role in numerous computation intensive applications, ranging from algebraic cryptanalysis to experimental mathematics.
The <a href="http://github.com/linbox-team/linbox">LinBox</a> library and its components, the <a href="http://github.com/linbox-team/givaro">Givaro</a> and the <a href="http://github.com/linbox-team/fflas-ffpack">FFLAS-FFPACK</a> libraries, form a C++ library ecosystem offering high performance implementation of the most common exact linear algebra routines.
These open-source libraries are integrated in the <a href="http://sagemath.org">SageMath</a> open-source mathematics software.</p>
<p>Recently the FFLAS-FFPACK library has been parallized for multi-core architectures which opens way to two development directions:</p>
<ul>
<li>propose a framework and new implementations for parallel architectures;</li>
<li>make SageMath’s linear algebra’s routines parallel by properly exposing the new features in the interface</li>
</ul>
<h1 id="mission">Mission</h1>
<p>The mission of the engineer will be to</p>
<ul>
<li>design development and maintainance of the 3 opensource libraries LinBox fflas-ffpack and Givaro;</li>
<li>design and implementation of new algorithms for parallel linear algebra;</li>
<li>maintain and enhance SageMath’s interface to these libraries exposing new parallel features;</li>
<li>Participate actively in regular European development and training meetings with the other OpenDreamKit participants.</li>
</ul>
<h1 id="skills-and-background-seeked">Skills and background seeked</h1>
<ul>
<li>
<p>Development in Linux-like environments;</p>
</li>
<li>
<p>Experience with git</p>
</li>
<li>
<p>Experience with code optimisation, parallelism (OpenMP, MPI), debugging (valgrind, gdb);</p>
</li>
<li>
<p>Fluency in several of C, C++, Python, Cython;</p>
</li>
<li>
<p>Fluency in English;</p>
</li>
</ul>
<p>The following are not prerequisites but would be very much appreciated:</p>
<ul>
<li>
<p>Experience in open-source development (collaborative development tools, interaction with the community, …);</p>
</li>
<li>
<p>Experience with computational mathematics software, in particular
<a href="http://sagemath.org">SageMath</a> or <a href="http://github.com/linbox-team/">LinBox</a>;</p>
</li>
<li>
<p>Mathematics background.</p>
</li>
</ul>
<h1 id="applications">Applications</h1>
<p>Applicants should send an email to <a href="mailto:clement.pernet@univ-grenoble-alpes.fr,jean-guillaume.dumas@univ-grenoble-alpes.fr">clement.pernet@univ-grenoble-alpes.fr and jean-guillaume.dumas@univ-grenoble-alpes.fr</a></p>
<ul>
<li>
<p>A complete CV</p>
</li>
<li>
<p>A motivation letter</p>
</li>
</ul>
<p>Both documents must be in an open format (pdf, ps, plain text, …) either in
french or english.</p>
<h1 id="context-1">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September 2015. This project brings together the open-source computational mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Led by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</p>
<p>The developer will work within a team of developpers of SageMath and LinBox, and researchers in the field of high performance computer algebra
at the <a href="http://www-ljk.imag.fr/">Laboratoire Jean Kuntzmann</a>.</p>
Composing mathematical computational software and databases: MitM to the rescue2018-05-16T00:00:00+00:00http://opendreamkit.org/2018/05/16/lmfdb-usecase<p>The comic above shows off the idea of an interoperability layer for mathematical software
systems base on the Math-in-the-Middle (MitM) paradigm using a combination of Singular,
Sage and GAP. In this blog post, we detail another use case with the computer algebra
systems SageMath and Pari, but now with the mathematical database LMFDB in the mix.</p>
<p>This use case is important, since exploring mathematical data sets often requires both
data base operations and computation: mathematical data bases like
<a href="http://lmfdb.org">LMFDB</a>, the <a href="http://oeis.org">OEIS</a>, or <a href="http://findstat.org">FindStat</a>
are conceptually condensed tabulations of prior computations that can be searched over for
completeness.</p>
<p>The use case described here is about Modular Forms, a class of objects of immense
importance in number theory and cryptography.</p>
<p>In a nutshell: The LMFDB contains many individual instances of several kinds of modular
forms (which are the “MF” in the <a href="http://lmfdb.org">LMFDB</a>). Each of these objects has
associated to it an algebraic number field called its Hecke field. A mathematically
interesting project is to study these Hecke fields for the modular forms in each
collection, collecting data about them such as their degree or class number (see the next
section for definitions) and investigating how these are distributed. Being able to access
this LMFDB data directly from within a package with the ability to carry out its own
computations with number fields would be a useful application.</p>
<p>To conduct this research, we need a tight two-ways integration between the
<a href="http://lmfdb.org">LMFDB</a> and computational systems like <a href="ttp://sagemath.org">SageMath</a> or
<a href="https://pari.math.u-bordeaux.fr/">Pari</a>. But as the <a href="http://lmfdb.org">LMFDB</a>, SageMath
and PARI are separate systems, designed with differing application areas and constraints
in mind, this integration is non-trivial on several levels. This is where the
<a href="https://github.com/OpenDreamKit/OpenDreamKit/blob/master/WP6/MACIS17-interop/crc.pdf">Math-in-the-Middle Paradigm</a>
comes to the rescue.</p>
<p><a href="/public/images/use-cases-lmfdb-mitm.svg"> <img src="/public/images/use-cases-lmfdb-mitm.svg" alt="The MitM Paradigm Connection Graph" class="display" /> </a></p>
<p>Systems (A, …, H in the image) are connected via formal representations their APIs
(data types and functions), which are in turn related to a central (system-independent)
mathematical ontology: the
<a href="https://mathhub.info/#/content/ODK">Math-in-the-Middle Ontology</a>. In the OpenDreamKit
implementation of the MitM Paradigm, the system APIs and the MitM Ontology are implemented
as <a href="http://uniformal.github.io">OMDoc/MMT theory graphs</a>.</p>
<p>At the system level, SageMath (or PARI) and LMFDB are connected via a MitM Mediator, which
receives requests from SageMath (in terms of the SageMath API), translates them to the
LMFDB API via their relations to the MitM Ontology, and back.</p>
<p><a href="/public/images/use-cases-lmfdb-mediator.svg"> <img src="/public/images/use-cases-lmfdb-mediator.svg" alt="The MitM Paradigm Mediator Architecture" class="display" /> </a></p>
<p>In essence, the knowledge about the connections between the system API theories and the
MitM Ontology acts as semantic glue between the systems.</p>
<p>John Cremona and David Lowry-Duda wrote up a <a href="https://github.com/OpenDreamKit/OpenDreamKit/blob/master/WP6/usecase-notes/UseCaseHecke.pdf">detailed mathematical use case</a> for integrating LMFDB with SageMath and/or Pari/GP via the <a href="https://github.com/OpenDreamKit/OpenDreamKit/blob/master/WP6/MACIS17-vt/crc.pdf">virtual theories in MMT</a> and the Math-in-the-Middle Paradigm.</p>
<p>Based on this document, work on fully realizing this use case is ongoing; the next steps
are:</p>
<ol>
<li>the <a href="http://kwarc.info">KWARC group</a> at <a href="http://fau.de">FAU</a> work on the virtual
theories for the tables involved in collaboration with the LMFDB group at Warwick.</li>
<li>the UPSud group extends the SAGE interface theories to cover the necessary interface functions.</li>
<li>we (together) make sure that the Math-in-the-Middle ontology has the necessary theories
(at least as stubs) and alignments.</li>
</ol>
<!-- LocalWords: dksbases sagemath pari-gp realizing
-->
Web data in Python Sheffield, 18th, 21st, 22nd and 23rd of June 20182018-05-05T00:00:00+00:00http://opendreamkit.org/2018/05/05/sheff-webdata<h2 id="main-goal">Main goal</h2>
<p>Teach Social Scientists how to use Python and Jupyter notebooks to interact with online data sets.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit was involved in this work via OpenDreamKit-funded team member Tania Allard.</p>
<h2 id="event-summary">Event summary</h2>
<p>Web data in Python for non computer scientists is a set of Jupyter notebooks and materials on how to use Python to collect, clean and analyse web data.</p>
<p>It was developed in conjunction with Sheffield Research Methods Institute (An organisation created to promote innovation in research methods that can be applied to the social sciences to ultimately help solve the big challenges facing today’s society.)</p>
<p>In addition to developing and open sourcing the materials a workshop on the topic and using the materials was taught on May 18th, 21st, 22nd and 23rd along with other Software Carpentry workshops.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>Materials are available online at https://github.com/trallard/WebData_Python</p>
Jupyter receives the ACM Software System Award!2018-05-02T00:00:00+00:00http://opendreamkit.org/2018/05/02/jupyter-acm<p><a href="https://blog.jupyter.org/jupyter-receives-the-acm-software-system-award-d433b0dfe3a2">Project Jupyter has been awarded</a>
the 2017
<a href="https://awards.acm.org/software-system">ACM Software System Award</a>,
joining an illustrious list of projects that contains major highlights
of computing history, including
<a href="https://en.wikipedia.org/wiki/Unix">Unix</a>,
<a href="https://en.wikipedia.org/wiki/TeX">TeX</a>,
<a href="https://en.wikipedia.org/wiki/S_%28programming_language%29">S</a> (R’s
predecessor), the <a href="https://en.wikipedia.org/wiki/World_Wide_Web">Web</a>,
<a href="https://en.wikipedia.org/wiki/NCSA_Mosaic">Mosaic</a>,
<a href="https://en.wikipedia.org/wiki/Java_%28programming_language%29">Java</a>,
<a href="https://en.wikipedia.org/wiki/Ingres_(database)">INGRES</a> (modern
databases) and
<a href="https://en.wikipedia.org/wiki/ACM_Software_System_Award">more</a>.
We are delighted of this strong recognition of a major piece
of the ecosystem OpenDreamKit is based upon and contributing to.</p>
<p>Congratulations to the Jupyter team, including our fellows Min
Ragan-Kelley and Thomas Kluyver!</p>
Reproducible Python Cleveland, USA, May 20182018-05-01T00:00:00+00:00http://opendreamkit.org/2018/05/01/sheff-pycon<h2 id="main-goals">Main goals</h2>
<p>The PyCon 2018 conference, which took place in Cleveland, is the largest annual gathering for the community using and developing the open-source Python programming language. It is produced and underwritten by the Python Software Foundation, the 501(c)(3) nonprofit organization dedicated to advancing and promoting Python. Through PyCon, the PSF advances its mission of growing the international community of Python programmers.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit Sheffield member, Tania Allard, developed and delivered a workshop using OpenDreamKit supported technology at PyCon2018.</p>
<h2 id="event-summary">Event summary</h2>
<p>Workshop and open materials on Reproducible analysis in Python: these materials cover the essentials on how to develop workflows with a reproducibility approach.</p>
<p>The workshop was first delivered in PyCon 2018 to over 60 attendees from all over the world.</p>
<p>The materials were further extended to add more content on Jupyter notebook validation using OpenDreamKit-developed nbval and also on property based testing</p>
<p>All the materials are licensed under CC-BY and can be found at https://github.com/trallard/ReproduciblePython and are also shared using binder</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>As a follow up for this workshop Tania Allard was been invited to give a talk about reproducibility in data pipelines at the RAPIDS conference in London (https://www.dotmesh.com/blog/rapids-2018/)</p>
Workshop on interfacing (math) software with low level libraries Cernay (France) 2018-04-30 to 2018-05-042018-04-30T00:00:00+00:00http://opendreamkit.org/2018/04/30/Cernay<p>## Main goals</p>
<p>This developer meeting was focused on
initiating long term work on OpenDreamKit tasks related to packaging,
portability and documentation tools for SageMath.</p>
<p>## OpenDreamKit implication</p>
<p>This event was organized and funded by
OpenDreamKit (Paris Sud).</p>
<p>## Event summary</p>
<p>An intensive week with some short informal
presentations, and many brainstorms and coding sprints.</p>
<p>## Demographic</p>
<p>11 OpenDreamKit participants from five sites together
with 14 other experts of a variety of systems such as Oscar,
Polymake, CERN’s Root, Cython, Jupyter and C++, Numba, Dask,
libsemigroups.</p>
<p>## Results and impact</p>
<p>(Math) Computational systems face a
tension between using high level languages (e.g. Python) for
expressivity, ease of use and prototyping, and low-level languages
(e.g. C/C++) for power and speed, and also for modularity (using
existing libraries, or writing reusable ones). To resolve this
tension, many approaches have been explored in the recent years, and
the frontier between the two worlds is becoming increasingly blurry.</p>
<p>This workshop brought together developers from a large variety of
horizons to share expertise, seek collaboration venues, and get into
concrete action. This was quite bold of an aim.</p>
<p>Nevertheless, thanks to an informal and flexible format the week
ended up being very productive, as testified by the progress reports
and exit survey: to the question “Would you recommend to your
colleagues to attend a workshop with this format”, 9 answers were
<code class="language-plaintext highlighter-rouge">yes definitely'' ad the other two</code>yes’’; a participant well
outside of OpenDreamKit’s usual scope wrote as testimony «The looseness made
it more productive than any workshop I’ve ever attended».</p>
<p>Notes were taken collectively with the eventual aim to author a
joint survey of the available technologies and best practices; the
current draft is available from
https://www.overleaf.com/read/wrmkhfvwmxnz.</p>
<p><img src="/public/images/events/2018-04-30-Cernay.jpg" alt="" /></p>
<p>Jim Pivarski, CERN Root developer, presenting on columnar
storage for HPC in the context of data analysis for high-energy
physics</p>
Spring school in Experimental Mathematics (MathExp2018)2018-04-18T00:00:00+00:00http://opendreamkit.org/2018/04/18/saint_flour-2018<p><a href="http://opendreamkit.org">OpenDreamKit</a> is organizing a
<a href="https://mathexp2018.sciencesconf.org/">spring school on experimental mathematics (“Mathématiques Expérimentales”)</a>.
The first week will focus on learning tools (4 courses given by expert in computer
science and mathematics as well as programming exercises). During the second week
each participant is asked to work on a programming project related to his or her
research.</p>
Introduction to HPC using the Jupyter notebook University of Sheffield, 12th of April 20182018-04-12T00:00:00+00:00http://opendreamkit.org/2018/04/12/sheff-hpc<h2 id="main-goals">Main goals</h2>
<p>Previous OpenDreamKit work provided us with Sun Grid Engine Support for Project Jupyter Hub (Deliverable 5.3) which gave users of High Performance Computing systems, such as the one at University of Sheffield, access to powerful supercomputer hardware via the notebook. This workshop acted to disseminate this work, encourage more people to use HPC via the newly developed interface and teach the basics of parallelisaton in Python.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Workshop materials were developed by Will Furnass (USH) and released online at https://github.com/RSE-Sheffield/hi-perf-ipynb. The workshop was delivered by Will Furnass and Mike Croucher at University of Sheffield.</p>
<h2 id="event-summary">Event summary</h2>
<p>A group of researchers from a cross-section of subjects were taught the basics of parallelization and HPC using OpenDreamKit developed technologies and tutorials. The event was publicized on twitter which lead to follow up queries about the underlying technology.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The event established that the technology is scalable enough to be used in production for a typical class size at HPC training events. The OpenDreamKit developed technology is now a permanent part of the HPC service at University of Sheffield with ongoing work continually released at https://github.com/RSE-Sheffield/jupyterhub-gridengine-sharc</p>
Atelier PARI/GP 2018b Roma (IT), 2018-04-16 to 2018-04-172018-04-04T00:00:00+00:00http://opendreamkit.org/2018/04/04/Atelier-PARI-GP<h2 id="main-goals">Main goals</h2>
<p>This was a teaching and dissemination meeting, by invitation from the Roman
Number Theory Association as a satellite event for their 4th
mini-symposium.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: B. Allombert and A. Page from Bordeaux.</p>
<p>OpenDreamKit funded travel and accomodation costs for the two instructors for about
3k Euro. The ALGANT consortium, LIA LYSM (CNRS) and University Roma Tre
co-funded the event.</p>
<h2 id="event-summary">Event summary</h2>
<p>This Atelier PARI/GP took place in Roma (Italy) from april 16th to
17rd, it was followed by a 3-day international research conference on
Number Theory. There were 30 participants for the Atelier.</p>
<p>The 2-day Atelier followed the same pattern as the preceding Oujda Atelier,
featuring a general introduction to PARI/GP and two
specialized courses (graduate level) in the mornings:</p>
<ul>
<li>Bill Allombert ``Elliptic curves over finite fields and number fields’’,</li>
<li>Aurel Page ``Algebraic number theory’’.</li>
</ul>
<p>Afternoons were devoted to practice sessions.</p>
<p>Slides for all talks are available at
http://pari.math.u-bordeaux.fr/Events/PARI2018b/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a successful teaching and dissemination event.</p>
Interfacing (math) software with low level libraries2018-03-19T00:00:00+00:00http://opendreamkit.org/2018/03/19/cernay-2018<p><a href="http://opendreamkit.org">OpenDreamKit</a> is organizing a
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/251">workshop on Interfacing (math) software with low level libraries</a>.</p>
Toward versatile JupyterHub deployments, with the Binder and JupyterHub convergence2018-03-15T00:00:00+00:00http://opendreamkit.org/2018/03/15/jupyterhub-binder-convergence<h1 id="about-this-document">About this document</h1>
<p>Nowadays, many institutions run a <a href="http://jupyterhub.org">JupyterHub</a>
server, providing their members with easy access to <a href="http://jupyter.org">Jupyter</a>-based
virtual environments (a.k.a. notebook servers), preinstalled with a
stack of computational software, tailored to the typical needs of the
institution’s members. Meanwhile, since a few years ago,
<a href="http://mybinder.org">Binder</a> lets any user on the internet define,
run, and share temporary virtual environments equipped with an
arbitrary software stack (<a href="https://jupyter.org/try">examples</a>).</p>
<p>In Fall 2017, Binder was revamped as BinderHub, a lightweight layer on
top of JupyterHub. The next step in this convergence is to bring
together the best of both worlds: think persistent authenticated
<a href="http://mybinder.org">Binder</a>; or
<a href="https://github.com/jupyter/repo2docker">repo2docker</a> enabled
<a href="http://jupyterhub.org">JupyterHub</a>. For now, let’s call them <em>versatile
JupyterHub deployments</em>.</p>
<p>This document brainstorms this convergence process: it sets up the
ground with a scenario and assumptions for a typical institution-wide
JupyterHub deployment, proposes specifications from a user perspective,
and describes some typical use cases that would be enabled by such
specifications. It further discusses security aspects and what remains
to be implemented, before concluding with more advanced features and
open questions.</p>
<p>This document started as a collection of private notes
reflecting on in-development
<a href="http://jupytercloud.lal.in2p3.fr/">JupyterHub deployment at Paris-Saclay</a>
and <a href="https://jupyterhub.fedcloud-tf.fedcloud.eu/">EGI</a> respectively,
with <a href="https://github.com/OpenDreamKit/OpenDreamKit.github.io/commits/master/_posts/2018-03-15-jupyterhub-binder-convergence.md">some additional contributions</a>.
They were largely informed by many discussions at
March 2018’s <a href="https://pad.unistra.fr/p/jupyterhub-dev">JupyterHub coding sprint in Orsay</a>
that involved dev-ops
of those deployments and two of the main JupyterHub and BinderHub
devs: Min Ragan Kelley and Chris Holdgraf. It was also inspired by
some of <a href="https://cocalc.com">cocalc</a> features. Silly ideas reflecting
here are mine, hard work is theirs; thank you all!!!</p>
<p>This document is meant for brainstorming; please
<a href="https://github.com/OpenDreamKit/OpenDreamKit.github.io/tree/master/_posts/2018-03-15-jupyterhub-binder-convergence.md">hop in and edit</a>.</p>
<h1 id="typical-scenario">Typical scenario</h1>
<p>An institution – typically a university, a national lab, a
transnational research infrastructure such as the <a href="http://xfel.eu">European
XFEL</a>, or transational infrastructure provider like
<a href="http://egi.eu">EGI</a> – wishes to provide its members and users with a
Jupyter service.</p>
<p>The service lets user spawn and access personal or collaborative
virtual environments: namely a web interface to a light weight virtual
machine, in which they can use Jupyter notebooks, run calculations,
etc. In the remainder of this document we will use JupyterHub’s
terminology and call such
virtual environments <em>notebook servers</em>.</p>
<p>To cater for a large variety of use cases in teaching and research,
the main aim of the upcoming specifications is to make the service as
versatile as possible. In particular, it should empower the users to
customize the service (available software stack, storage setup, …),
without a need for administrator intervention.</p>
<h1 id="assumptions">Assumptions</h1>
<p>The institution has access to:</p>
<ul>
<li>
<p>An authentication service (Single Sign-On)</p>
<p>Examples: Paris-Sud’ Adonis internal SSO, the federated “Recherche
et Enseignement Supérieur” authentication service of Renater, EGI
CheckIn, …</p>
</li>
<li>
<p>Computing resources</p>
<p>Examples: a local cluster, access to a externalized cloud (GC, AWS,
Azure, …)</p>
</li>
<li>
<p>A shared volume service using the above authentication service</p>
<p>E.g. a local NextCloud service, or …</p>
</li>
<li>
<p>(Optional) a forge</p>
<p>Examples: a local gitlab service, github, … if private
repositories are needed, the forge presumably will need the same
authentication service</p>
</li>
</ul>
<h1 id="specifications--user-story">Specifications / User Story</h1>
<h2 id="main-page-of-the-service">Main page of the service</h2>
<p>After authentication, the user faces a page that is similar to
binder’s main page:</p>
<ul>
<li>
<p>A form to describe and launch the desired persistent notebook
server.</p>
<p>For the sake of simplicity, the form could optionally start hidden,
and be replaced by two items: “Choose preconfigured notebook server”
/ “Create my own notebook server”.</p>
</li>
<li>
<p>Links to documentation</p>
</li>
<li>
<p>Warnings about potential security concerns, to inform the user
choices.</p>
<p>Alternatively, such warnings could be displayed in a later security
confirmation dialog “with the given configuration, a malicious
image/collaborator could …; do you trust it? Proceed/Edit
Configuration/Cancel/Don’t ask again”</p>
</li>
<li>
<p>Institutional credits (service provided by …)</p>
</li>
</ul>
<p>The form consists of:</p>
<ul>
<li>The usual binder items:
<ul>
<li>the description of the computing environment: a repo-to-docker-style git repo+branch+…</li>
<li>the file/url to open on startup</li>
<li>…</li>
<li>a UI to get a URL/badge referencing the machine</li>
</ul>
</li>
<li>Persistence and access options:
<ul>
<li>
<p>server_name: name to give to the server</p>
<p>If server_name is not specified, create a random server name?</p>
</li>
<li>
<p>mount options: [[mount point, volume url], […]]</p>
<p>This assumes that the user has appropriate credentials to access
the given volumes through the authentication service</p>
</li>
<li>
<p>collaborators=[….]: (optional) a white list of other users of this jupyterhub that can access this server</p>
</li>
<li>
<p>a flag allowing public ‘temporary read-only’ access (meaning
that the container and all changes are thrown away at the end of
the session; and that any ‘mounted’ data sources are read-only
during the session)</p>
<p>alternatively</p>
</li>
<li>credentials: whether to pass the user credentials into the container (as environment variable, or file)</li>
<li>resources scaling (optional): memory, number of processors,
duration (time or time of inactivity after which the server will
be automatically stopped / destroyed)</li>
</ul>
</li>
</ul>
<h2 id="behavior-upon-clicking-launch">Behavior upon clicking Launch:</h2>
<ul>
<li>If a notebook server with the given name already exists and the
parameters are not changed (or not set): connect to that server,
restarting it if needed</li>
<li>If the parameters have been changed, update the existing server when
possible? Or should the user just delete the server?</li>
<li>Otherwise, create and launch</li>
</ul>
<h2 id="behavior-upon-following-a-server-urlbadge">Behavior upon following a server URL/badge:</h2>
<ul>
<li>Display the authentication page (if not yet authenticated)</li>
<li>Display a security confirmation dialog as above (if origin is not within the
jupyterhub), with a description of the configuration and origin.</li>
<li>As above after clicking “Launch”</li>
</ul>
<h1 id="some-use-cases">Some use cases</h1>
<h2 id="local-binder-better-name-binderhome">Local binder (better name? [Binder@home?])</h2>
<p>Scenarios:</p>
<ul>
<li>Luc, a researcher, discovered a nice computing environment
on Binder. Maybe a notebook demonstrating an interesting workflow to
analyze data. He wants to use it more intensively on his own data.</li>
<li>Lucy has found a notebook & binder environment published with a
paper, and she wants to re-execute the notebook to reproduce the
published results and start her research in the field. However, no
binder (compute) resources are available in the cloud. The
computation takes 20 minutes on a standard PC and she would like to
run this calculation on her local server.</li>
</ul>
<p>Setup:</p>
<p>They recreate the same environment on their local server (for example
by just changing the server name in the binder URL).</p>
<p>More advanced scenario to explore: Lucy would like to use her Desktop
PC because that resource is readily available and idles 99% of the
time.</p>
<h2 id="easy-sharing-of-computational-environments">Easy sharing of computational environments</h2>
<p>Scenarios:</p>
<ul>
<li>Sarah, a power user, is using some specialized stack of software on
a daily basis; maybe she authored some of it. She wants her
colleagues in her lab to try out that software.</li>
<li>Paul organizes a training session for his colleagues.</li>
<li>Alice has authored notebooks that she wants to share with her
colleagues. Maybe the notebooks automatize some complicated
processes and present them in the form of interactive web
applications built using <a href="http://jupyter.org/widgets">Jupyter widgets</a>
(<a href="https://mybinder.org/v2/gh/binder-examples/appmode/master?urlpath=apps%2Findex.ipynb">demo</a>).
Think of a wizard to setup parameters and data, run a computation,
and visualize the results.</li>
</ul>
<p>Setup:</p>
<p>They prepare a notebook server with the appropriate software stack
installed configured and access to the user’s shared home directory.
Maybe they provide some document. They then just have to share the URL
with their colleagues. No lengthy software installation. The
colleagues can then start working right away, in their own
environment, using their own data, saving their work in their home
directory.</p>
<p>In all cases, the explicit description of the computing environment
(and the use of open source software!) eases:</p>
<ul>
<li>the publication of the same computational environment / notebooks
elsewhere, e.g. on a public Binder;</li>
<li>the installation the same software on the user’s personal computer.</li>
</ul>
<h2 id="collaboration">Collaboration</h2>
<p>Scenario: Alice and Bob want to collaborate on some data analysis.</p>
<p>Setup:</p>
<p>They create a shared volume. Then either:</p>
<ul>
<li>They set up each their own notebook server, and let them share the
same volume.</li>
<li>Alice sets up a single server, with Bob as collaborator. Within the
server, they are considered as the same user.</li>
</ul>
<p>At this stage, they should not edit the same notebook simultaneously.
However the stable version of JupyterLab, due sometime in 2018, should
enable real-time collaboration in both setups, thanks to
a <a href="https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type">CRDT</a>
file format for notebooks.</p>
<h2 id="class-management">Class management</h2>
<p>Scenario: using the server for a class’ computer labs and assignments</p>
<p>Desired features:</p>
<ul>
<li>Full customizability of the computing environment by the teacher;</li>
<li>Support for live manipulation of the class notes;</li>
<li>Support for submission, collection and auto-grading of assignments;</li>
<li>Access from the computer labs or outside (home, …);</li>
<li>Possibility to either use the server, needing only a web browser (no
software installation required; supports phones, tablets, …), or
install and run the software locally.</li>
</ul>
<p>Prerequisites:</p>
<ul>
<li>A JupyterHub instance, configured as above, accessible from the teachers and students;</li>
<li>A forge such as gitlab or github, accessible from JupyterHub</li>
<li>A shared drive service (e.g. next cloud/nsf/…), serving home directories,
and letting teachers setup shared volumes</li>
<li>A shared authentication (e.g. SSO), so that notebook servers in JupyterHub can access
the shared drive.</li>
<li>Some web server</li>
</ul>
<p>Procedure for the teacher(s):</p>
<ul>
<li>Set up a shared volume for the whole class</li>
<li>
<p>Prepare a computing environment in a git repository on the forge.</p>
<p>Typically includes: computational software, [nbgrader] + configuration, …</p>
</li>
<li>Prepare the course material typically in a git repository on the forge
(the same one or another)</li>
<li>Use JupyterHub’s form UI to setup (and test) a full description of the student’s
notebook servers, with mounting of the home directory (or subdirectory thereof?) and shared volume.
Possibly add the teacher(s) as collaborator(s) ???
Get the corresponding URL.</li>
<li>Possibly prepare a variant thereof for teachers of the class.</li>
<li>
<p>Set up a web page for the class, with hyperlink(s) to the above URL.</p>
<p>There can typically be an hyperlink for each session pointing directly
to the exercises for that particular session.</p>
</li>
</ul>
<p>Fetching the class material:</p>
<ul>
<li>Option 1: manually download from the web (wget in the terminal, or web upload, …) or shared volume</li>
<li>Option 2: use <a href="https://github.com/data-8/nbgitpuller">nbgitpuller</a> from the command line</li>
<li>Option 3: use <a href="https://github.com/jupyter/nbgrader">nbgrader</a>, either from the command line or with the UI to get the files from the shared volume</li>
<li>Option 4: automatize the above using a notebook server extension such as that for nbgitpuller</li>
</ul>
<p>Submitting assignments:</p>
<ul>
<li>Use nbgrader, either from the command line or with the UI to push the files to the shared volume</li>
</ul>
<p>To explore: integration with local e-learning platforms like Moodle,
typically using <a href="https://www.imsglobal.org/activity/learning-tools-interoperability">LTI</a>,
in particular for class management and grades reporting.
There already exists an <a href="https://github.com/jupyterhub/ltiauthenticator">LTI authenticator for Jupyter</a>.</p>
<h1 id="security-concerns">Security concerns</h1>
<p>A malicious image description, image, or collaborator can:</p>
<ul>
<li>Take any action within the image being built or within the notebook server</li>
<li>Waste computing resources (cpu/memory);</li>
<li>With internet: connect to any website, without specific priviledges
(e.g. participate to a DOS attack); abuse computing resources, e.g.
for bitcoining. The image building almost certainly needs internet
access.</li>
<li>With persistent storage and internet access: access and leak
information from the storage; e.g. private information from the user;</li>
<li>With read-write persistent storage: corrupt the storage (e.g. the user’s home directory)</li>
<li>With credentials: take any action on behalf of the user in any
service that use the same authentication.</li>
</ul>
<h1 id="implementation-status">Implementation status</h1>
<p>Most of the features are already there. Here are some of the missing steps:</p>
<ul>
<li>Extending binder’s form as described above;</li>
<li>Implementing the logic to mount shared volumes;</li>
<li>
<p>Instructions / scripts for setting up a local docker registry;</p>
<p>The current Binder installation tutorial assumes that a docker
registry is already available; e.g. that provided by google cloud
services.</p>
<p>For a smaller setup using the same host for both building images and
running notebook servers, no docker registry is needed. In this
case, JupyterHub could just run repo2docker locally before launching
the notebook server. repo2docker however does not implement image
caching; so a simplified version of the image cache mechanism of
Binder needs to be implemented.</p>
</li>
</ul>
<h1 id="alternatives">Alternatives</h1>
<p>It should be noted that there is basically no coupling between
JupyterHub/Binder and Jupyter. The former is merely a general purpose
web service for provisioning web-based virtual environments. For
example, JupyterHub/Binder has also been used to serve R-Studio based
virtual environments. Reciprocally, there are alternative services to
do such provisioning from which to get inspiration, like
<a href="https://www.simulagora.com/">Simulagora</a>.</p>
<h1 id="advanced-features--open-questions">Advanced features & open questions</h1>
<h2 id="redirection">Redirection</h2>
<p>The main form could contain an additional item:</p>
<ul>
<li>URL input / dropdown menu to choose another jupyterhub instances to
redirect to.</li>
</ul>
<p>Use cases:</p>
<ul>
<li>A user finds a nice image on binder; he wants to customize it to
run it on his institution’s jupyterhub; possibly adding persistent
storage to it. Or reciprocally: a user wants to share on binder a
local image of his.</li>
<li>An institution wants to advertise other jupyterhub instances; this
could e.g. be used by a single entry point for federating a
collection of instances (e.g. all French academic JupyterHub’s).</li>
</ul>
<h2 id="marketplace-of-images">Marketplace of images</h2>
<p>With the URL mechanism, any (power) user can prepare a dedicated image
and share it with his collaborators. Images can be more or less
specific: from just a computing environment to a fully specified
machine, with mount points, …</p>
<p>Thanks to the above there is no need for a tightly coupled Marketplace.
Nevertheless it may be useful to have one location (or more) for
collecting and publicizing links to popular images. Some minimal
coupling may be needed if one would want to sort the images according
to their popularity.</p>
<p>Note: at this stage, a user cannot produce an image by setting up a
machine “by hand” and save its state. The construction must be fully
scripted. On the plus side, this encourages users to script their
images, making them more reproducible.</p>
<p>National and international initiatives such as the <a href="http://opendreamkit.org/2017/12/06/EOSC/">European Open
Science Cloud</a> may help
providing such a catalog of relevant Jupyter notebooks/images.</p>
<h2 id="default-volume-configuration">Default volume configuration</h2>
<ul>
<li>Choose good defaults, if at all possible compatible with binder.
Main question: where should the files provided by the binder
environment be copied? In a subdirectory of the persistent home? In
the home directory, with the persistent home being mounted in a
subdirectory thereof?</li>
</ul>
<h2 id="intensive-use-and-resource-management--accounting">Intensive use and resource management / accounting</h2>
<p>The above has been written with casual use in mind. For extensive use,
some form of accounting and controlling of the resources used would be needed. For
example, for LAL’s cloud we may want to have some form of bridge
between the OpenStack dashboard and the hub. UI to be designed. Could
the user provision a machine using the dashboard, and then specify on
JupyterHub that the container shall be run on that machine?</p>
OpenDreamKit at the RSE conference2018-03-07T00:00:00+00:00http://opendreamkit.org/2018/03/07/opendreamkit-at-the-rse-conference<h1 id="open-dream-kit-presence-at-the-second-rse-conference">Open Dream Kit presence at the second RSE conference</h1>
<p>The Manchester Museum of Science and Industry (MOSI) saw the <a href="http://rse.shef.ac.uk/blog/a-successful-2nd-rse-conference/">second Research
Software Engineering (RSE) conference</a> on September 7-8th, 2017.
Over 200 attendees gathered to discuss ways to better support and advance science via software, innovative computational tools, and policy making.
There were more than 40 talks and 15 workshops covering a diverse range of topics, from community building to imposter syndrome, data visualization, and High-Performance Computing.</p>
<p>Attending the conference is an excellent opportunity to integrate within the international <a href="http://rse.ac.uk/">RSE
community</a> and appreciate how much this has grown over the last few years.
All thanks to the great work done by RSEs within their institutions and their efforts
to make software a valuable research asset.
It will be, for certain, interesting to see how this will continue to grow and
evolve as policy changes take place and more research councils, funding bodies,
and research institutions acknowledge the importance of research software and
its scientific impact.</p>
<p>OpenDreamKit member, Tania Allard, ran a hands-on workshop on <strong>Jupyter notebooks
for reproducible research</strong>. This workshop focused on the use of Jupyter notebooks
as a means to disseminate reproducible analysis workflows and how this can be
leveraged using tools
such as <a href="https://github.com/jupyter/nbdime">nbdime</a> and <a href="https://github.com/computationalmodelling/nbval">nbval</a>. Both nbdime and nbval
were developed by members of the OpenDreamKit project as a response to the
growing popularity of the Jupyter notebooks and the lack of native integration
between these technologies and existing version control and validation/testing tools.</p>
<p>An exceptional win was that this workshop was, in fact, <strong>one of the most popular
events of the conference and we were asked to run it twice as it was massively
oversubscribed</strong>.
This reflects, on one hand, the popularity of Jupyter notebooks due to the boom of
literate programming and its focus on human-readable code. Allowing
researchers to share their findings and the code they used along the way in a
compelling narrative. On the other hand, it demonstrates the importance of
reproducible science and the need for tools that help RSE and researchers to
achieve this goal, which aligns perfectly with the goals of OpenDreamKit.</p>
<p>The workshop revolved around 3 main topics:</p>
<ol>
<li>Version control of the Jupyter notebooks</li>
<li>Notebooks validation</li>
<li>The basics of reproducible software practices.</li>
</ol>
<p>The main focus was on how tools like nbdime and nbval can support people already using
Jupyter notebooks but have struggled to integrate these with software best development
practices due to a number of limitations on the existing tools. Then, we followed on
other actions that can be taken to ensure that their data analysis workflows were
reproducible and sustainable.
This lead to a number of interesting discussions about the topic and allowed for
the attendees to share their previous experiences regarding reproducibility and/or the lack
thereof in different research areas.</p>
<p>We plan to run a set or workshops around reproducibility over the duration of the
ODK project and we’ll make sure to report on them here too. Finally, all the materials are licensed under
CC-BY and can be found <a href="https://github.com/trallard/JNB_reproducible">in this GitHub repository</a> .</p>
JupyterHub and Binder coding sprint Université Paris Sud, March 7-82018-03-07T00:00:00+00:00http://opendreamkit.org/2018/03/07/jupyterhub-orsay<p>## Main goals</p>
<p>This event was organized as a satellite of a
Jupyter dissemination conference at École Polytechnique. It brought
together developers of the Virtual Environments JupyterHub and
BinderHub and DevOps working on deployments in Orsay and at EGI for
a coding sprint. The goal was to improve those deployments, as a use
case from which to learn and share procedures and best practices.</p>
<p>## OpenDreamKit implication</p>
<p>Organization and funding of OpenDreamKit participants; $\approx$ 1k Euro.</p>
<p>## Results and impact</p>
<p>Turnkey deployment instructions already
existed for deploying JupyterHub and BinderHub on top of cloud
infrastructure provided by e.g. Google Cloud, or Microsoft Azure.
This workshop was a major step for the first deployment of a
JupyterHub and BinderHub instance on top of an OpenStack cloud
infrastructure. This is important as OpenStack is widely adopted in
academia-run cloud infrastructures, yet raises some unique
challenges due to its high customizability. Many notes were taken
and shared. In addition a blog post summarized a brainstorm on the
upcoming convergence between JupyterHub and BindherHub, toward
providing versatile JupyterHub deployments that lets its user
define, run, and share virtual environments equipped with an
arbitrary software stack.</p>
<p>https://opendreamkit.org/2018/03/15/jupyterhub-binder-convergence/</p>
Research Software Engineer position opening at Université Paris-Sud (filled)2018-02-21T00:00:00+00:00http://opendreamkit.org/2018/02/21/position-paris-sud<p>This is an announcement for a research software engineer position
opening at Université Paris-Sud, working on web-based user interfaces
and semantic interoperability layers for mathematical computational
systems and databases.</p>
<h1 id="time-line">Time line</h1>
<p>Interviews in March 2018 for a recruitment as soon as possible.</p>
<p>Update (March 27th): after the interviews on March 21st, we selected
and ranked two candidates and made an offer to the first one. Pending
administrative hoops, they will take the position.</p>
<h1 id="salary">Salary</h1>
<p>For a full-time position, and depending on the applicant’s past
experience, between 2000€ and 3000€ of monthly “salaire net” (salary
after non-wage labour cost but before income tax). Equivalently, what
this salary represents for is a “salaire brut” of up to 46200€ yearly.
We have secured funding until the end of the project (August 2019).</p>
<h1 id="location">Location</h1>
<p>The research software engineer will work at the
<a href="http://www.lri.fr">Laboratoire de Recherche en Informatique</a> of
<a href="http://www.u-psud.fr">Université Paris Sud</a>, in the
Orsay-Bures-Gif-Saclay campus, 25 km South-West of Paris city centre.</p>
<h1 id="mission-and-activities">Mission and activities</h1>
<p>Paris Sud is the leading site of OpenDreamKit, with eight participants
involved in all the work packages. The research software engineer will
join that team and support its efforts in WP4 and WP6, targeting
respectively Jupyter-based user interfaces and interoperability for
mathematical computational systems and databases. A common theme is how
to best exploit the mathematical knowledge embedded in the systems.
For some context, see e.g. the <a href="http://opendreamkit.org/2017/10/15/WP6-Usecase/">recent
publications</a> describing
the Math-In-The-Middle approach.</p>
<p>More specifically, a successful candidate will be expected to
contribute significantly to some of the following tasks (see also
<a href="https://github.com/OpenDreamKit/OpenDreamKit/raw/master/Proposal/proposal-www.pdf">OpenDreamKit’s Proposal</a>:</p>
<ul>
<li>
<p>Dynamic documentation and exploration system (Task 4.5)</p>
<p>Introspection has become a critical tool in interactive computation,
allowing user to explore, on the fly, the properties and
capabilities of the objects under manipulation. This challenge
becomes particularly acute in systems like Sage where large parts of
the class hierarchy is built dynamically, and static documentation
builders like Sphinx cannot anymore render all the available
information.</p>
<p>In this task, we will investigate how to further enhance the user
experience. This will include:</p>
<ul>
<li>
<p>On the fly generation of Javadoc style documentation, through
introspection, allowing e.g. the exploration of the class
hierarchy, available methods, etc.</p>
</li>
<li>
<p>Widgets based on the HTML5 and web component standards to display
graphical views of the results of SPARQL queries, as well as populating data
structures with the results of such queries,</p>
</li>
<li>
<p>D4.16: Exploratory support for semantic-aware interactive Jupyter widgets
providing views on objects of the underlying computational or
database components. Preliminary steps are demonstrated in the
<a href="http://www.larchenvironment.com/">Larch Environment</a> project (see
demo videos) and
<a href="https://github.com/jbandlow/sage-explorer">sage-explorer</a>. The
ultimate aim would be to automatically generate LMFDB-style
interfaces.</p>
</li>
</ul>
<p>Whenever possible, those features will be implemented generically
for any computation kernel by extending the Jupyter protocol with
introspection and documentation queries.</p>
</li>
<li>
<p>Memoisation and production of new data (Task 6.9)</p>
<p>Many CAS users run large and intensive computations, for which they
want to collect the results while simultaneously working on software
improvements. GAP retains computed attribute values of objects
within a session; Sage currently has a limited <code class="language-plaintext highlighter-rouge">cached_method</code>.
Neither offers storage that is persistent across sessions or
supports publication of the result or sharing within a
collaboration. We will use, extend and contribute back to, an
appropriate established persistent memoisation infrastructure, such
as <code class="language-plaintext highlighter-rouge">python-joblib</code>, <code class="language-plaintext highlighter-rouge">redis-simple-cache</code> or <code class="language-plaintext highlighter-rouge">dogpile.cache</code>, adding
features needed for storage and use of results in mathematical
research. We will design something that is simple to deploy and
configure, and makes it easy to share results in a controlled
manner, but provides enough assurance to enable the user to rely on
the data, give proper credit to the original computation and rerun
the computation if they want to.</p>
</li>
<li>
<p>Knowledge-based code infrastructure (Task 6.5)</p>
<p>Over the last decades, computational components, and in particular
Axiom, MuPAD, GAP, or Sage, have embedded more and more
mathematical knowledge directly inside the code, as a way to better
structure it for expressiveness, flexibility, composability,
documentation, and robustness. In this task we will review the
various approaches taken in these software (e.g. categories and
dynamic class hierarchies) and in proof assistants like Coq
(e.g. static type systems), and compare their respective strength
and weaknesses on concrete case studies. We will also explore
whether paradigms offered by recent programming languages like Julia
or Scala could enable a better implementation. Based on this we will
suggest and experiment with design improvements, and explore
challenges such as the compilation, verification, or
interoperability of such code.</p>
</li>
</ul>
<h1 id="skills-and-background-requirements">Skills and background requirements</h1>
<ul>
<li>
<p>Degree in mathematics or computer science; PhD appreciated but not required;</p>
</li>
<li>
<p>Strong programming experience with languages such as Python, Scala,
Javascript, etc; experience with web technologies in general and the
<a href="http://jupyter.org/">Jupyter</a> stack in particular appreciated;</p>
</li>
<li>
<p>Experience in software design and practical implementation in large
software projects; experience with computational mathematics
software (e.g. <a href="http://sagemath.org">SageMath</a>) appreciated;</p>
</li>
<li>
<p>Experience in open-source development (collaborative development
tools, interaction with the community, …);</p>
</li>
<li>
<p>Strong communication skills;</p>
</li>
<li>
<p>Fluency in oral and written English; speaking French is not a
prerequisite.</p>
</li>
</ul>
<h1 id="context">Context</h1>
<p>The position will be funded by
<a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<p>Within this ecosystem, the applicant will work primarily on the free
open-source mathematics software system
<a href="http://sagemath.org">Sagemath</a>. Based on the
<a href="http://www.python.org">Python</a> language and many existing open-source
math libraries, SageMath is developed since 10 years by a worldwide
community of 300 researchers, teachers and engineers, and has reached
1.5M lines of code.</p>
<p>The applicant will work within one of the largest teams of SageMath
developers, composed essentially of researchers in mathematics and
computer science, at the <a href="http://www.lri.fr/">Laboratoire de Recherche en
Informatique</a> (LRI) and in nearby institutions.
The LRI also hosts a strong team working on proof systems.</p>
<h1 id="applications">Applications</h1>
<p>To apply for this position, please send an e-mail to
upsud-recruitement-research-engineer at opendreamkit.org
by March 10, with the following documents (in English) attached:</p>
<ul>
<li>
<p>cover_letter.pdf: a cover letter explaining your interest in this particular position;</p>
</li>
<li>
<p>CV.pdf: a CV, highlighting among other things your skills and
background and your contributions to open source software;</p>
</li>
<li>
<p>degree.pdf: copy of your most recent degree including (if
applicable) the reviewers reports;</p>
</li>
<li>
<p>reference letters: files reference_letter_<contactname>.pdf or contact information of potential referees.</contactname></p>
</li>
</ul>
<p>Applications sent after March 10 will be considered until the
position is filled.</p>
Sage Days 93 Olot (ES), 2018-02-19 to 2018-03-042018-02-19T00:00:00+00:00http://opendreamkit.org/2018/02/19/SageDays93<h2 id="main-goals">Main goals</h2>
<p>The main goal of the Sage Days 93 were to
gather researcher specialists in Lie groups and advanced
SageMath and PARI/GP developers. The two goals were to implement
more Lie group theory in the softwares and conversely produce
experimental results for researchers.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit organizer: V. Delecroix.</p>
<p>OpenDreamKit provided the funding source for the workshop (accommodation,
subsistence and travel expenses) for about 20k Euro.</p>
<h2 id="event-summary">Event summary</h2>
<p>The Sage Days 93 took place in Olot (Spain) from february 19th to
march 4th. It was focused on the theory of Lie groups and their
subgroups.</p>
<p>There were 11 registered participants. A typical day of the workshop
consisted in talks and hacking session.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a development workshop dedicated to researcher in Lie groups
and their lattices. We obtain interesting feedback and developed
a lot of features related to Lie groups in SageMath and PARI/GP.</p>
Remote project meeting2018-02-06T00:00:00+00:00http://opendreamkit.org/2018/02/06/project-meeting<p>This is an online project meeting to review all achievements since March 2017.</p>
<ul>
<li><a href="http://opendreamkit.org/meetings/2018-02-01-SteeringCommitteeMeeting/ProgressReports/">Progress reports</a></li>
</ul>
Advances in Magnetism 2018 - Computational micromagnetics with JOOMMF tutorial La Thuile, Italy, 04-07 Fabruary 20182018-02-03T00:00:00+00:00http://opendreamkit.org/2018/02/03/AIM<h2 id="main-goals">Main goals</h2>
<p>We gave a tutorial about computational micromagnetics and JOOMMF to all conference participants.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>JOOMMF was developed as a part of the OpenDreamKit project and one participant from the OpenDreamKit was present to deliver the tutorial (Marijan Beg). The tutorial was fully funded from the OpenDreamKit funds and the total costs were 1207.73 euros.</p>
<h2 id="event-summary">Event summary</h2>
<p>This tutorial was in the official part of the conference programme available to all atendees. We had about 100 participants and we introduced to them the basics of computational micromagnetics as well as gave them an introduction to JOOMMF. At the end, we answered any specific question attendees had.</p>
<h2 id="demographic">Demographic</h2>
<p>We had around 100 participants, but the organisers did not allow us to have their personal details due to the data protection.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>We informed the community of potential users about the benefits of JOOMMF and provided to them enough information if they want to start using it.</p>
Jupyter Widgets Workshop École Polytechnique, January 23-262018-01-23T00:00:00+00:00http://opendreamkit.org/2018/01/23/JupyterWidgets<h2 id="main-goals">Main goals</h2>
<p>Bringing together developers and power users of Jupyter Widgets.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Some organizational support; funding of OpenDreamKit participants; $\approx$ 1k Euro.</p>
<h2 id="event-summary">Event summary</h2>
<p>This was the first workshop specifically
centered on Jupyter widgets. If featured tutorials on how to implement
widgets, demonstrations of widget uses for 3D visualization and big
data analysis, improvements of core widget functionalities for better
integration, and implementation of new widgets. In addition the
workshop was the occasion to advance other Jupyter related task in
OpenDreamKit’s WorkPackage 4.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The intense cross-interactions between
developers from academia, Jupyter-centric SMÉ’s, and big institutions
(banks, US military) highlighted and further pushed the wide adoption
of this technology and the building of a community around it.</p>
Expérimentation mathématique et combinatoire avec Sage2018-01-23T00:00:00+00:00http://opendreamkit.org/2018/01/23/ENS<p>Viviane Pons gave a two hours lecture on mathematical experimentation, research and open-source mathematical software development for a seminar organized by the students of the prestigious school <em>Ecole Normale supérieure de Lyon</em>.</p>
<ul>
<li><a href="https://www.lri.fr/~pons/docs/2018_ENS.pdf">Slides</a></li>
<li><a href="https://github.com/VivianePons/public-notebooks/blob/master/2018-01-22-ENSLyon.ipynb">Sage Jupyter Notebook</a></li>
</ul>
Software tools for mathematics 2018-01 Morelia Morelia, Mexico, 2018-01-22--2018-01-262018-01-22T00:00:00+00:00http://opendreamkit.org/2018/01/22/STM-Morelia<h2 id="main-goals">Main goals</h2>
<p>The goal of the event was for mathematicians
to improve their coding skills and knowledge of mathematical software.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Samuel Lelièvre (OpenDreamKit member from UPSud) was one of
the organisers. The OpenDreamKit funds at Paris-Sud were used to fund travel
and stays of speakers, and lunch and coffee breaks for all participants.
Centro de Ciencias Matemáticas (CCM), the mathematics department at
UNAM Morelia, co-funded the event, paying travel and accommodation for
some participants.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event consisted in a two-day Software Carpentry
workshop (teaching participants the Unix shell, version control with Git,
and programming with Python) followed by three days on mathematical software
with mini-courses on CoCalc, GAP, Jupyter, PARI/GP, SageMath, YAGS, as well
as talks on other mathematical software and databases, and on mathematical
research using software. A problem session allowed participants to submit
mathematical problems they cared about and thought software might help with,
several of which were solved in the following days by other participants.</p>
<h2 id="demographic">Demographic</h2>
<p>Over 140 people registered: 36
10
and researchers, 10
50 participants, we focused on PhD students, postdocs, professors and
researchers.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>During the Software Carpentry workshop, Tania Hernandez taught the Unix shell,
Nelly Selem taught version control with Git, Leticia Vega taught programming
with Python. These courses were taught in Spanish, although using supporting
materials in English. This may have been the first time a Software Carpentry
workshop was taught entirely in Spanish. An effort is underway to translate
the Software Caprentry and Data Carpentry lesson plans to Spanish.</p>
<p>During the mathematical software part, Alexander Hulpke taught GAP, Samuel
Lelièvre taught CoCalc, Jupyter and SageMath, Miguel Pizaña taught YAGS
(a GAP package for working with graphs), Miguel Raggi presented Discreture
(a library for enumerating combinatorial objects), Emmanuel Royer taught
PARI/GP, Adrián Soto presented TeXmacs and gave a talk on Rauzy fractals,
Janoš Vidali presented DiscreteZOO, Rafael Villarroel presented Emacs,
Russ Woodroofe gave two talks (“The story of a calculation” and “The story
of a figure”), Katja Berčič gave a talk on “Databases of theorems”, and
Uziel Silva gave a presentation of Macaulay2, Greuel and Reveal.js.</p>
<p>Many participants told the organisers, orally or by email, that this workshop
was transformative for them; often they felt they had passed some confidence
threshold: whereas before the conference they were interested in mathematical
software but unsure how to install and use them, they were now confident how
to do that, and felt they had the necessary resources to learn more.</p>
<p>One of the participants who is part of the board of the Mexican Math Society
initially intended to visit briefly to check out our workshop briefly, and
ended up staying the whole week, staying at the install party during the free
afternoon, and got convinced of the importance of having a software component
in future mathematics conferences in his area. As a result, David Sanders
gave a course on “numerical methods for dynamical systems”, based on Julia,
at the next national dynamical systems conference in Mexico in June 2018.</p>
<p>Another of the organisers, Katja Berčič, a Slovenian post-doc currently in
Morelia, Mexico, liked the format of this workshop so much that a new workshop
on the same format is planned for September 2018 in Koper, Slovenia.</p>
Atelier PARI/GP 2018 Besan\c{c}on (FR), 2018-01-15 to 2018-01-192018-01-01T00:00:00+00:00http://opendreamkit.org/2018/01/01/Atelier-PARI-GP<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: B. Allombert, K. Belabas, V. Delecroix, J. Demeyer,
J.-P. Flori, L. de Feo.</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and travel expenses), for about 13k Euro. The Besan\c{c}on
institute of mathematics co-funded the event.</p>
<h2 id="event-summary">Event summary</h2>
<p>The 10th Atelier PARI/GP took place in Besan\c{c}on (France) from january
15h to 19th.</p>
<p>There were 38 registered participants from 19 different institutions
(no registration fees).</p>
<p>A typical day of the workshop had introductory talks and tutorials
in the morning; afternoons allowed ample time for hacking sessions,
discussions and training.</p>
<p>The Atelier featured 8 morning talks on mathematical topics and
implementation projects including 4 talks by OpenDreamKit members</p>
<ul>
<li>Karim Belabas ``Modular Forms’’</li>
<li>Bill Allombert ``New GP features’’, \texttt{lfun} and Artin
$L$-functions.</li>
<li>Luca de Feo and Jean-Pierre Flori ``Algorithms for lattices of
compatibly embedded finite fields’’.</li>
</ul>
<p>Slides for all talks are available at
http://pari.math.u-bordeaux.fr/Events/PARI2018/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop was very productive and particularly beneficial to WP5
(high-performance computing), it provided final feedback on recent PARI/GP
modules and interfaces that paved the way for the release of
pari-2.10-alpha (2018/05) and pari-2.11-stable (2018/07, the first major
release since 2016).</p>
<p>It also was a successful dissemination event: 14 participants had not come to
a previous Atelier.</p>
State of the European Open Science Cloud and strategy of OpenDreamKit2017-12-06T00:00:00+00:00http://opendreamkit.org/2017/12/06/EOSC<p>During the last week of November two events related to the European Open Science Cloud (EOSC) took place in Brussels: the <a href="https://eoscpilot.eu/eosc-stakeholder-forum-shaping-future-eosc">EOSC stakeholder forum</a> on 28-29 November and the 2017 edition of the <a href="https://www.digitalinfrastructures.eu/">DI4R (Digital Infrastructures for Research)</a>. These two events were closely related in regard with the momentum period 2018-2020 for the Scientific Community and the advancement of open and digital science.</p>
<h2 id="the-vision">The vision</h2>
<p>The European Open Science Cloud (EOSC) is currently a process to build a digital platform that is inspired by the F.A.I.R. principle. <strong>F.A.I.R. stands for: Findable, Accessible, Interoperable and Reusable.</strong></p>
<p>FAIR is the idea between the recent actions taken by the European Commission and various funding agencies in favour of open data and open science.
The EOSC platform, also called the EOSC-Hub, intends <em>in fine</em> to make possible to the whole European scientific Community and beyond: exchange of data, easy access to knowledge and access to all useful infrastructures for all scientific disciplines.</p>
<p>The word “Open” in EOSC must be understood in every possible way. It means (not exhaustively) that the platform will :</p>
<ul>
<li><strong>promote open source</strong> software and open access culture</li>
<li>make data <strong>as open as possible but as closed as necessary</strong> in regard with confidentiality and IPR (Intellectual Property Rights) concerns,</li>
<li>be a <strong>mix of paid and free services and/or software</strong></li>
<li>be open to <strong>all users</strong>: research centres, public bodies, companies, citizens</li>
<li>be accessible <strong>worldwide</strong></li>
<li>gather all infrastructures and technologies that are of help for scientists from <strong>all possible disciplines</strong></li>
</ul>
<p>Technically speaking, the general idea is to federate existing and yet to appear services, and to make these services <strong>interoperable with one another</strong>. Services and software can concern for example data and computation hosting, authentication, indexing, collaborative tools, data and service catalogues, etc. Thanks to the interoperability the researchers will be able to discover, navigate, use and re-use, data and combine them suing a various number of infrastructures. The interpretation of data is (or at last should be in our reading) quite broad here: it includes metadata and provenance, data models, tools and generally knowledge required to make sense of the data.</p>
<p><img src="/public/images/EOSC-photo-big.JPG" alt="EOSC photo" class="img-responsive" /></p>
<p>The core role of EOSC-Hub is to coordinate, foster interoperability, develop glue services, and generally speaking steer the efforts toward the needs of researchers. The outcome of the process is still unknown but the ambition is grand. Indeed, an EU official compared the EOSC to the internet. The WWW has been and still is a process, and it has changed the face of the Earth. <strong>The ambition of the EOSC is to change the way we do Science.</strong>
An explanatory video is <a href="https://www.youtube.com/watch?time_continue=141&v=SC4-O8BmI4I">available here.</a></p>
<p>The EOSC will eventually be linked to the future pan-European HPC, as well as with the future European Data Infrastructure (EDI) which are funded alongsie the EOSC by the European Commission. The EDI and the EOSC may at the end of the day merge, since differences of purpose for each is not yet fully clear.</p>
<p><img src="/public/images/EOSC-sketch.JPG" alt="EOSC sketch" class="img-responsive" /></p>
<h2 id="political-support-and-public-funding">Political support and public funding</h2>
<p>The EOSC is part of two strategies of the European Commission: the <a href="https://ec.europa.eu/digital-single-market/en/policies/shaping-digital-single-market">Digital Single Market</a> on one hand and the <a href="http://ec.europa.eu/research/era/index_en.htm">European Research Area</a> on the other hand.</p>
<p>The EOSC is the combination of the two, as <strong>it aims at creating a single research community without country or technical barriers</strong> (interoperability between software and services).</p>
<p>In its <a href="http://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-infrastructures_en.pdf">Work Programme 2018-2020 for European Research Infrastructures</a>, the European Commission has put 375 million Euros on the table for “implementing the European Open Science Cloud”. It was initially planned to open a topic of <strong>79M€ for adding other services and infrastructures to the EOSC, but it was postponed after 2020</strong> for the 9th Framework Programme for research and Innovation. According to Augusto Burgueño Arjon, Head of the Unit “eInfrastructure & Science Cloud” at the DG Communication and Networks at the European Commission, <strong>it is not yet decided what this fund will be used for as it will depend on the evolution of the EOSC-Hub</strong>. AS according to the Commission and main supporters <strong>there is no room for failure</strong>, the next 3 years will be crucial. If EOSC implementation is a success, the first half of FP9 will focus on aggregating remaining infrastructures and services while the second half of FP9 will focus on sustaining and scaling the EOSC.</p>
<p>Member States are pushing forward the EOSC as well. Germany and the Netherlands issued in May 2017 a <a href="https://www.government.nl/documents/reports/2017/05/30/joint-position-paper-on-the-european-open-science-cloud">joint position paper</a> on the EOSC, which France <a href="https://twitter.com/VidalFrederique/status/936650264899178497">officially joined</a> on the 01/12/2017. Furthermore 13 Member States have of to date signed an agreement to start a pan-European HPC programme. This new infrastructure will follow Prace and will be closely connected to EOSC.</p>
<h2 id="infrastructures-involved">Infrastructures involved</h2>
<p>Outside of public institutions there are 6 major players pushing forward the EOSC:</p>
<ul>
<li><a href="https://www.egi.eu/">EGI</a> (federation of cloud providers and data centres)</li>
<li><a href="https://eudat.eu/">EUDAT</a> (data services),</li>
<li><a href="https://www.openaire.eu/">OpenAire</a> (open publication services),</li>
<li><a href="https://www.geant.org/">Géant</a> (network services)</li>
<li><a href="https://www.indigo-datacloud.eu/">Indigo Data-Cloud</a> (cloud services for science)</li>
</ul>
<p>Together they built a consortium of 74 partners coordinated by EGI, in order to answer the call H2020-EINFRA-12-2017(a) within the Work Programme 2018-2020. This project named “EOSC-Hub project” will receive 30M€ by the Commission for the 2018-2020 period. Beneficiaries include Research Infrastructures, national e-Infrastructure providers, SMEs and academic institutions. This consortium will be focused on and addressing issues of the EOSC such as interoperability, adoption of open standards and protocols, governance structure etc</p>
<p>Indeed, our impression is that - while the vision is clear - <strong>the design and realisation of this vision has no clear shape yet</strong>: many ideas and components are floating around, some fully or half realised but simultaneously many questions remain unanswered or have not even been asked yet. This state of affairs is not surprising: the vision is grand and has the potential to disrupt (positively) the way in which research is carried out today; the actors are human beings, scientists, institutes, funding bodies and states all with their own priorities and constraints.
Furthermore, <strong>there are real technical challenges in putting this “Cloud” together, and there are also cultural challenges to move more and more research activities towards Open Science</strong>. Existing metrics for academics and research institutions do not generally incentivise open science; which makes change of behaviour difficult. The biggest challenge for the EOSC is then maybe the challenge of skills and habit because as it was heard from an EU official: <strong>“if we do all this and no one is using it, it will be worthless”.</strong></p>
<p>Some infrastructures and services were presented during the DI4R conference as “EOSC building blocks”. The presentations are all available following <a href="https://indico.egi.eu/indico/event/3455/timetable/#all">this link</a>.</p>
<p>Two presentations attracted our attention:</p>
<ul>
<li><a href="https://help.hubzero.org/">Hubzero</a> <a href="https://indico.egi.eu/indico/event/3455/session/7/contribution/137/material/slides/0.pdf">presentation</a>: open source platform for scientific and educational collaboration</li>
<li><a href="https://indico.egi.eu/indico/event/3455/session/7/contribution/137/material/slides/0.pdf">Presentation</a> introducing to FAIR: nothing new but explains principles for EOSC</li>
</ul>
<p>Contributions are open on Github to <a href="https://github.com/FAIRMetrics/Metrics">help develop FAIR metrics</a> for EOSC.</p>
<h2 id="opendreamkit-and-the-eosc">OpenDreamKit and the EOSC</h2>
<p>It seems that no e-infrastructure or service presented at the events specifically targets math-based research and teaching, so there could be some room for components of the OpendreamKit VRE.</p>
<h3 id="partnership-with-egi">Partnership with EGI</h3>
<p>Indeed there is an <strong>existing collaboration between EGI and OpenDreamKit</strong> for the deployment of JupyterHub in EGI services. This collaboration will become official in the next weeks with the signature of a Memorandum of Understanding between the two parties.
Depending on the success of this joint work and of the need of the EOSC post-2020, the collaboration could be extended.</p>
<h3 id="lobby">Lobby</h3>
<p>OpenDreamKit as a consortium promoting Open Source software and Open Data in the name of large communities can take lobbying actions to have an impact on the shape of the future EOSC. The following actions can and will be taken:</p>
<ul>
<li><strong>Become a stakeholder of the <a href="https://ec.europa.eu/research/openscience/index.cfm?pg=open-science-cloud">EOSC</a>:</strong></li>
</ul>
<p>1) Endorse the principles of the <a href="https://ec.europa.eu/research/openscience/pdf/eosc_declaration.pdf#view=fit&pagemode=none">EOSC declaration</a> by sending an official statement by mail</p>
<p>2) Commit to take some of the specific actions forward</p>
<p>Endorsement and commitment must be sent at <a href="mailto:RTD-EOSC@EC.EUROPA.EU">RTD-EOSC@EC.EUROPA.EU</a>.</p>
<ul>
<li><strong>Contact experts implied in the EOSC:</strong>
<ul>
<li>
<p>From External Board of EOSC working for the Commission: several experts including Jean-François ABRAMATIC (INRIA) and the chairperson John Womersley (European Spallation Source)</p>
</li>
<li>
<p>From the External Advisory Board of the EOSC-pilot project (launched in preparation of EOSC-Hub project): the list is available <a href="https://eoscpilot.eu/about/external-advisory-board">on their website</a>. One expert, Françoise GENOVA, is also member of the OpenDreamKit Advisory Board.</p>
</li>
</ul>
</li>
<li>The European Commission <strong>Project Officer for OpenDreamKit is also deeply implied in the talks for the EOSC implementation</strong>, and the Coordinator will be in close relation with her in the next months and years.</li>
</ul>
<p><em>Reference: Hans fangohr’s <a href="https://fangohr.github.io/blog/towards-the-european-open-science-cloud.html">blogpost</a> on his group page</em></p>
LinBox Days December 2017 Camaret sur Aigue (FR), 2017-12-13 to 2017-12-152017-12-01T00:00:00+00:00http://opendreamkit.org/2017/12/01/linbox<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: C. Pernet, J.-G. Dumas, H. Zhu.</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and travel expenses), for about 2.1k Euro.</p>
<h2 id="event-summary">Event summary</h2>
<p>The Camaret LinBox developer meeting took place in Grignan (France), from December
13th to 15th, 2017.</p>
<p>There were 9 participants from 3 different institutions (UGA, Université
Montpellier 2, US Naval Academy).
No registration fees were applied.</p>
<p>After a round table branstorming on development projects for the meeting,
developers gathered in groups each addressing a given project.
The main achievements of the meeting include</p>
<ul>
<li>Progress on the distributed parallel ration solver (\delivref{hpc}{LinBox-distributed});</li>
<li>Improvement on the polynomial matrix algorithmic;</li>
<li>wider support of SIMD micro-architectures;</li>
<li>Cleanup and simplification of the build system;</li>
<li>Improvement of code robustness and release preparation (\delivref{hpc}{LinBox-algo}).</li>
</ul>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop was very productive and particularly beneficial to WP5
(\delivref{hpc}{LinBox-algo} and \delivref{hpc}{LinBox-distributed}).</p>
Diversity and Inclusion in Scientific Computing New York, 29th and 30th November 20182017-11-29T00:00:00+00:00http://opendreamkit.org/2017/11/29/sheff-DISCU<h2 id="main-goals">Main goals</h2>
<p>NumFOCUS’s Diversity \& Inclusion in Scientific Computing (``DISC’’) Program strives to help create a more diverse community through initiatives and programming devoted to increasing participation by and inclusion of underrepresented people.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit Sheffield member, Tania Allard was invited to participate in this 2 day event.</p>
<h2 id="event-summary">Event summary</h2>
<p>At the inaugural Diversity and Inclusion in Scientific Computing (DISC) Unconference in November 2017 in New York, one of the topics that came up was organizing inclusive and diverse events and conferences. How to organize such an event? Is there a checklist? Is it hard? Participants decided to build on work begun by the NumFOCUS DISC Committee creating a cookbook for organizing inclusive and diverse events, with the aim of encouraging and supporting such events.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The output and its creation are described at https://numfocus.org/blog/discover-cookbook</p>
Subgroups and lattices of Lie groups2017-11-15T00:00:00+00:00http://opendreamkit.org/2017/11/15/lattices-in-Lie-groups<p><a href="http://opendreamkit.org">OpenDreamKit</a> is hosting a workshop on “Subgroups and lattices of Lie groups”
to take place at the <a href="http://faberresidency.cat/">Faber residency</a> in Olot, Spain from monday 19th
of February to Saturday 3rd of March 2018. The aim is to bring together experts in the geometry, algebra
and combinatorics together with software developers in order to improve algorithms and
functionalities of open source packages concerning Lie groups and their subgroups.</p>
<p>The organization page of the event is <a href="https://wiki.sagemath.org/days93">https://wiki.sagemath.org/days93</a>.</p>
Bioinformatics Awareness Days University of Sheffield, November 20172017-11-06T00:00:00+00:00http://opendreamkit.org/2017/11/06/sheff-BAD<h2 id="main-goals">Main goals</h2>
<p>The ecosystem of tools supported by the OpenDreamKit project are suitable for a wide range of scientific disciplines. The development of this workshop, in collaboration with a bioinformatics practitioner, was an experiment in how to apply OpenDreamKit supported technologies to the field of Bioinformatics.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Workshop materials were developed by Tania Allard in collaboration with Statistician and Bioinformatics researcher, Dr Luisa Cutillo. The materials were released online at https://github.com/trallard/BAD_days. The workshop was delivered by Dr Cutillo with support from Tania Allard.</p>
<h2 id="event-summary">Event summary</h2>
<p>A group of researchers based within the Sheffield Institute for Translational Neuroscience (SITRAN, http://sitran.org/) were taught about Bioinformatics workflows using Jupyter notebooks with computation provided by the free Micosost Azure Notebook service.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The event demonstrated that OpenDreamKit supported technologies could be applied to the field of Bioinformatics and led to a new collaboration between Dr Cutillo and OpenDreamKit member Mike Croucher.</p>
<p>Following the success of this workshop, Dr Cutillo independently taught an introductory workshop on statistics using Jupyter notebooks on Azure at Parthenope University of Naples (Materials at https://github.com/luisacutillo78/RbasicStats)</p>
<p>Dr Cutillo has since moved to University of Leeds where she will be teaching statistics to 200+ undergraduates. She plans to use OpenDreamKit developed technologies in collaboration with the Research Software Engineering group at Leeds.</p>
<p>The event required the development of a website that was linked to the Jupyter notebooks (https://bitsandchips.me/BAD_days/}). The website caught the attention of Eleni Vasilaki, Head of Machine Learning at University of Sheffield who wanted to do something similar for her course on Adaptive Intelligence. We supported her in this endeavour and the result is at (\url{http://bitsandchips.me/COM3240_Adaptive_Intelligence/).</p>
<p>In order to better support this, OpenDreamKit member Tania Allard, developed a Jekyll template for use by academics and researchers using Jupyter notebooks for course materials and dissemination. Such a template allows the creation of Jupyter notebooks based websites using Jekyll, which is the default static website framework supported by GitHub. It also allows for easy display of notebooks connected to cloud computing resources such as Microsoft Azure Notebooks (D2.17, T2.6).</p>
<p>This led to the development of a Python package: nbjekyll (https://github.com/trallard/nbjekyll) that complements the Jekyll template. This package converts Jupyter notebooks into .md files that can be readily usable by Jekyll (this uses nbformat for the conversion). It also uses OpenDreamKit-developed nbval to perform notebook validation and add custom headers indicating the last update date, version and test status of the notebook.</p>
<p>As well as being used internally at Sheffield, The nbjekyll package received some attention on twitter https://twitter.com/jdblischak/status/1009800776305332224} and \url{https://twitter.com/walkingrandomly/status/1009414151716909057 receiving a total of 42 retweets and 80 ‘likes’</p>
MMM 2017 - JOOMMF drop-in sessions Pittsburgh, PA, USA 06-10 November 20172017-11-06T00:00:00+00:00http://opendreamkit.org/2017/11/06/MMM2017<h2 id="main-goals">Main goals</h2>
<p>In two sessions during the conference, users had a chance to talk to us, give us feedback, and request features.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>JOOMMF was developed as a part of the OpenDreamKit project and one participant from the OpenDreamKit was present to deliver the workshop (Marijan Beg). The workshop was fully funded by the OpenDreamKit and total costs were 2048.67 euros.</p>
<h2 id="event-summary">Event summary</h2>
<p>In two drop-in sessions, we helped users with installation, looked at their simulation requirements, and received some feedback and feature requests.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>During the workshop we received the feedback from the participants about our Python interface to OOMMF.</p>
Publishing reproducible logbooks2017-11-02T00:00:00+00:00http://opendreamkit.org/2017/11/02/use-case-publishing-reproducible-notebooks<h2 id="scenario">Scenario</h2>
<p>Jane has written a (math) paper based on her experiments. She would
like anyone to be able to reproduce, check, and improve her calculations.</p>
<h2 id="suggestion-of-solution">Suggestion of solution</h2>
<p><a href="/public/images/use-cases/reproducible_logbook.png"> <img src="/public/images/use-cases/reproducible_logbook.png" alt="picture of the suggested solution" /></a></p>
<ol>
<li>
<p>She describes the experiments as <a href="http://jupyter.org">Jupyter notebooks</a>,
mixing prose, code, visualization, together with resources: source
code, data, media (think of them as logbooks);</p>
</li>
<li>
<p>She publishes them on a publicly hosted repository
(e.g. on <a href="https://github.com">GitHub</a>, …);</p>
</li>
<li>
<p>She makes that repository <a href="/tag/binder">Binder-ready</a> by describing the
software required to run the notebooks; for details, see the
<a href="https://mybinder.readthedocs.io/en/latest/using.html#preparing-a-repository-for-binder">Binder documentation</a>,
and check the configuration of the examples below.</p>
</li>
</ol>
<h2 id="some-instances">Some instances</h2>
<ul>
<li>
<p><a href="https://github.com/defeo/ffisom">Computing isomorphisms and embeddings of finite fields</a>,
Ludovic Brieulle, Luca De Feo, Javad Doliskani, Jean-Pierre Flori and Éric Schost,
<a href="https://arxiv.org/abs/1705.01221">arXiv:1705.01221 [cs.SC]</a>
<a href="https://mybinder.org/v2/gh/defeo/ffisom/master?filepath=notebooks"><img src="https://mybinder.org/badge_logo.svg" alt="launch binder" /></a><br />
<a href="/public/images/use-cases/reproducible_logbook_example.png"> <img src="/public/images/use-cases/reproducible_logbook_example.png" alt="a binder logbook screenshot" class="small-screenshot" /> </a></p>
</li>
<li>
<p><a href="https://github.com/hivert/Jupyter-Notebooks">The 0-Rook monoid and its representation theory</a>,
Hivert and Gay
<a href="https://mybinder.org/v2/gh/hivert/Jupyter-Notebooks/master?filepath=rook-0.ipynb"><img src="https://mybinder.org/badge_logo.svg" alt="launch binder" /></a></p>
</li>
<li>
<p><a href="https://github.com/fdslrm/EBLUP-NE">Estimating variances in time series linear regression models using
empirical BLUPs and convex
optimization</a>, Martina Hančová,
Gabriela Vozáriková, Andrej Gajdoš and Jozef Hanč,
<a href="https://arxiv.org/abs/1905.07771">arXiv:1905.07771</a></p>
<ul>
<li><a href="https://mybinder.org/v2/gh/fdslrm/Binder-CVXPY/master?filepath=index.ipynb"><code class="language-plaintext highlighter-rouge">Python based notebooks using CVXPY and Scipy</code></a> - <a href="https://mybinder.org/v2/gh/fdslrm/Binder-CVXPY/master?filepath=index.ipynb"><img src="https://mybinder.org/badge_logo.svg" alt="Binder" /></a></li>
<li><a href="https://mybinder.org/v2/gh/fdslrm/Binder-Sage/master?filepath=index.ipynb"><code class="language-plaintext highlighter-rouge">Python based notebooks using SageMath</code></a> - <a href="https://mybinder.org/v2/gh/fdslrm/Binder-Sage/master?filepath=index.ipynb"><img src="https://mybinder.org/badge_logo.svg" alt="Binder" /></a></li>
<li><a href="https://mybinder.org/v2/gh/fdslrm/Binder-R/master?filepath=index.ipynb"><code class="language-plaintext highlighter-rouge">R based notebooks</code></a> - <a href="https://mybinder.org/v2/gh/fdslrm/Binder-R/master?filepath=index.ipynb"><img src="https://mybinder.org/badge_logo.svg" alt="Binder" /></a></li>
</ul>
</li>
</ul>
<h2 id="discussion">Discussion</h2>
<p>By publishing the log books and resources on a publicly hosted
repository, Jane also guarantees their long term archival thanks
to the <a href="http://softwareheritage.org/">Software Heritage</a> project.</p>
<p>The proposed solution takes care of many of the basic hurdles for
reproducibility, especially if following the recommended best
practices (like pinpointing the versions of the dependencies). Full
reproducibility however is intrinsically hard and many aspects are not
tackled, like numerical instability, long term availability of
software or long term backward compatibility of software and hardware.
Also, only relatively lightweight calculations are covered.
Nevertheless this hopefully covers the 20% of Pareto’s principle.</p>
<!-- Bonus: make the paper itself active !-->
<!-- TODO: explore using e.g. [latexml](https://dlmf.nist.gov/LaTeXML/)+[thebe](https://github.com/minrk/thebelab)?.!-->
<!-- TODO: publicity, indexing, referencement !-->
<!-- If executing the examples requires a non-trivial install/build step,
also consider
[using a `Dockerfile`](http://mybinder.readthedocs.io/en/latest/dockerfile.html),
and auto-building the Docker image on <https://hub.docker.com/>. !-->
<h2 id="to-do">To do</h2>
<ul>
<li>Estimate the number of such instances;</li>
<li>Provide a template.</li>
</ul>
<h2 id="time-and-expertise-required">Time and expertise required</h2>
<p>Assuming Jane is familiar with version control and Jupyter (basic lab
skills taught at <a href="http://software-carpentry.org/">Software Carpentry</a>,
that the experiments were prepared as notebooks, and the software
required is packaged (conda, debian, docker container, …), the
publishing part could take two hours the first time, and half an hour
later on.</p>
<h2 id="whats-new-since-opendreamkit-started">What’s new since OpenDreamKit started</h2>
<ul>
<li>Apparition of <a href="http://mybinder.org">Binder</a>;</li>
<li>Expansion of the <a href="http://jupyter.org">Jupyter</a> technology;</li>
<li>Better packaging and interfacing of math software.</li>
</ul>
<h2 id="opendreamkit-contribution">OpenDreamKit contribution</h2>
<ul>
<li>Development and contributions to Jupyter interfaces (kernels) for
math software (GAP, Pari/GP, SageMath, Singular) and C++;
see <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/96">D.47</a>.</li>
<li>Contributions to the packaging of math software (GAP, Pari/GP,
SageMath, Singular, …); see
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/58">D3.1</a>
and
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/59">D3.10</a>;</li>
<li>Early adoption of Binder;</li>
<li>Contributions to the
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/238">deployment of new Binder instances</a>;</li>
<li>Advertising, training, providing a template (TODO!), …</li>
</ul>
<h2 id="credits">Credits</h2>
<p><small></small></p>
<ul><li>Illustration: Juliette Taka & Nicolas M. Thiéry. (2018).<br />
Publishing reproducible logbooks explainer comic strip.<br />
Zenodo. <a href="https://doi.org/10.5281/zenodo.4421040"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.4421040.svg" alt="DOI" /></a>. CC-BY-SA.
</ul>
</small>
</li></ul>
Post doc position opening at Université Paris-Sud2017-11-02T00:00:00+00:00http://opendreamkit.org/2017/11/02/postdoc-position-paris-sud<p>This is an announcement for a postdoc position opening at Université
Paris-Sud, working on the interplay between Data, Knowledge, and
Software in Mathematics, and in particular the exploitation of
mathematical knowledge for increased interoperability across
computational mathematics software and mathematical databases.</p>
<h1 id="time-line">Time line</h1>
<p>Interviews in early december, for a recruitment from early 2018 to
Fall 2019. Since we have a strong candidate for a half time position,
we will also consider candidates interested in a half-time or shorter
duration position.</p>
<h1 id="salary">Salary</h1>
<p>For a full-time month work and depending on the applicant’s past experience,
between 2000€ and 3000€ of monthly “salaire net” (salary after non-wage
abour cost but before income tax).</p>
<p>Equivalently, what this salary represents for is a “salaire brut”
of up to 46200€ yearly (for a full-time position).</p>
<h1 id="location">Location</h1>
<p>The postdoc will work at the
<a href="http://www.lri.fr">Laboratoire de Recherche en Informatique</a> of
<a href="http://www.u-psud.fr">Université Paris Sud</a>, in the
Orsay-Bures-Gif-Saclay campus, 25 km South-West of Paris city centre.</p>
<h1 id="mission-and-activities">Mission and activities</h1>
<p>OpenDreamKit’s Work Package 6 explores the interplay between Data,
Knowledge and Software in Mathematics. In particular, it aims at
exploiting mathematical knowledge for increased interoperability
across computational mathematics software and mathematical databases
(known as the Math-In-The-Middle approach). See e.g. the <a href="http://127.0.0.1:4000/2017/10/15/WP6-Usecase/">recent
publications</a> on that
topic, and Section 3.1.6 ``Workpackage Description’’ of the
<a href="https://github.com/OpenDreamKit/OpenDreamKit/raw/master/Proposal/proposal-www.pdf">OpenDreamKit
Proposal</a>.</p>
<p>A successful candidate will be expected to do significant progress, in
close collaboration with the other OpenDreamKit participants and the
community, on some of the tasks of this Work Package:</p>
<ul>
<li>
<p>D6.8: Currated Math-in-the-Middle Ontology and Alignments for GAP /
Sage / LMFDB</p>
</li>
<li>
<p>T6.5: Knowledge-based code infrastructure</p>
<p>Over the last decades, computational components, and in particular
Axiom, MuPAD, \GAP, or \Sage, have embedded more and more
mathematical knowledge directly inside the code, as a way to better
structure it for expressiveness, flexibility, composability,
documentation, and robustness. In this task we will review the
various approaches taken in these software (e.g. categories and
dynamic class hierarchies) and in proof assistants like Coq
(e.g. static type systems), and compare their respective strength
and weaknesses on concrete case studies. We will also explore
whether paradigms offered by recent programming languages like Julia
or Scala could enable a better implementation. Based on this we will
suggest and experiment with design improvements, and explore
challenges such as the compilation, verification, or
interoperability of such code.</p>
</li>
</ul>
<p>The candidate will be welcome to work on closely related though more
technical tasks:</p>
<ul>
<li>
<p>T4.5: Dynamic documentation and exploration system</p>
<p>Introspection has become a critical tool in interactive computation,
allowing user to explore, on the fly, the properties and
capabilities of the objects under manipulation. This challenge
becomes particularly acute in systems like Sage where large parts of
the class hierarchy is built dynamically, and static documentation
builders like Sphinx cannot anymore render all the available
information.</p>
<p>In this task, we will investigate how to further enhance the user
experience. This will include:</p>
<ul>
<li>
<p>On the fly generation of Javadoc style documentation, through
introspection, allowing e.g. the exploration of the class
hierarchy, available methods, etc.</p>
</li>
<li>
<p>Widgets based on the HTML5 and web component standards to display
graphical views of the results of SPARQL queries, as well as populating data
structures with the results of such queries,</p>
</li>
<li>
<p>D4.16: Exploratory support for semantic-aware interactive widgets
providing views on objects of the underlying computational or
database components. Preliminary steps are demonstrated in the
<a href="http://www.larchenvironment.com/">Larch Environment</a> project (see
demo videos) and
<a href="https://github.com/jbandlow/sage-explorer">sage-explorer</a>. The
ultimate aim would be to automatically generate LMFDB-style
interfaces.</p>
</li>
</ul>
<p>Whenever possible, those features will be implemented generically
for any computation kernel by extending the Jupyter protocol with
introspection and documentation queries.</p>
</li>
<li>
<p>T6.9: Memoisation and production of new data</p>
<p>Many CAS users run large and intensive computations, for which they
want to collect the results while simultaneously working on software
improvements. GAP retains computed attribute values of objects
within a session; Sage currently has a limited <code class="language-plaintext highlighter-rouge">cached_method</code>.
Neither offers storage that is persistent across sessions or
supports publication of the result or sharing within a
collaboration. We will use, extend and contribute back to, an
appropriate established persistent memoisation infrastructure, such
as <code class="language-plaintext highlighter-rouge">python-joblib</code>, <code class="language-plaintext highlighter-rouge">redis-simple-cache</code> or <code class="language-plaintext highlighter-rouge">dogpile.cache</code>, adding
features needed for storage and use of results in mathematical
research. We will design something that is simple to deploy and
configure, and makes it easy to share results in a controlled
manner, but provides enough assurance to enable the user to rely on
the data, give proper credit to the original computation and rerun
the computation if they want to.</p>
</li>
</ul>
<h1 id="skills-and-background-requirements">Skills and background requirements</h1>
<ul>
<li>
<p>Strong experience in the design and practical implementation of
mathematics software: computational mathematics software (e.g.
<a href="http://sagemath.org">SageMath</a>), knowledge management systems, or
proof systems;</p>
</li>
<li>
<p>PhD in mathematics or computer science;</p>
</li>
<li>
<p>Experience in open-source development (collaborative development
tools, interaction with the community, …);</p>
</li>
<li>
<p>Fluency in programming languages such as Scala, Python, Julia, etc
appreciated;</p>
</li>
<li>
<p>Strong communication skills;</p>
</li>
<li>
<p>Fluency in oral and written English; speaking French is not a
prerequisite.</p>
</li>
</ul>
<h1 id="context">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<p>Within this ecosystem, the developer will work primarily on the free
open-source mathematics software system
<a href="http://sagemath.org">Sagemath</a>. Based on the
<a href="http://www.python.org">Python</a> language and many existing open-source
math libraries, SageMath is developed since 10 years by a worldwide
community of 300 researchers, teachers and engineers, and has reached
1.5M lines of code.</p>
<p>The developer will work within one of the largest teams of SageMath
developers, composed essentially of researchers in mathematics and
computer science, at the <a href="http://www.lri.fr/">Laboratoire de Recherche en
Informatique</a> (LRI) and in nearby institutions.
The LRI also hosts a strong team working on proof systems.</p>
<h1 id="applications">Applications</h1>
<p>To apply for this position, please send an e-mail to Nicolas.Thiery at
u-psud.fr before December 1st, with the following documents attached:</p>
<ul>
<li>
<p>cover_letter.pdf: a cover letter, in English (why are you interested in this particular position);</p>
</li>
<li>
<p>CV.pdf: a CV, highlighting among other things your skills and
background and your contributions to open source software;</p>
</li>
<li>
<p>phd_reports.pdf: PhD reports (when applicable);</p>
</li>
<li>
<p>reference letters (each named reference_letter_<contactname>.pdf),
or alternatively reference contact information.</contactname></p>
</li>
</ul>
<p>Applications sent after December 1st will be considered until the
position is filled.</p>
Atelier PARI/GP 2017c Oujda (MO), 2017-11-22 to 2017-11-232017-11-01T00:00:00+00:00http://opendreamkit.org/2017/11/01/Atelier-PARI-GP<h2 id="main-goals">Main goals</h2>
<p>This was a teaching and dissemination meeting, by invitation from the math
institute in Oujda as a satellite event for their ``Journ'ees Alg`ebre,
Th'eorie des Nombres et Application’’.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: B. Allombert and A. Page from Bordeaux.</p>
<p>OpenDreamKit funded travel costs for the two instructors for about 1k Euro. The
organizing Oujda institute of mathematics co-funded the event, paying for
all local expenses.</p>
<h2 id="event-summary">Event summary</h2>
<p>This Atelier PARI/GP took place in Oujda (Morocco) from november 22nd to
23rd, it was followed by a 2-day Moroccan national research conference on
Algebra, Number Theory and their Applications. There were 70 participants
from all over Morocco (no registration fees).</p>
<p>The 2-day Atelier featured a general introduction to PARI/GP and two
specialized courses in the mornings (graduate level):</p>
<ul>
<li>Bill Allombert ``Cryptographie et courbes elliptiques’’,</li>
<li>Aurel Page ``Th'eorie alg'ebrique des nombres’’.</li>
</ul>
<p>Afternoons were devoted to practice sessions. Bill Allombert also gave
a research talk on ``Aspects combinatoires et algorithmiques des fonctions
$L$ d’Artin’’ in the ensuing conference.</p>
<p>Slides for all talks are available at
http://pari.math.u-bordeaux.fr/Events/PARI2017c/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was the first purely instructional PARI/GP event, it allowed us to test
the format and teaching material, thereby paving the way for future such events.</p>
Live Structured Documents Location and date2017-10-23T00:00:00+00:00http://opendreamkit.org/2017/10/23/live-documents<h2 id="main-goals">Main goals</h2>
<p>To facilitate development of live structured documents, using Jupyter infrastructure, such as interactive documentation and publication.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The workshop was hosted at OpenDreamKit site, Simula Research Laboratory in Oslo, Norway. The only cost to OpenDreamKit was travel reimbursements to bring five participants to Oslo, Norway for three days.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event was a workshop gathering OpenDreamKit participants and others interested in interactive documents of various kinds.
Participants discussed available technologies and goals,
and worked together to build tools in this area,
especially collaborating to integrate across projects with different stakeholders.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>One of the main impacts of the workshop
was the creation of a new package, <a href="https://github.com/minrk/thebelab">thebelab</a>,
which builds on Jupyter technology
to add interactive code execution to any webpage,
with execution running on <a href="https://mybinder.org">mybinder.org</a> or a local Jupyter server
(see \longdelivref{UI}{ipython-kernels})..
During the workshop, thebelab was put to use in <a href="https://more-sagemath-tutorials.readthedocs.io/en/latest/">documentation for
Sage</a> and tested with documentation for both Singular and GAP.
Since then thebelab has been steadily gaining adoption by the community.
This serves \WPref{UI} by adding the possibility of interactivity to documentation for projects in the OpenDreamKit community,
and benefiting from kernels for GAP and other OpenDreamKit projects developed in \longtaskref{UI}{ipython-kernels},
showing the value of integrating OpenDreamKit systems into a shared Jupyter ecosystem.</p>
Netmath presentation Kaiserslautern, Oct. 20, 20172017-10-20T00:00:00+00:00http://opendreamkit.org/2017/10/20/Netmath<h2 id="main-goals">Main goals</h2>
<p>Presentation of Sage, Jupyter and OpenDreamKit to the Netmath community.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Viviane from OpenDreamKit did the presentation and the travel cost was covered by OpenDreamKit.</p>
<h2 id="event-summary">Event summary</h2>
<p>The Netmath community is an online network for university mathematic teaching, especially interested in sharing innovative methods and advice. This was their annual gathering and they invited us to present the project and how it can be used for teaching.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>Some OpenDreamKit tools such as Sage and Jupyter can be very useful for teaching. Many OpenDreamKit partners have worked into this direction and directly used some features developed by OpenDreamKit and others for their teaching. This was an occasion to share our knowledge and expertise with another community. We received very good feedback from the Netmath organizers.</p>
Presentation of OpenDreamKit2017-10-19T00:00:00+00:00http://opendreamkit.org/2017/10/19/Netmath<p>Viviane Pons presented the OpenDreamKit project and its impact for teaching to the <a href="https://www.netmath.de/">Netmath community</a></p>
<ul>
<li><a href="https://www.lri.fr/~pons/docs/2017_Kaiserslautern.pdf">Slides</a></li>
</ul>
WP6 Math-in-the-Middle Integration Use Case to be Published at MACIS-2017 (two papers)2017-10-15T00:00:00+00:00http://opendreamkit.org/2017/10/15/WP6-Usecase<p>OpenDreamKit WP6 (Data/Knowledge/Software-Bases) has reported on the first use cases in
two papers to be publised at <a href="https://macis2017.sba-research.org/">MACIS 2017</a>.</p>
<ul>
<li><a href="https://github.com/OpenDreamKit/OpenDreamKit/blob/master/WP6/MACIS17-interop/crc.pdf">Knowledge-Based Interoperability for Mathematical Software Systems</a> (Michael Kohlhase, Luca De Feo, Dennis Müller, Markus Pfeiffer, Florian Rabe, Nicolas M. Thiéry, Victor Vasilyxev, and Tom Wiesing) develops the Math-in-the-Middle Ontology for computational group theory and shows a three-system use case (Sage, GAP, Singular).</li>
<li><a href="https://github.com/OpenDreamKit/OpenDreamKit/blob/master/WP6/MACIS17-vt/crc.pdf">Virtual Theories - A Uniform Interface to Mathematical Knowledge Bases</a> (Tom Wiesing, Michael Kohlhase, and Florian Rabe) extends the MMT theories to “Virtual theories”, which do not have the “load-into-memory-fully” restrictions of “concrete theories” and are therefore suited for very large data sources like the <a href="http://lmfdb.org">LMFDB</a> or <a href="http:/oeis.org">OEIS</a>.</li>
</ul>
Release: SageMath for Windows2017-10-11T00:00:00+00:00http://opendreamkit.org/2017/10/11/SageWindows<h1 id="introduction">Introduction</h1>
<p>One of the main tasks for OpenDreamKit
(<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/50">T.31</a>]) is
improving portability of mathematical software across hardware platforms and
operating systems.</p>
<p>One particular such challenge, which has dogged the SageMath project
practically since its inception, is getting a fully working port of Sage on
Windows (and by extension this would mean working Windows versions of all
the CAS’s and other software Sage depends on, such as GAP, Singular, etc.)</p>
<p>This is particularly challenging, not so much because of the Sage Python
library (which has some, but relatively little system-specific code).
Rather, the challenge is in porting all of Sage’s 150+ standard
dependencies, and ensuring that they integrate well on Windows, with a
passing test suite.</p>
<p>Although UNIX-like systems are popular among open source software developers
and some academics, the desktop and laptop market share of Windows computers
is estimated to be <a href="https://en.wikipedia.org/wiki/Usage_share_of_operating_systems#Desktop_and_laptop_computers">more than
75%</a>
and is an important source of potential users, especially students.</p>
<p>However, for most of its existence, the only way to “install” Sage on
Windows was to run a Linux virtual machine that came pre-installed with
Sage, which is made available on Sage’s downloads page. This is clumsy and
onerous for users–it forces them to work within an unfamiliar OS, and it
can be difficult and confusing to connect files and directories in their
host OS to files and directories inside the VM, and likewise for web-based
applications like the notebook. Because of this Windows users can feel like
second-class citizens in the Sage ecosystem, and this may turn them away
from Sage.</p>
<p>Attempts at Windows support almost as old as Sage itself (initial Sage
release in 2005). Microsoft offered funding to work on Windows version <a href="https://groups.google.com/d/msg/sage-devel/LTWgQ_24cPs/WW25BGe7KjwJ">as
far back as
2007</a>
but was far too little for the amount of effort needed.</p>
<p>Additional work done was done off and on through 2012, and partial support
was possible at times. This included admirable work to try to support
building with the native Windows development toolchain (e.g. MSVC). There
was even at one time an earlier version of a Sage installer for Windows, but
long since abandoned.</p>
<p>However, Sage development (and more importantly Sage’s dependencies)
continued to advance faster than there were resources for the work on
Windows support to keep up, and work mostly stalled after 2013.
OpenDreamKit has provided a unique opportunity to fund the kind of sustained
effort needed for Sage’s Windows support to catch up.</p>
<h1 id="sage-for-windows-overview">Sage for Windows overview</h1>
<p>As of SageMath version 8.0, Sage will be available for 64-bit versions of
Windows 7 and up. It can be downloaded through the <a href="https://www.sagemath.org/download-windows.html">SageMath
website</a>, and up-to-date
installation instructions are being developed at the <a href="http://wiki.sagemath.org/SageWindows">SageMath
wiki</a>. A 32-bit version had been
planned as well, but is on hold due to technical limitations that will be
discussed later.</p>
<p>The installer contains all software and documentation making up the standard
Sage distribution, all libraries needed for Cygwin support, a bash shell,
numerous standard UNIX command-line utilities, and the
<a href="https://mintty.github.io/">Mintty</a> terminal emulator, which is generally
more user-friendly and better suited for Cygwin software than the standard
Windows console.</p>
<p>It is distributed in the form of a single-file executable installer, with a
familiar install wizard interface (built with the venerable
<a href="http://www.jrsoftware.org/isinfo.php">InnoSetup</a>. The installer comes in
at just under a gigabyte, but unpacks to more than 4.5 GB in version 8.0.</p>
<p><img src="/public/images/2017-10-11-SageWindows/installer1.png" alt="Sage for Windows Installer" class="img-responsive" /></p>
<p><img src="/public/images/2017-10-11-SageWindows/installer2.png" alt="Sage for Windows Installer" class="img-responsive" /></p>
<p>Because of the large number of files comprising the complete SageMath
distribution, and the heavy compression of the installer, installation can
take a fair amount of time even on a recent system. On my Intel i7 laptop
it takes about ten minutes, but results will vary. Fortunately, this has
not yet been a source of complaints–beta testers have been content to run
the installer in the background while doing other work–on a modern
multi-core machine the installer itself does not use overly many resources.</p>
<p>If you don’t like it, there’s also a standard uninstall:</p>
<p><img src="/public/images/2017-10-11-SageWindows/uninstall.png" alt="Sage for Windows Uninstaller" class="img-responsive" /></p>
<p>The installer includes three desktop and/or start menu shortcuts:</p>
<p><img src="/public/images/2017-10-11-SageWindows/shortcuts.png" alt="Sage for Windows start menu shortcuts" class="img-responsive" /></p>
<p>The shortcut titled just “SageMath 8.0” launches the standard Sage command
prompt in a text-based console. In general it integrates well enough with
the Windows shell to launch files with the default viewer for those file
types. For example, plots are saved to files and displayed automatically
with the default image viewer registered on the computer.</p>
<p><img src="/public/images/2017-10-11-SageWindows/console.png" alt="Sage for Windows console" class="img-responsive" /></p>
<p>(Because Mintty supports <a href="https://en.wikipedia.org/wiki/Sixel">SIXEL</a> mode
graphics, it may also be possible to embed plots and equations directly in
the console, but this has not been made to work yet with Sage.)</p>
<p>“SageMath Shell” runs a bash shell with the environment set up to
run software in the Sage distribution. More advanced users, or users who
wish to directly use other software included in the Sage distribution (e.g.
GAP, Singular) without going through the Sage interface. Finally,
“SageMath Notebook” starts a Jupyter Notebook server with Sage configured as
the default kernel and, where possible, opens the Notebook interface in the
user’s browser.</p>
<p>In principle this could also be used as a development environment for doing
development of Sage and/or Sage extensions on Windows, but the current
installer is geared primarily just for users.</p>
<h1 id="rationale-for-cygwin-and-possible-alternatives">Rationale for Cygwin and possible alternatives</h1>
<p>There are a few possible routes to supporting Sage on Windows, of which
Cygwin is just one. For example, before restarting work on the Cygwin port
I experimented with a solution that would run Sage on Windows using
<a href="https://www.docker.com/">Docker</a>. I built an installer for Sage that would
install Docker for Windows if it was not already installed, install and
configure a pre-build Sage image for Docker, and install some desktop
shortcuts that attempted to launch Sage in Docker as transparently as
possible to the user. That is, it would ensure that Docker was running,
that a container for the Sage image was running, and then would redirect I/O
to the Docker container.</p>
<p>This approach “worked”, but was still fairly clumsy and error-prone. In
order to make the experience as transparent as possible a fair amount of
automation of Docker was needed. This could get particularly tricky in
cases where the user also uses Docker directly, and accidentally interferes
with the Sage Docker installation. Handling issues like file system and
network port mapping, while possible, was even more complicated. What’s
worse, running Linux images in Docker for Windows still requires
virtualization. On older versions this meant running VirtualBox in the
background, while newer versions require the Hyper-V hypervisor (which is
not available on all versions of Windows–particularly “Home” versions).
Furthermore, this requires hardware-assisted virtualization (HAV) to be
enabled in the user’s BIOS. This typically does not come enabled by default
on home PCs, and users must manually enable it in their BIOS menu. We did
not consider this a reasonable step to ask of users merely to “install
Sage”.</p>
<p>Another approach, which was looked at in the early efforts to port Sage to
Windows, would be to get Sage and all its dependencies building with the
standard Microsoft toolchain (MSVC, etc.). This would mean both porting the
code to work natively on Windows, using the MSVC runtime, as well as
developing build systems compatible with MSVC. There was a time when,
remarkably, many of Sage’s dependencies did meet these requirements. But
since then the number of dependencies has grown too much, and Sage itself
become too dependent on the GNU toolchain, that this would be an almost
impossible undertaking.</p>
<p>A middle ground between MSVC and Cygwin would be to build Sage using the
MinGW toolchain, which is a port of GNU build tools (including binutils,
gcc, make, autoconf, etc.) as well as some other common UNIX tools like the
bash shell to Windows. Unlike Cygwin, MinGW does not provide emulation of
POSIX or Linux system APIs–it just provides a Windows-native port of the
development tools. Many of Sage’s dependencies would still need to be
updated in order to work natively on Windows, but at the very least their
build systems would require relatively little updating–not much more than
is required for Cygwin. This would actually be my preferred approach, and
with enough time and resources it could probably work. However, it would
still require a significant amount of work to port some of Sage’s more
non-trivial dependencies, such as GAP and Singular, to work on Windows
without some POSIX emulation.</p>
<p>So Cygwin is the path of least resistance. Although bugs and shortcomings
in Cygwin itself occasionally require some effort to work around (as a
developer–users should not have to think about it), for the most part it
<em>just works</em> with software written for UNIX-like systems. It also has the
advantage of providing a full UNIX-like shell experience, so shell scripts
and scripts that use UNIX shell tools will work even on Windows. However,
since it works directly on the native filesystem, there is less opportunity
for confusion regarding where files and folders are saved. In fact, Cygwin
supports both Windows-style paths (starting with <code class="language-plaintext highlighter-rouge">C:\\</code>) and UNIX-style
paths (in this case starting with <code class="language-plaintext highlighter-rouge">C:/</code>).</p>
<p>Finally, a note on the Windows Subsystem for Linux (WSL), which debuted
shortly after I began my Cygwin porting efforts, as I often get asked about
this: “Why not ‘just’ use the ‘bash for Windows’?” The WSL is a new effort
by Microsoft to allow running executables built for Linux directly on
Windows, with full support from the Windows kernel for emulation of Linux
system calls (including ones like <code class="language-plaintext highlighter-rouge">fork()</code>). Basically, it aims to provide
all the functionality of Cygwin, but with full support from the kernel, and
the ability to run Linux binaries directly, without having to recompile
them. This is great of course. So the question is asked if Sage can run in
this environment, and experiments suggest that it works pretty well
(although the WSL is still under active development and has room for
improvement).</p>
<p>I wrote more about the WSL in a <a href="http://opendreamkit.org/2016/10/04/emerging-technologies-wsl/">blog
post</a> last
year, which also addresses why we can’t “just” use it for Sage for Windows.
But in short: 1) The WSL is currently only intended as a developer tool:
There’s no way to package Windows software for end users such that it uses
the WSL transparently. And 2) It’s only available on recent updates of
Windows 10–it will never be available on older Windows versions. So to
reach the most users, and provide the most hassle-free user experience, the
WSL is not currently a solution. However, it may still prove useful for
developers as a way to do Sage <em>development</em> on Windows. And in the future
it may be the easiest way to install UNIX-based software on Windows as well,
especially if Microsoft ever expands its scope.</p>
<h1 id="development-challenges">Development challenges</h1>
<p>The main challenge with porting Sage to Windows/Cygwin has relatively little
to do with the Sage library itself, which is written almost entirely in
Python/Cython and involves relatively few system interfaces (a notable
exception to this is the advanced signal handling provided by
<a href="https://github.com/sagemath/cysignals">Cysignals</a>, but this has been found
to work almost flawlessly on Cygwin thanks to the Cygwin developers’ heroic
efforts in emulating POSIX signal handling on Windows). Rather, most of the
effort has gone into build and portability issues with Sage’s more than 150
dependencies.</p>
<p>The majority of issues have been build-related issues. Runtime issues are
less common, as many of Sage’s dependencies are primarily mathematical,
numerical code–mostly CPU-bound algorithms that have little use of
platform-specific APIs. Another reason is that, although there are some
anomalous cases, Cygwin’s emulation of POSIX (and some Linux) interfaces is
good enough that most existing code just works as-is. However, because
applications built in Cygwin are native Windows applications and DLLs, there
are Windows-specific subtleties that come up when building some non-trivial
software. So most of the challenge has been getting all of Sage’s
dependencies building cleanly on Cygwin, and then <em>maintaining</em> that support
(as the maintainers of most of these dependencies are not themselves testing
against Cygwin regularly).</p>
<p>In fact, maintenance was the most difficult aspect of the Cygwin port (and
this is one of the main reasons past efforts failed–without a sustained
effort it was not possible to keep up with the pace of Sage development). I
had a snapshot of Sage that was fully working on Cygwin, with all tests
passing, as soon as the end of summer in 2016. That is, I started with one
version of Sage and added to it all the fixes needed for that version to
work. However, by the time that work was done, there were many new
developments to Sage that I had to redo my work on top of, and there were
many new issues to fix. This cycle repeated itself a number of times.</p>
<h2 id="continuous-integration">Continuous integration</h2>
<p>The critical component that was missing for creating a sustainable Cygwin
port of Sage was a <em>patchbot</em> for Cygwin. The Sage developers maintain a
(volunteer) army of patchbots–computers running a number of different OS
and hardware platforms that perform continuous integration testing of all
proposed software changes to Sage. The patchbots are able, ideally, to
catch changes that break Sage–possibly only on specific platforms–before
they are merged into the main development branch. Without a patchbot
testing changes on Cygwin, there was no way to stop changes from being
merged that broke Cygwin. With some effort I managed to get a Windows VM
with Cygwin running reliably on UPSud’s OpenStack infrastructure, that
could run a Cygwin patchbot for Sage. By continuing to monitor this
patchbot the Sage community can now receive prior warning if/when a change
will break the Cygwin port. I expect this will impact only a small number
of changes–in particular those that update one of Sage’s dependencies.</p>
<p>In so doing we are, indirectly, providing continuous integration on Cygwin
for Sage’s many dependencies–something most of those projects do not have
the resources to do on their own. So this should be considered a service to
the open source software community at large. (I am also planning to
piggyback on the work I did for Sage to provide a Cygwin buildbot for
Python–this will be important moving forward as the official Python source
tree has been broken on Cygwin for some time, but is one of the most
critical dependencies for Sage).</p>
<h2 id="runtime-bugs">Runtime bugs</h2>
<p>All that said, a few of the runtime bugs that come up are non-trivial as
well. One particular source of bugs is subtle synchronization issues in
multi-process code, that arise primarily due to the large overhead of
creating, destroying, and signalling processes on Cygwin, as compared to
most UNIXes. Other problems arise in areas of behavior that are not
specified by the POSIX standard, and assumptions are made that might hold on,
say, Linux, but that do not hold on Cygwin (but that are still
POSIX-compliant!) For example, a difference in (undocumented, in both
cases) memory management between Linux and Cygwin made for a particularly
challenging <a href="https://trac.sagemath.org/ticket/22633">bug in PARI</a>.
Another <a href="https://trac.sagemath.org/ticket/21388">interesting bug</a> came up in
a test that invoked a stack overflow bug in Python, which only came up on
Cygwin due to the smaller default stack size of programs compiled for
Windows. There are also occasional bugs due to small differences in
numerical results, due to the different implementation of the standard C
math routines on Cygwin, versus GNU libc. So one should not come away with
the impression that porting software as complex as Sage and its dependencies
to Cygwin is completely trivial, nor that similar bugs might not arise in
the future.</p>
<h2 id="challenges-with-32-bit-windowscygwin">Challenges with 32-bit Windows/Cygwin</h2>
<p>The original work of porting Sage to Cygwin focused on the 32-bit version of
Cygwin. In fact, at the time that was the only version of Cygwin–the first
release of the 64-bit version of Cygwin was not until 2013. When I picked
up work on this again I focused on 64-bit Cygwin–most software developers
today are working primarily on 64-bit systems, and so from many projects
I’ve worked on the past my experience has been that they have been more
stable on 64-bit systems. I figured this would likely be true for Sage and
its dependencies as well.</p>
<p>In fact, after getting Sage working on 64-bit Cygwin, when it came time to
test on 32-bit Cygwin I hit some significant snags. Without going into too
many technical details, the main problem is that 32-bit Windows applications
have a user address space limited to just 2 GB (or 3 GB with a special boot
flag). This is in fact not enough to fit all of Sage into memory at once.
The good news is that for most cases one would never try to use all of Sage
at once–this is only an issue if one tries to load every library in both
Sage, and all its dependencies, into the same address space. In practical
use this is rare, though this limit can be hit while running the Sage test
suite.</p>
<p>With some care, such as reserving address space for the most likely to be
used (especially simultaneously) libraries in Sage, we can work around this
problem for the average user. But the result may still not be 100% stable.</p>
<p>It becomes a valid question whether it’s worth the effort. There are
unfortunately few publicly available statistics on the current market share
of 64-bit versus 32-bit Windows versions among desktop users. Very few new
desktops and laptops sold anymore to the consumer market include 32-bit
OSes, but it is still not too uncommon to find on some older, lower-end
laptops. In particular, some laptops sold not too long ago with Windows 7
were 32-bit. According to <a href="https://www.netmarketshare.com/operating-system-market-share.aspx?qprid=10&qpcustomd=0">Net Market
Share</a>,
as of writing Windows 7 still makes up nearly 50% of all desktop operating
system installments. This still does not tell us about 32-bit versus
64-bit. The popular (<a href="http://www.gamespot.com/articles/steam-reaches-new-concurrent-user-record/1100-6431895/">12.5 million concurrent
users</a>)
Steam PC gaming platform publishes the results of their <a href="http://store.steampowered.com/hwsurvey/">usage statistics
survey</a>, which as of writing shows
barely over 5% of users with 32-bit versions of Windows. However, computer
gamers are not likely to be representative of the overall market, being more
likely to upgrade their software and hardware.</p>
<p>So until some specific demand for a 32-bit version of SageMath for Windows
is heard, we will not likely invest more effort into it.</p>
<h1 id="conclusion-and-future-work">Conclusion and future work</h1>
<p>Focusing on Cygwin for porting Sage to Windows was definitely the right way
to go. It took me only a few months in the summer of 2016 to get the vast
majority of the work done. The rest was just a question of keeping up with
changes to Sage and fixing more bugs (this required enough constant effort
that it’s no wonder nobody managed to quite do it before). Now, however,
enough issues have been addressed that the Windows version has remained
fairly stable, even in the face of ongoing updates to Sage.</p>
<p>Porting more of Sage’s dependencies to build with MinGW and without Cygwin
might still be a worthwhile effort, as Cygwin adds some overhead in a few
areas, but if we had started with that it would have been too much effort.</p>
<p>In the near future, however, the priority needs to be improvements to user
experience of the Windows Installer. In particular, a better solution is
needed for installing Sage’s optional packages on Windows (preferably
without needing to compile them). And an improved experience for using Sage
in the Jupyter Notebook, such that the Notebook server can run in the
background as a Windows Service, would be nice. This feature would not be
specific to Sage either, and could benefit all users of the Jupyter Notebook
on Windows.</p>
<p>Finally, I need to better document the process of doing Sage development on
Cygwin, including the typical kinds of problems that arise. I also need to
better document how to set up and maintain the Cygwin patchbot, and how to
build releases of the Sage on Windows installer so that its maintenance does
not fall solely on my shoulders.</p>
2017 International Research Software Engineering Conference Manchester MOSI, 7th and 8th September 20172017-09-07T00:00:00+00:00http://opendreamkit.org/2017/09/07/sheff-rseconf<h2 id="main-goals">Main goals</h2>
<p>The second international Research Software Engineering (RSE) conference took place on the 7th and 8th of September 2017 at the Museum of Science and Industry (MOSI). There were over 200 attendees, 40 talks, 15 workshops, 3 keynote talks. This is the largest event in the Research Software Engineering calendar.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit members had significant involvement in this event. Sheffield’s Tania Allard developed and delivered a workshop called ‘Jupyter notebooks for reproducible research’ which demonstrated and taught how technologies developed in OpenDreamKit significantly improved how computational research across all fields could be made more reproducible. The workshop was one of the most popular workshops at the event and had to be delivered twice due to high demand. The only workshop with higher demand was Microsoft’s introduction to Azure Cloud which included several hundred dollars of free cloud time per participant.</p>
<p>Workshop materials are freely available at https://github.com/trallard/JNB_reproducible</p>
<p>Tania also served as Diversity Chair on the RSE conference committee.</p>
<p>Fellow Sheffield OpenDreamKit member, Mike Croucher, gave an invited keynote speech at the event called ‘I, Research Software Engineer’. Slides at https://mikecroucher.github.io/RSE_2017_keynote_presentation/</p>
<h2 id="event-summary">Event summary</h2>
<p>Research software engineers are the people who support computational research across all of academia. Influencing this group has the potential to influence how a large subset of computational research is performed.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The event was incredibly successful for OpenDreamKit since it introduced a significant proportion of the RSE community to OpenDreamKit-developed tools. It also led to invitations to deliver more workshops and talks at other prominent events.</p>
Eurocomb 2017 Vienna, Aug. 28 -- Sept. 1st, 20172017-09-01T00:00:00+00:00http://opendreamkit.org/2017/09/01/EuroComb<h2 id="main-goals">Main goals</h2>
<p>Eurocomb for <strong>European Conference on Combinatorics, Graph Theory and Applications</strong> is one of the main international conferences on combinatorics. In 2017, it was organized in Vienna and gathered around 200 participants. This was a good occasion to present Sage and OpenDreamKit to the community. We organized a small presentation which was included to the conference planing. Around 20 participants attended.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Viviane Pons from OpenDreamKit was present at the conference and funded by OpenDreamKit.</p>
<h2 id="event-summary">Event summary</h2>
<p>We gave a small presentation to introduce Sage and OpenDreamKit to the participants. This included a demo on the CoCalc platform.</p>
<h2 id="demographic">Demographic</h2>
<p>Participants did not register specifically to the Sage presentation, so we do not have any demographic information.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>It is important that these short presentations happen regularly during main scientific events. Indeed, they keep the community up to date with current development. We received good feedback from the participants. It was an occasion for some of them to get introduced to Sage for the first time and to open source math software in general.</p>
Introduction to SageMath2017-08-31T00:00:00+00:00http://opendreamkit.org/2017/08/31/eurocomb<p>Viviane Pons gave a Sage Introduction talk at the internation combinatorics conference <a href="http://www.dmg.tuwien.ac.at/eurocomb2017/">Eurocomb 2017</a></p>
<p><a href="https://www.lri.fr/~pons/docs/2017_Eurocomb_sage.pdf">Slides</a></p>
Workshop on live structured documents2017-08-26T00:00:00+00:00http://opendreamkit.org/2017/08/26/live-documents-workshop<p><a href="http://opendreamkit.org">OpenDreamKit</a> is hosting a workshop on live structured documents
to take place at Simula Research Laboratory in Oslo, Norway
from Monday 16. October to Friday 20. October.</p>
<p>The workshop is dedicated to various aspects of live documents, including:</p>
<ul>
<li>Authoring language and tools (ReST, XML, latex, …),</li>
<li>Converters (Sphinx, pandoc, docbook, …),</li>
<li>Infrastructure for rendering documents,</li>
<li>Infrastructure for interacting with the computing backends (e.g. <a href="https://github.com/oreillymedia/thebe">Thebe</a> based on the Jupyter protocol, <a href="https://github.com/sympy/sympy-live/">sympy live</a>, …),</li>
<li>Online services providing computing backends (e.g. <a href="https://try.jupyter.org">tmpnb</a>, <a href="https://sagecell.sagemath.org/">sagecell</a>, <a href="https://mybinder.org">binder</a>, …),</li>
<li>Integrated collaborative authoring environments (sharelatex, SMC, texmacs, …)</li>
</ul>
<p>Participants can register <a href="https://www.eventbrite.com/e/opendreamkit-workshop-on-live-structured-documents-tickets-37364670736">via eventbrite</a>.</p>
Code First: Girls University of Sheffield, 2017-20182017-08-08T00:00:00+00:00http://opendreamkit.org/2017/08/08/sheff-codefirstgirls<h2 id="main-goals">Main goals</h2>
<p>Code First: Girls https://www.codefirstgirls.org.uk/ is an international organization that works with companies and women to increase the numbers of women in technology.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit Sheffield team member, Tania Allard, worked with Code First:Girls chapters in Sheffield and Manchester over a 12 months period.</p>
<h2 id="event-summary">Event summary</h2>
<p>Several events were run over a 12 month period which eventually provided training to over 130 women. Tania developed materials on ‘Advanced Python’ using Jupyter Notebook which are now freely available at https://github.com/trallard/CodeFirst-Python_material</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>130 women received training and new materials were developed for use in future Code First: Girls events.</p>
OpenDreamKit at Groups St Andrews in Birmingham2017-07-29T00:00:00+00:00http://opendreamkit.org/2017/07/29/GroupsStAndrews<h2 id="opendreamkit-at-groups-st-andrews-in-birmingham">OpenDreamKit at Groups St Andrews in Birmingham</h2>
<p><a href="http://www.groupsstandrews.org/">Groups St Andrews</a> is a conference series with
a conference every four years. This year’s Groups St Andrews will be in
Birmingham, and I will attend, bring
a <a href="/public/groups-st-andrews-2017.pdf">poster</a>, give a contributed talk
about computing in permutation groups, and teach a course on GAP.</p>
<p>This post serves the main purpose of providing a page on the OpenDreamKit website
that can hold all the links that will appear on the poster, and possible further
information.</p>
<h2 id="links">Links</h2>
<ul>
<li>
<p><a href="http://opendreamkit.org">OpenDreamKit</a></p>
</li>
<li>
<p><a href="https://jupyter.org">Jupyter</a></p>
</li>
<li><a href="http://www.gap-system.org">GAP</a>, <a href="https://github.com/gap-system">GAP GitHub</a>, <a href="https://github.com/gap-packages/">GAP Packages on GitHub</a></li>
<li>
<p><a href="https://github.com/gap-packages/jupyter-kernel-gap">GAP Jupyter Kernel</a></p>
</li>
<li><a href="https://sagemath.org">SageMath</a></li>
<li>
<p><a href="https://cocalc.com">CoCalc (formerly SageMathCloud</a>)</p>
</li>
<li><a href="https://openmath.github.io">OpenMath</a></li>
<li><a href="https://mathhub.info">MathHub</a></li>
<li><a href="https://uniformal.github.io">MMT</a></li>
<li><a href="https://mathhub.info/MitM">Maths in the Middle</a></li>
<li>
<p><a href="https://mathhub.info/mh/mmt/graphs/tgview.html">Theory Graphs</a></p>
</li>
<li><a href="https://lmfdb.org">L-Functions and Modular Forms Database</a></li>
</ul>
PASCO 2017 Kaiserslautern, Germany, July 23-24, 20172017-07-27T00:00:00+00:00http://opendreamkit.org/2017/07/27/PASCO<h2 id="main-goals">Main goals</h2>
<p>The 8th International Workshop on Parallel Symbolic Computation (PASCO) is the
latest instance in a series of workshops dedicated to the promotion and
advancement of parallel algorithms and software in all areas of symbolic
mathematical computation. It is a two days event which is a satellite of
ISSAC, one of the main conference in symbolic computation.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Florent Hivert was invited keynote speaker. The cost
for OpenDreamKit was therefore null.</p>
<h2 id="event-summary">Event summary</h2>
<p>Florent Hivert was invited to give a keynote talk presenting his work on WP5
T5.6. The talk was entitled \emph{High Performance Computing Experiments in
Enumerative and Algebraic Combinatorics}. Here is the abstract:</p>
<p>In this talk, I will report on several experiments around large scale
enumerations in enumerative and algebraic combinatorics. In a first part,
I’ll present a small framework implemented in Sagemath allowing to perform
map/reduce like computations on large recursively defined sets. Though it
doesn’t really qualify as HPC, it allowed to efficiently parallelize a dozen
of experiments ranging from Coxeter group and representation theory of monoids
to the combinatorial study of the C3 linearization algorithm used to compute
the method resolution order (MRO) in script language such as Python and Perl.</p>
<p>In a second part, I’ll describe a methodology used to achieve large speedups
in several enumeration problems. Indeed, in many combinatorial structures
(permutations, partitions, monomials, young tableaux), the data can be encoded
as a small sequence of small integers that can often efficiently be handled by
a creative use of vector instructions. Through the challenging example of
numerical monoids, I will then report on how Cilkplus allows for a extremely
fast parallelization of the enumeration. Indeed, we have been able to
enumerate sets with more that $10^15$ elements on a single multicore machine.
This is joint work with Jean Fromentin.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The talk was a chance to disseminate OpenDreamKit work in
a wider audience and to present the result on deliverable D5.1 and the ongoing
progress on the overall work package. The fact that it was an invited keynote
talk witnesses that the community is particularly interested and attentive on
the OpenDreamKit progress on this matters.</p>
CICM2017 --- Conference on Intelligent Computer Mathematics Edinburgh, UK, 17th-21st of July 20172017-07-17T00:00:00+00:00http://opendreamkit.org/2017/07/17/cicm<h2 id="main-goals">Main goals</h2>
<p>Digital and computational solutions are becoming the prevalent means for the generation, communication, processing, storage and curation of mathematical information.
Separate communities have developed to investigate and build computer based systems for computer algebra, automated deduction, and mathematical publishing as well as novel user interfaces.
While all of these systems excel in their own right, their integration can lead to synergies offering significant added value.
The Conference on Intelligent Computer Mathematics (CICM) offers a venue for discussing and developing solutions to the great challenges posed by the integration of these diverse areas.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Florian Rabe (JU) was track chair for Mathematical Knowledge Management.
One workshops was co-organized by Michael Kohlhase (FAU).
Olexandr Konovalov and Markus Pfeiffer (USTAN) attended the 2th OpenMath workshop and gave an invited talk ``SCSCP and OpenMath in OpenDreamKit’’.
There were no costs to OpenDreamKit other than travel costs for OpenDreamKit members.</p>
<h2 id="event-summary">Event summary</h2>
<p>There were 17 formal paper presentations, 7 formal system presentations, 3 invited presentations, and 5 informal presentations.
Also, 2 workshops took place.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The OpenDreamKit participants presented the paper ``Classification of Alignments between Concepts of Formal Mathematical Systems’’ that is integral to the MitM approach developed in OpenDreamKit WP 6.
The authors included external collaborators in order to bundle system integration efforts.</p>
<p>Patrick Ion and Eric Weisstein presented ``The Special Function Concordance’’, a development very closely related to the development of the MitM ontology of OpenDreamKit WP6.</p>
<p>Contact with Nathan Carter (Bentley University) resulted in his year-long stay on sabbatical at the University of St Andrews, started in July 2018.</p>
PyLadies Paris Paris2017-07-07T00:00:00+00:00http://opendreamkit.org/2017/07/07/PyLadies<h2 id="main-goals">Main goals</h2>
<p>The PyLadies organization is an international mentorship group with a focus on helping more women become active participants and leaders in the Python open-source community. PyLadies Paris is the local Paris chapter.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Viviane Pons from OpenDreamKit has been the co-organizer of the PyLadies Paris events for the last two years. The Meetup group has 494 members and has had 57 events.</p>
<h2 id="event-summary">Event summary</h2>
<p>Meetup include networking events for women coders such as <code class="language-plaintext highlighter-rouge">Talking Bar'' and</code>Coding Café’’. They usually gather between 3 and 10 participants. Some events include presentations and / or tutorials on some specific subjects.</p>
<h2 id="demographic">Demographic</h2>
<p>These events are open to women who are interested in python. This include professional python developers, academics, scientist, students, and enthusiast learners.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>We believe such events are very important to support women coders. They allow to build a strong professional network. It is also an occasion to discuss python practices and, in particlar, OpenDreamKit developments in an informal and relaxed atmosphere.</p>
Sphinx documentation of Cython code using "binding=True"2017-06-09T00:00:00+00:00http://opendreamkit.org/2017/06/09/CythonSphinx<p>One of the deliverables (<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/87">D4.13</a>)
of the OpenDreamKit project is refactoring the documentation system of <a href="http://www.sagemath.org/">SageMath</a>.
The SageMath documentation is built using a heavily customized <a href="http://www.sphinx-doc.org/en/stable/">Sphinx</a>.
Many of the customizations are neccessary to support autodoc
(automatically generated documentation from docstrings) for <a href="http://cython.org/">Cython</a> files.</p>
<p>Thanks to some changes I made to Sphinx, autodoc for Cython now works provided that:</p>
<ol>
<li>
<p>You use Sphinx version 1.6 or later.</p>
</li>
<li>
<p>The Cython code is compiled with the <code class="language-plaintext highlighter-rouge">binding=True</code>
<a href="http://cython.readthedocs.io/en/latest/src/reference/compilation.html#compiler-directives">directive</a>.
See <a href="http://cython.readthedocs.io/en/latest/src/reference/compilation.html#how-to-set-directives">How to set directives</a>
in the Cython documentation.</p>
</li>
<li>
<p>A small monkey-patch is applied to <code class="language-plaintext highlighter-rouge">inspect.isfunction</code>.
You can put this in your Sphinx <code class="language-plaintext highlighter-rouge">conf.py</code> for example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> def isfunction(obj):
return hasattr(type(obj), "__code__")
import inspect
inspect.isfunction = isfunction
</code></pre></div> </div>
</li>
</ol>
<p>This was used successfully for the documentation of
<a href="http://cysignals.readthedocs.io/en/latest/">cysignals</a>
and <a href="http://fpylll.readthedocs.io/en/latest/">fpylll</a>.
There is <a href="https://trac.sagemath.org/ticket/22747">ongoing work</a> to do the same for SageMath.</p>
<h2 id="implementation-of-functions-in-python">Implementation of functions in Python</h2>
<p>To understand why items 2 and 3 on the above list are needed,
we need to look at how Python implements functions.
In Python, there are two kinds of functions
(we really mean functions here, not methods or other callables):</p>
<ol>
<li>
<p>User-defined functions, defined with <code class="language-plaintext highlighter-rouge">def</code> or <code class="language-plaintext highlighter-rouge">lambda</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> >>> def foo(): pass
>>> type(foo)
<class 'function'>
>>> type(lambda x: x)
<class 'function'>
</code></pre></div> </div>
</li>
<li>
<p>Built-in functions such as <code class="language-plaintext highlighter-rouge">len</code>, <code class="language-plaintext highlighter-rouge">repr</code> or <code class="language-plaintext highlighter-rouge">isinstance</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> >>> type(len)
<class 'builtin_function_or_method'>
</code></pre></div> </div>
</li>
</ol>
<p>In the CPython implementation, these are completely independent classes
with different behaviours.</p>
<h2 id="user-defined-functions-binding-as-methods">User-defined functions binding as methods</h2>
<p>Just to give one example, built-in functions do not have a
<code class="language-plaintext highlighter-rouge">__get__</code> method, which means that they do not become methods when used in a class.</p>
<p>Let’s consider this class:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class X(object):
def printme(self):
return repr(self)
</code></pre></div></div>
<p>This is essentially equivalent to</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class X(object):
printme = (lambda self: repr(self))
>>> X().printme()
'<__main__.X object at 0x7fb342f960b8>'
</code></pre></div></div>
<p>However, directly putting the built-in function <code class="language-plaintext highlighter-rouge">repr</code> in the class
does not work as expected:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class Y(object):
printme = repr
>>> Y().printme()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: repr() takes exactly one argument (0 given)
</code></pre></div></div>
<p>This is simply something that built-in functions do not support.</p>
<h2 id="user-defined-vs-built-in-functions">User-defined vs. built-in functions</h2>
<p>Here is a list of the main differences between user-defined and built-in functions:</p>
<ul>
<li>
<p>User-defined functions are implemented in Python,
built-in functions are implemented in C.</p>
</li>
<li>
<p>Only user-defined functions support <code class="language-plaintext highlighter-rouge">__get__</code> and can become methods
(see above).</p>
</li>
<li>
<p>Only user-defined functions support introspection such as
<code class="language-plaintext highlighter-rouge">inspect.getargspec()</code> and <code class="language-plaintext highlighter-rouge">inspect.getsourcefile()</code>.</p>
</li>
<li>
<p>CPython has specific optimizations for calling built-in functions.</p>
</li>
<li>
<p>The <code class="language-plaintext highlighter-rouge">inspect</code> module and profiling make
a difference between the two kinds of functions.</p>
</li>
</ul>
<p>Cython generates C code, so Cython functions must be built-in functions.
This has unfortunate disadvantages, such as the lack of introspection support,
which is particularly important for Sphinx.</p>
<h2 id="the-cython-function-type-cyfunction">The Cython function type: cyfunction</h2>
<p>Luckily, the Cython developers came up with a solution:
they invented a completely new function type (called <code class="language-plaintext highlighter-rouge">cyfunction</code>),
which is implemented like built-in functions
but which behaves as much as possible like user-defined functions.</p>
<p>By default, functions in Cython are built-in functions.
With the directive <code class="language-plaintext highlighter-rouge">binding=True</code>, functions in Cython become cyfunctions.
Since cyfunctions are not specifically optimized by CPython,
this comes with a performance penalty.
More precisely, calling cyfunctions from Python is slower than
calling built-in functions from Python.
The slowdown can be significant for simple functions.
Within Cython, cyfunctions are as fast as built-in functions.</p>
<p>Since a <code class="language-plaintext highlighter-rouge">cyfunction</code> is not a built-in function
nor a user-defined function (those two types are not subclassable),
the <code class="language-plaintext highlighter-rouge">inspect</code> module (and hence Sphinx) does not recognize it as being a function.
So, to have full inspect support for Cython functions,
we need to change <code class="language-plaintext highlighter-rouge">inspect.isfunction</code>.
After various attempts, I came up with <code class="language-plaintext highlighter-rouge">hasattr(type(obj), "__code__")</code>
to test whether the object <code class="language-plaintext highlighter-rouge">obj</code> is a function (for introspection purposes).
This will match user-defined functions and cyfunctions
but not built-in functions, nor any other Python type that I know of.</p>
<h2 id="the-future-a-pep-to-change-the-function-types">The future: a PEP to change the function types?</h2>
<p>I have some vague plans for a Python Enhancement Proposal (PEP)
to change the implementation of the Python function types.
The goal is that Cython functions can be implemented
on top of some standard Python function type,
with all features that cyfunctions currently have,
the performance of built-in functions
and introspection support of user-defined functions.</p>
<p>At this point, it is too early to say anything about the implementation
of this hypothetical future Python function type.
If anything happens, I will surely post an update.</p>
Atelier PARI/GP 2017b Clermont-Ferrand (FR), 2017-06-19 to 2017-06-232017-06-01T00:00:00+00:00http://opendreamkit.org/2017/06/01/PARI-GP<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: B. Allombert, K. Belabas, J.-P. Flori</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and travel expenses), for about 12k Euro. The following
institutions co-funded the event through Clermont institute of mathematics
(Laboratoire
Blaise Pascal): GDR Structuration de la Théorie des Nombres (CNRS), Région
Auvergne Rhône-Alpes, Conseil départemental Puy-de-Dôme, Université
Auvergne-Rhônes-Alpes and the city of Clermont-Ferrand.</p>
<h2 id="event-summary">Event summary</h2>
<p>The 8th Atelier PARI/GP took place in Clermont-Ferrand (France) from june
19th to 23rd with a special focus on Elliptic curves, Modular Forms and
$L$-Functions.</p>
<p>There were 36 registered participants from 16 different institutions
(no registration fees).</p>
<p>A typical day of the workshop had introductory talks and tutorials
in the morning; afternoons allowed ample time for hacking sessions,
discussions and training.</p>
<p>The Atelier featured 13 morning talks.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a dissemination event with a special focus on recently developped
packages (Elliptic curves over number fields, Modular forms), before they were
merged into a formal release. We got lots of feedback on the interface and
documentation and a number of bugs and glitches were fixed as a result.</p>
Release: nbdime 0.3.02017-05-05T00:00:00+00:00http://opendreamkit.org/2017/05/05/nbdime-0.3.0<p>We are happy to announce release 0.3.0 of <a href="https://nbdime.readthedocs.io">nbdime</a>,
continuing to improve the process of working with Jupyter notebooks in version control.</p>
<p>The highlight of 0.3 is much improved integration with git, making it easier than ever
to get started with nbdime in git:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install --upgrade nbdime # install nbdime
nbdime config-git --global --enable # tell git to use nbdime when it sees notebooks
</code></pre></div></div>
<p>and you can get nice GUI diffs directly from git refs on the command-line:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nbdiff-web master mynotebook.ipynb
</code></pre></div></div>
Sage Days 89 University of science and technology Houari Boumediene, Algiers, Algeria, 2017-05-02 to 2017-05-072017-05-01T00:00:00+00:00http://opendreamkit.org/2017/05/01/sage-days-89<h2 id="main-goals">Main goals</h2>
<p>The first goal was to give some training on the SageMath system in order to implement new features and/or to fix some bugs in SageMath. The second goal was to give some courses on Polya theory.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit financed the exhibitor’s travel and accommodation.</p>
<h2 id="event-summary">Event summary</h2>
<p>The Polya class was held in the morning from 9h to 13h every day for a week
for 10-15 students (Master 2 and PhD).
The course on Sage took place in the afternoon and evening from 15h to 21h-22h. (small committee of 4-5 participants).</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>Lessons on Polya and Sage system was correctly finished. The sage lesson was done by fixing a bug and producing the ticket 22979 on the trac server (http://trac.sagemath.org/ticket/22979).</p>
Debriefing from a successful Formal Project Review by the EU Commission2017-04-26T00:00:00+00:00http://opendreamkit.org/2017/04/26/first-project-review-debriefing<p>On April 26th, OpenDreamKit underwent its first formal review by the
European Commission. We presented the achievements of the first 18
months of the project, including
<a href="https://github.com/OpenDreamKit/OpenDreamKit/issues?q=label%3AReportingPeriod1">30 deliverables</a>
(<a href="/events_activities/reports/">reports</a>, <a href="/meetings/2017-04-26-ProjectReviewPresentations/">slides</a>).
Overall, <strong>the feedback was very positive</strong>, with language such as
“enthusiast”, “brilliant”, “amazing job”, or “things have come along
fantastically”. We made a strong point in our reports and
presentations that a vast majority of what’s happening comes from the
ecosystem we support. All we do is exploit the special resources the
EU is entrusting us to knock down some tough hurdles that are
preventing the ball to roll. <strong>Kudos to our communities!</strong></p>
<h2 id="debriefing-notes-i-sent-to-the-opendreamkit-participants">Debriefing notes I sent to the OpenDreamKit participants</h2>
<p>About twenty of us were in Brussels early this week for the
OpenDreamKit Month 18 formal review. After two days of intensive
preparation, we presented our work on Wednesday to our project officer
and reviewers.</p>
<p>There are a few points that we need to think about (not unexpected).
But otherwise the hard work we all put since the beginning of the
project came out as quite a show. The panel gave very constructive
feedback and were overall really happy. They appreciate our approach,
our work, our spirit.</p>
<p>Now is the time to enjoy that appreciation and build on that energy to
do even better in the coming years. Pass this on to our communities!</p>
<p>Speaking of funding: the reviewers made a strong point that we bear a
big responsibility: apparently mathematics does not have a good press
in the high spheres these days. We were <em>very lucky</em>, as a math
project, to be funded; it’s really because they appreciated so much
the strength of the proposal and our “clever and creative
interpretation of the call” that we made it through. “No other math
project is being funded” (this quote obviously does not apply to
ERCs; the scope is plausibly that of H2020 projects).</p>
<p>They now need strong ammunition to make sure that future calls leave
room for mathematics. So not only do we have to succeed because we
care so much about our aims (and should investigate followups to
pursue them further), but also for the sake of other projects
elsewhere in mathematics. We also need to proactively explain and
highlight to a wider audience what we do in collaboration with our
communities. There is very good stuff going on, let it be seen.</p>
<p>Some further thoughts needs to be put in how to achieve that. For now,
the take home message is simple: <strong><em>If you witness something nice
happening, from a technical achievement with a wow factor to a thought
provoking anecdote, write a blog post about it</em></strong>. See the
<a href="https://github.com/OpenDreamKit/OpenDreamKit.github.io/blob/master/_posts/README.md">instructions</a>,
or even just send a brief draft text by e-mail to
<a href="mailto:m.croucher@sheffield.ac.uk">Mike Croucher</a> with
<a href="mailto:Nicolas.Thiery@u-psud.fr">me</a> in CC.</p>
<p>Let me conclude by thanking the whole band that came to Brussels (with
a special nod to the presenters on which we dumped the most delicate
presentations). I was frustrated as you all were spending all this
time together without tackling what we all really care most. However
we built on our image in the Commission, and used the occasion to
strengthen our group around a joint vision. This is a worthwhile long
term investment.</p>
<p>Thank you everybody for all the enthusiastic, dedicated and beautiful
work. It’s an honor and a pleasure to be working with such a team.</p>
<p>Remember: pass it on to those supporting you and to your communities.</p>
<p>Cheers,
Nicolas</p>
Formal Project Review for OpenDreamKit's first reporting period (Sept. 2015 to Feb. 2017)2017-04-24T00:00:00+00:00http://opendreamkit.org/2017/04/24/first-project-review<p>At the occasion of its first
(<a href="/2017/04/26/first-project-review-debriefing/">very successful</a>!)
<a href="/meetings/2017-04-26-ProjectReviewPresentations/">formal review</a>
by the EU commission, twenty OpenDreamKit participants met on the last
week of April 2017 at the <a href="http://www.clora.eu/en/accueil">CLORA</a>
(Club of associated research organisations) headquarters in Brussels.</p>
<h2 id="monday-24-tuesday-25">Monday 24, Tuesday 25</h2>
<ul>
<li>Brief Steering Committee (<a href="http://opendreamkit.org/2017/04/24/brussels-steering-committee/">Agenda & Minutes</a> and <a href="https://mensuel.framapad.org/p/vQTnSyplRr">Pad</a>)</li>
<li><a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/222">Preparation for the review</a></li>
</ul>
<h2 id="wednesday-26">Wednesday 26</h2>
<ul>
<li><a href="/meetings/2017-04-26-ProjectReviewPresentations/">Formal review</a> (<a href="/2017/04/26/first-project-review-debriefing/">debriefing notes</a>)</li>
</ul>
<h2 id="thursday-27-friday-28">Thursday 27, Friday 28</h2>
<ul>
<li><a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/223">Coding sprints</a></li>
</ul>
Steering Committee meeting before the 1st Project Review2017-04-24T00:00:00+00:00http://opendreamkit.org/2017/04/24/brussels-steering-committee<h2 id="participants">Participants</h2>
<p>A framadate <a href="https://framadate.org/MPxYiQTu1JZjHuJp">poll</a> was created</p>
<h3 id="were-present-on-the-steering-committee-day">Were present on the steering committee day:</h3>
<p>Nicolas THIERY; Benoît PILORGET; Erik BRAY; Viviane PONS; Vincent DELECROIX; Michael KOHLHASE; Dennis MUELLER; Florian RABE; Tom WIESING; Clément PERNET; Wolfram DECKER; William HART; Dmitrii PASECHNIK; Marcin KOSTUR; Mike CROUCHER; Hans FANGOHR; Olexandr KONOVALOV; Stephen LINTON; Luca DE FEO; John CREMONA; Paul-Olivier DEHAYE; Benjamin RAGAN-KELLEY; Jeroen DEMEYER; Konrad HINSEN</p>
<h2 id="agenda">Agenda</h2>
<ul>
<li>Second amendment to the Grant</li>
<li>WP7: overview of the situation and of the measures taken and to be planned</li>
<li>Overview of the deliverables due for M18</li>
<li>Best practice for the Project Review</li>
</ul>
<h2 id="minutes">Minutes</h2>
<h3 id="second-amendment-to-the-grant">Second amendment to the Grant</h3>
<p>Benoît Pilorget (BP) announced that the Second Amendment should be over in April-May. All the modifications concerning deliverables and the scientific context were accepted. The remaining blocking points were purely administrative and require some time. At the moment these notes are being written (19/05/2017), the Commission seems to have fully agreed on all terms and is about to sign the amendment.</p>
<p>Related to this amendment, the consortium expressed their congratulations to Hans Fangohr (HF) for his new position at XFEL, in Hamburg.
All points of the amendment can be found on the github issue <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/193">#193</a></p>
<h3 id="wp7-overview-of-the-situation-and-of-the-measures-taken-and-to-be-planned">WP7: overview of the situation and of the measures taken and to be planned:</h3>
<p>An open brainstorming session took place.
Resulting from the retirement of Ursula Martin, it appears that some aspects of WP7 that require research-grade expertise in sociology will be hardly achievable as it is organised today with the current consortium. Therefore solutions must be found so that we don’t just tick the boxes but actually deliver high quality material.</p>
<p>Several options have been discussed:</p>
<p>1) Hire new staff specialised in sociology or likewise field. This solution would probably lead to the transfer of some funding within the consortium, unless it turns out enough Person-Months are planned at UOXF</p>
<p>2) Subcontract the planned work not feasible. For this solution to work out, one must find an adequate subcontractor (providing enough funds are available within UOXF or the consortium) and sign an amendment to the Grant with the Commission</p>
<p>3) Rethink the scientific content (objectives, tasks, deliverables), to take into account all we have learned since the writing of the proposal, and make the best use of the available ressources and consortium expertise. This of course would require a negotiation with the EU and probably a new amendment to the grant agreement.</p>
<p>The consortium is expecting official feedback from the Project Officer and reviewers after the formal review. In the meantime, the Coordinator and Principal Investigators of WP7 will be informally brainstorming all possibilities. Were an amendment necessary, it will be written after the current amendment for the addition of FAU Erlangen and XFEL is signed by the Commission.</p>
<h3 id="overview-of-the-deliverables-due-for-m18-and-24">Overview of the deliverables due for <a href="https://github.com/OpenDreamKit/OpenDreamKit/milestone/62?closed=1">M18</a> and <a href="https://github.com/OpenDreamKit/OpenDreamKit/milestone/65">24</a></h3>
<p>After a tour de table, the Coordinator ensured that all deliverables due for Month 24 (31/08/2017) have a leader and a definite working plan.</p>
<h3 id="best-practice-for-the-project-review">Best practice for the Project Review:</h3>
<p>BP reminded the consortium of the <a href="http://opendreamkit.org/meetings/2017-01-19-EdinburghSteeringCommittee/Review-presentation/">support slides</a> for the Review that were presentend at the Edinburgh steering committee meeting.</p>
<h3 id="feedback-from-the-quality-review-board">Feedback from the Quality Review Board</h3>
<p>HF, the chair of the Quality Review Board (QRB), expressed rough positive feedback from the first QRB meeting. A full report will be made available for all participants.</p>
Intermag 2017 - Computational micromagnetics with JOOMMF workshop Dublin, Ireland, 24 April 20172017-04-24T00:00:00+00:00http://opendreamkit.org/2017/04/24/Intermag<h2 id="main-goals">Main goals</h2>
<p>We introduced the basics of computational micromagnetics as well as taught the participants how to run JOOMMF simulations.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>JOOMMF was developed as a part of the OpenDreamKit project and two participants from the OpenDreamKit project were present to deliver the workshop (Marijan Beg, and Ryan A. Pepper). The workshop was co-funded by the conference organisers and the EPSRC CCP Computational Magnetism Network (EP/M022668/1) grant. No costs of the workshop were paid from the OpenDreamKit funds.</p>
<h2 id="event-summary">Event summary</h2>
<p>In this workshop we provided a brief introduction to computational micromagnetics. We introduced and taught the use of a Python interface to drive the OOMMF simulation package. At the beginning of the workshop, we provided a lecture style introduction, which was followed by practical exercises where attendees had an opportunity to carry out small micromagnetic calculations, modify given examples and ask more specific questions. Several days after the main event, we held a follow up session, where we were able to talk to participants about their specific needs, get requests for features, and get general feedback.</p>
<h2 id="demographic">Demographic</h2>
<p>We had 57 participants, but the organizers did not allow us to have their personal details.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>During the workshop we received the feedback from the participants about our Python interface to OOMMF as well as gained experience which helped us to structure future workshops.</p>
<p><img src="/public/images/events/IntermagPhoto1.jpg" alt="" />
![]IntermagPhoto2.jpg)</p>
<p>Intermag2017 conference in Dublin, Ireland.</p>
Sage/GAP Days 86: Algebra and Words in Combinatorics Montréal (Canada) 2017-04-17 to 212017-04-17T00:00:00+00:00http://opendreamkit.org/2017/04/17/sd86<p>## Main goals</p>
<p>This training workshop was organized at the
occasion of a four month CRM thematic semester
[Algebra and words in combinatorics(http://www.crm.umontreal.ca/Combinatorics2017/index.php/)
at the <a href="http://lacim.uqam.ca/en/">Laboratoire de combinatoire et d’informatique mathématique (LaCIM)</a> of
<a href="http://www.uqam.ca/">Université du Québec à Montréal</a>.</p>
<p>Taking advantage of the participation of Nicolas M. Thiéry to the thematic semester (as
member of the scientific committee), this workshop was followed by a <a href="https://wiki.sagemath.org/Montreal">weekly half-day
meeting</a> around computational tools for
combinatorics until July 2017, and <a href="https://more-sagemath-tutorials.readthedocs.io/en/latest/2017-05-29-CRM/">computational
sessions</a> at
the occasion of certain spring schools of the semester.</p>
<p>## OpenDreamKit implication</p>
<p>The Sage Days was organized by Nicolas M.
Thiéry (OpenDreamKit coordinator, Paris Sud) and Sébastien Labbé (since then
OpenDreamKit participant at CNRS). OpenDreamKit funded some of the invited speakers
and OpenDreamKit participants. Later events were organized by Nicolas (no
funding needed).</p>
<p>## Event summary</p>
<p>An intensive week with many brainstorms and
coding sprints + follow up meetings.</p>
<p>## Demographic</p>
<p>31 participants (including 2 from OpenDreamKit), about
half of which being female.</p>
<p>## Results and impact</p>
<p>Beside training many new Sage users
and developers, the workshop and followup events boosted an active
local user group at LaCIM, including a new generation of graduate
students and postdocs, but also François Bergeron, a prominent
figure in experimental combinatorics who used the occasion to switch
to Sage. Discussions prompted the organization by this group of
Sage Days 95, a one week Women in Sage workshop in July 2018,
where OpenDreamKit member (Sheffield) and Research Software Engineer Tania
Allard was invited as role model.</p>
<p>Altogether
<a href="{https://trac.sagemath.org/query?status=closed&status=needs_info&status=needs_review&status=needs_work&status=new&status=positive_review&keywords=~days86&col=id&col=summary&col=status&col=time&col=changetime&col=author&col=reviewer&order=status}">18 Sage tickets</a> were actively worked on during the week.</p>
The story behind our website2017-04-06T00:00:00+00:00http://opendreamkit.org/2017/04/06/website<p>OpenDreamKit uses a <em>static website</em> powered by Jekyll and GitHub. Ever wondered
what it means? <a href="http://openpyviv.com/2017/04/05/static/">Read this post</a> to discover.</p>
Report on WomenInSage2017-04-06T00:00:00+00:00http://opendreamkit.org/2017/04/06/WomenInSage<p>Last January, Viviane Pons, Jessica Striker and Jennifer Balakrishnan organized
the first <a href="https://wiki.sagemath.org/days82">WomenInSage</a>
event in Europe with OpenDreamKit. 20 women spent a week together coding and learning in a rented
house in the Paris area.</p>
<h1 id="the-workshop">The Workshop</h1>
<h2 id="opening-event">Opening event</h2>
<p>To open the workshop, Viviane, Jessica, and Jennifer gave a series of <a href="https://wiki.sagemath.org/days82_sageintro">introduction
to Sage</a> lectures at the Institut Henri Poincaré in Paris,
covering combinatorics and number theory.</p>
<h2 id="the-week">The Week</h2>
<p>The workshop then moved to the rented house. There, we organized
<em>short talk sessions</em> to get to know our respective research fields and expectations
for the week. After that, we were able to split into small groups to work on many different
projects: STL export, Krummer surfaces, Kuznyechik cipher, Motzkin words, Shioda invariants,
and more. We also had presentations on <em>How to contribute to Sage</em> (with a crash course
on git) and <em>How to write a Sage package</em>. Every evening, we had a <em>Status report</em>
session to share our progress with the group. You can read our program and final status
reports on <a href="https://wiki.sagemath.org/days82">the event wikipage</a>.</p>
<h2 id="special-pyladies-coding-cafe">Special PyLadies coding cafe</h2>
<p>Viviane Pons is one of the organizer of the local Paris chapter for <a href="http://www.pyladies.com/">PyLadies</a>.
She organized a meeting between the WomenInSage mathematician and the PyLadies developers.
We were welcomed by Algolia for an afternoon of coding-and-chatting with the PyLadies.</p>
<p><img src="/public/images/womenInSage/pyladies-WIS.jpg" alt="Group picture with PyLadies" class="img-responsive" /></p>
<h1 id="impact">Impact</h1>
<p><em>The data presented here come from a post-event questionnaire sent to the participants.</em></p>
<p>The gender gap is very important in the mathematic development community. In the
OpenDreamKit project, among the 54 participants we are only 3 women. This reflects
the global situation in the field. Many mathematician women are still hesitant to join
our community and lack confidence in their abilities as developers. Organizing a
women targeted event is a way to motivate them and building up self-confidence in a safe
and casual atmosphere.</p>
<p>The women who attended the conference had various level of programming experience
ranging from 1 (no experience) to 5 (a lot of experience).</p>
<p><img src="/public/images/womenInSage/programming-knowledge.png" alt="1: 1 people, 2: 7 people, 3: 3 people , 4: 5 people, 5: 1 people" class="img-responsive" /></p>
<p>This disparity also reflected in their knowledge of Sage.</p>
<p><img src="/public/images/womenInSage/sage-knowledge.png" alt="1: 1 people, 2: 8 people, 3: 3 people , 4: 3 people, 5: 2 people" class="img-responsive" /></p>
<p>As for contributions, only 4 participants had contributed to Sage in the past
which included the 3 organizers. Also, a majority of participants had never attended
a Sage Days before. Actually, 6 of them had never even heard of Sage Days and 2
of them said they did not think it was “for them”.</p>
<p><img src="/public/images/womenInSage/sage-days-attendance.png" alt=""I had never heard of Sage Days": 35.3%, "I thought Sage Days were not for me": 11.8%, "I wanted to but never had the time nor the occasion": 17.6% ,
"I attended once": 17.6%, "I attended many times": 17.6%" class="img-responsive" /></p>
<p>To the question “How did the fact that the event was targeted to women impact your decision to come? (Would you have participated in a classical SageDays)”,
Many participants answered that it was indeed a factor a their decision.</p>
<p><cite>Yes, but it helped. I didn´t feel so sure about my skills and being surrounded by women made things easier.</cite></p>
<p><cite>It was a new experience that I don’t regret at all.</cite></p>
<p><cite>I might have participated, but would have been less confident.</cite></p>
<p><cite>I have participated in and benefited from classical SageDays, but found this event to be even better at creating an atmosphere where everyone felt empowered to learn and contribute.</cite></p>
<p><cite>I made a special effort I would not have done for regular sage days.</cite></p>
<p>One of the participant said she would not have felt comfortable sharing a house
with men but that this event was such positive experience that she would now consider
it for other Sage days. The event helped building up the confidence of the participants, 9 of them said
they felt more confident to attend classical Sage Days after the event.</p>
<p><img src="/public/images/womenInSage/future_sage_days.png" alt="What are the chances you attend a "normal" sage days in the future? 1:0 people, 2: 0 people, 3: 5 people, 4: 2 people, 5: 10 people" class="img-responsive" /></p>
<p>We took advantage of the diverse knowledge background of our group to work together
and learn from each other. It was an occasion for many “first times” among participants
who had very little experience with Sage:</p>
<ul>
<li>5 participants installed a source version of Sage for the first time (so that they could edit the source).</li>
<li>3 used git for the first time.</li>
<li>5 used git within Sage for the first time.</li>
<li>11 got their first Trac account .</li>
<li>5 got their first contribution to a Sage ticket.</li>
<li>8 are in the process of getting their first code integrated to Sage.</li>
</ul>
<p>We worked on <a href="https://trac.sagemath.org/search?q=days82">14 tickets</a> during the week,
6 of those which have been merged since the conference. All participants said they
had learned new things and it would impact their careers.</p>
<p><img src="/public/images/womenInSage/learn_new_things.png" alt="Did you learn new things at this conference? 1:0 people, 2: 0 people, 3: 2 people, 4: 6 people, 5: 9 people" class="img-responsive" /></p>
<p><img src="/public/images/womenInSage/impact.png" alt="Did you think what you learned will have an impact on your future research/career? 1:0 people, 2: 0 people, 3: 1 people, 4: 10 people, 5: 6 people" class="img-responsive" /></p>
<p>This also was an occasion to start projects and form more research and development collaborations for future.</p>
<p><img src="/public/images/womenInSage/people.png" alt="Did you meet interesting people? 1:0 people, 2: 0 people, 3: 0 people, 4: 2 people, 5: 15 people" class="img-responsive" /></p>
<p><img src="/public/images/womenInSage/projects.png" alt="Have you started (or carried on) any project that could lead to more research and development in the future? I have started things and will try to follow up: 38.8%. I have started things and willdefinitely follow up: 35.3%. Nothing: 5.3%." class="img-responsive" /></p>
<p>All of this happened in a very casual and welcoming atmosphere. We used the common
rooms of the house to work. We cooked international, vegetarian friendly meals (some
participants had brought food and recipes from their home countries). We got to know
each other and shared more than code. All participants agreed that it was a very
positive experience. When asked to rate the general atmosphere of the conference,
all of them gave a 5.</p>
<p><img src="/public/images/womenInSage/atmosphere.png" alt="How did you like the general atmosphere of the conference? 1:0 people, 2: 0 people, 3: 0 people, 4: 0 people, 5: 17 people" class="img-responsive" /></p>
<p>As an organizer, it was also very rewarding and it motivates me to do it again. To
the question: “Any other comment you might have?”, we only got one answer.</p>
<p><cite>All three organizers were so very generous with their time and expertise, and created a wonderful supportive environment. Thank-you!</cite></p>
IOP Magnetism 2017 - Computational micromagnetics with JOOMMF workshop University of York, UK, 04 April 20172017-04-04T00:00:00+00:00http://opendreamkit.org/2017/04/04/IOP-Magnetism<h2 id="main-goals">Main goals</h2>
<p>We introduced the basics of micromagnetics as well as taught the participants how to run OOMMF simulations using our Python interface - JOOMMF.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>JOOMMF was developed as a part of the OpenDreamKit project and three participants from the OpenDreamKit were present to deliver the workshop (Hans Fangohr, Marijan Beg, and Ryan A. Pepper). The workshop was co-funded by the conference organisers and the EPSRC CCP Computational Magnetism Network (EP/M022668/1) grant. No costs of the workshop were paid from the OpenDreamKit funds.</p>
<h2 id="event-summary">Event summary</h2>
<p>In this workshop we provided a brief introduction to computational micromagnetics, Python, and the Jupyter notebook. We taught participants to use our Python interface to drive OOMMF micromagnetic simulation by guiding them through tutorials. At the beginning of the workshop, we provided a lecture style introduction, which was followed by practical exercises where attendees had an opportunity to carry out small micromagnetic calculations, modify given examples and ask more specific questions. This workshop was held together with Michael Donahue, NIST - one of the main developers of the OOMMF package.</p>
<h2 id="demographic">Demographic</h2>
<p>We had about 30 participants during the workshop, but due to the data protection regulations, the organisers did not allow us to have demographics information. During the workshop, 19 participants gave us their details with demographics: 13 males and 6 females.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>During the workshop we received the feedback from the participants about our Python interface to OOMMF as well as gained experience which helped us to structure future workshops.</p>
Exporting mathematical knowledge from Sage2017-03-25T00:00:00+00:00http://opendreamkit.org/2017/03/25/Sage-MMT-export<p><br /></p>
<h2 id="berlin-workshop">Berlin workshop</h2>
<p>This week the KWARC team (Michael Kohlhase, Florian Rabe, Dennis Müller) and myself met in Berlin at the <a href="https://www.wias-berlin.de/">WIAS</a>. The goal was to meet some of the modelers working there, who are very interested in the <a href="https://uniformal.github.io/doc/">MMT system</a> and the work in <a href="http://www.opendreamkit.org">OpenDreamKit</a>. Their entry point is <a href="https://github.com/OpenDreamKit//OpenDreamKit/tree/master/WP6">Work Package 6</a> (interoperability), motivated by the benefits they would get intrinsically from formalizing the mathematical work they do into the OMDoc/MMT language (<em>e.g.</em> addressability of mathematical models), but also with an eye on all the other work packages from OpenDreamKit (<em>e.g.</em> <a href="https://github.com/OpenDreamKit//OpenDreamKit/tree/master/WP4">interactive documents</a>). Personally, I was focused on working out what I could of a semantic interchange between Sage and GAP of mathematical objects.</p>
<h2 id="formalization-of-mathematical-concepts">Formalization of mathematical concepts</h2>
<p>To start, we decided to do a bit of prototyping around transitive groups. The first step in the Math-in-the-Middle methodology for interoperability between computer algebra systems is to formalize the mathematical concept itself. Recent progress on the MMT language has actually made this very practical (see also <a href="https://gl.mathhub.info/MitM/smglom/blob/master/source/permutation-group.mmt">here</a>):</p>
<center><img src="/assets/transitive_groups.jpg" alt="MMT syntax for transitive groups" style="width: 800px;" /></center>
<p><br /></p>
<p>A mathematician should be able to point to this and get near universal agreement in the community on what that means.</p>
<p>Line 24 is of course critical to the definition, but one can see that the rest is well structured and readable. I have omitted here the first five lines, which consist of <code class="language-plaintext highlighter-rouge">include</code> statements, and make the whole thing a completely formal definition yet implemented at a very high level of abstraction. You could slim down those <code class="language-plaintext highlighter-rouge">include</code>s and build the same thing on flexiformal foundations, <em>e.g.</em> not bother with the logic “deep down”.</p>
<p>Overall, not many mathematicians might be able to write this, but almost any mathematician can navigate her way through it. It also helps that the jEdit editor and the MathHub webserver have drastically improved, especially in ease of use (work done as part of <a href="https://github.com/OpenDreamKit//OpenDreamKit/tree/master/WP4">Work Package 4</a>), but also installation and resilience (work done as part of <a href="https://github.com/OpenDreamKit//OpenDreamKit/tree/master/WP3">Work Package 3</a>).</p>
<h2 id="math-in-the-middle-methodology">Math-in-the-Middle methodology</h2>
<p><br /></p>
<center><img src="/assets/mitm.jpg" alt="MitM paradigm" style="width: 800px;" /></center>
<p><br /></p>
<p>Now that we have a target formalization, the idea is to separately make Sage and GAP interact with it. In the <a href="https://arxiv.org/abs/1603.06424">Math-in-the-Middle (MitM) formalism</a> adopted for Work Package 6, we think of having in the “center” a system-independent flexiformalization of the mathematical domains (represented in this diagram in blue; replace in your head EC for elliptic curves with TG for transitive groups).</p>
<p>The next step is to work on the reddish clouds, which are the interface theories between this center and the other systems. These interface theories mainly flexiformalize the system-specific aspects of the domain.</p>
<p>On the GAP side, GAP generates for those interfaces OMDoc/MMT Content Dictionaries (CDs) that contain name, type, and documentation for all API functions (constructors, predicates, methods, …). This is automated, has good coverage and is very rich semantically (more on that towards the end of the post). The next step of the plan is then to align the generated system CDs with the MitM formalization by the MMT <code class="language-plaintext highlighter-rouge">implements</code> relation of aligment (<em>e.g.</em> an aligment could be: GAP-<code class="language-plaintext highlighter-rouge">transitive_group</code> MMT-<code class="language-plaintext highlighter-rouge">implements</code> MitM-<code class="language-plaintext highlighter-rouge">transitive group</code>). If equivalent Sage CDs were available, as well as Sage alignments, we would get a semantic crosswalk between GAP and Sage by composing the MitM alignments between all those different CDs. This would provide the necessary framework for interoperability.</p>
<p>At the moment Sage does export some of its knowledge into CDs, thanks to what was implemented by Nicolas Thiéry, leveraging his <code class="language-plaintext highlighter-rouge">category</code> framework. This is unfortunately not enough to cover transitive groups, which have rich structure as category objects (but the “Category of Transitive Groups” does not exist in Sage). Given the circumstances of this workshop, I thus decided to focus on the Sage side, and see what information I could extract about transitive groups.</p>
<h2 id="exporting-knowledge-from-sage">Exporting knowledge from Sage</h2>
<p>If you look at Sage’s <code class="language-plaintext highlighter-rouge">TransitiveGroup</code>, a lot of mathematical knowledge is acquired from elsewhere through the class hierarchy lying above <code class="language-plaintext highlighter-rouge">TransitiveGroup</code>, and the category framework that instruments that hierarchy. This lead me to first try to build a model of how the Sage class <code class="language-plaintext highlighter-rouge">TransitiveGroup</code> was actually implemented and what it was doing, but this was a mistake. Indeed, it was very difficult, as I got lost between meta-logics and what I was actually trying to do: modeling Sage? modeling how Sage models math? how python uses Sage to model math? I was trying to do too much, too early and was probably the wrong person to do that.</p>
<p>If you look back at the methodology, the MitM CDs don’t need to link up to the Math-in-the-Middle content dictionary right away. This is actually up to the alignments, that come later (and could be done by a different person). I was trying to do both at once, while my focus should really have been: “how do I export, but not align, as much of the math knowledge as possible embedded into Sage into a language that can easily be processed by the KWARC team?” (for the <code class="language-plaintext highlighter-rouge">categories</code> export built by Nicolas Thiéry, the export went through JSON).</p>
<p>OK then, the question now becomes: “where is math knowledge embedded in Sage that is relevant to the mathematical concept of transitive group?” The first response is of course still “Everywhere!”, but where are actually the low hanging fruits?</p>
<h3 id="a-math-skeleton">A math skeleton</h3>
<p>I found that the best way to communicate around this issue with the KWARC team is by extracting from Sage code a “math skeleton”. For this, the Sage-specific module <code class="language-plaintext highlighter-rouge">sageinspect</code> was very useful. I thus introspected the sage object corresponding to the class <code class="language-plaintext highlighter-rouge">TransitiveGroup</code>, and related objects:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># sage/src/sage/structure/sage_object.pyx
cdef class SageObject:
# sage/local/lib/python2.7/site-packages/sage/categories/category.py
class Category(UniqueRepresentation, SageObject):
# sage/src/sage/structure/category_object.pyx
cdef class CategoryObject(SageObject):
# sage/local/lib/python2.7/site-packages/sage/structure/parent.pyx
cdef class Parent(category_object.CategoryObject):
# sage/src/sage/groups/group.pyx
cdef class Group(Parent):
# sage/src/sage/groups/group.pyx
cdef class FiniteGroup(Group):
# sage/local/lib/python2.7/site-packages/sage/groups/perm_gps/permgroup.py
class PermutationGroup_generic(group.FiniteGroup):
# sage/local/lib/python2.7/site-packages/sage/groups/perm_gps/permgroup_named.py
class PermutationGroup_unique(CachedRepresentation, PermutationGroup_generic):
# sage/local/lib/python2.7/site-packages/sage/groups/perm_gps/permgroup_named.py
class TransitiveGroup(PermutationGroup_unique):
</code></pre></div></div>
<p>What is mathematical here? Clearly, just about everything, but that is because I was selective in the printout given above: I worked up the class hierarchy from <code class="language-plaintext highlighter-rouge">TransitiveGroup</code> by hand, but excluded all the python objects that <em>don’t</em> inherit from <code class="language-plaintext highlighter-rouge">SageObject</code>. For instance, you don’t see in that list:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># sage/local/lib/python2.7/site-packages/sage/structure/unique_representation.py
class CachedRepresentation:
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">CachedRepresentation</code> is only relevant, from a mathematical standpoint, in <em>where</em> it appears as a superclass. Its own internals are pure design decisions for CAS software, not mathematics.</p>
<p>The criterion to use for “related objects” is thus that only objects inheriting from <code class="language-plaintext highlighter-rouge">SageObject</code> need to be navigated. So we are navigatin in the class hierarchy diamond between <code class="language-plaintext highlighter-rouge">TransitiveGroup</code> and <code class="language-plaintext highlighter-rouge">SageObject</code>, collecting classes, which I manually imported from the <code class="language-plaintext highlighter-rouge">sage</code> library (obviously this could be automated):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>from sage.structure.sage_object import SageObject
from sage.structure.category_object import Category # not strictly in the class hierarchy, but included to facilitate discussion
from sage.structure.category_object import CategoryObject
from sage.structure.parent import Parent
from sage.groups.group import Group
from sage.groups.group import FiniteGroup
from sage.groups.perm_gps.permgroup import PermutationGroup_generic
from sage.groups.perm_gps.permgroup_named import PermutationGroup_unique
from sage.groups.perm_gps.permgroup_named import TransitiveGroup
</code></pre></div></div>
<p>This is how I selected the objects from which I wanted to extract more information, producing the list of class definitions above.</p>
<p>[Note by the way the weird changes in the path to <code class="language-plaintext highlighter-rouge">sageinspect.sage_getsource</code> in the listing above (why??? because of interactions between <code class="language-plaintext highlighter-rouge">import</code> statements?)]</p>
<h3 id="more-flesh-on-the-skeleton">More flesh on the skeleton</h3>
<p>The next step is to add a bit of flesh to that skeleton export. Obviously this is going to be more intricate. I have included <a href="https://gl.mathhub.info/ODK/Sage/blob/master/sage/transitive_group_example.py">here</a> what you get when you look at all the methods <em>coming out of the source code</em> for <code class="language-plaintext highlighter-rouge">TransitiveGroup</code>, <code class="language-plaintext highlighter-rouge">PermutationGroup_unique</code>, etc. In other words, a completely static navigation to the specific methods. This was the right thing to do for communicating with the KWARC team, but is wrong for our ultimate purpose. It was the right thing to do to communicate with KWARC (or in a blog post) as it distilled Sage to its most interesting bits, and we could fill the gaps relying on comment concepts (like “class hierarchy”). However, as a quicker way to get more consistent and richer Sage output, I could have navigated dynamically to the relevant classes, and extracted all the methods available from the live objects. This is of course because tons of methods get added when the object gets created, with a lot of mathematics packed into that. The same math could be reconstructed from the source code, but obviously that would be harder to do as we would be re-emulating a lot of what python does.</p>
<p>In any case, here is the full printout of what I get for just the method declarations for <code class="language-plaintext highlighter-rouge">PermutationGroup_generic</code>, the <code class="language-plaintext highlighter-rouge">Parent</code> that is most interesting:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
# sage/local/lib/python2.7/site-packages/sage/groups/perm_gps/permgroup.py
class PermutationGroup_generic(group.FiniteGroup):
def __init__(self, gens=None, gap_group=None, canonicalize=True, domain=None, category=None):
def construction(self):
def _has_natural_domain(self):
def _gap_init_(self):
def _magma_init_(self, magma):
def __cmp__(self, right):
def _element_class(self):
def __call__(self, x, check=True):
def _coerce_impl(self, x):
def list(self):
def __contains__(self, item):
def has_element(self, item):
def __iter__(self):
def gens(self):
def gens_small(self):
def gen(self, i=None):
def identity(self):
def exponent(self):
def largest_moved_point(self):
def degree(self):
def domain(self):
def _domain_gap(self, domain=None):
def smallest_moved_point(self):
def representative_action(self,x,y):
def orbits(self):
def orbit(self, point, action="OnPoints"):
def transversals(self, point):
def stabilizer(self, point, action="OnPoints"):
def base(self, seed=None):
def strong_generating_system(self, base_of_group=None):
def _repr_(self):
def _latex_(self):
def _order(self):
def order(self):
def random_element(self):
def group_id(self):
def id(self):
def group_primitive_id(self):
def center(self):
def socle(self):
def frattini_subgroup(self):
def fitting_subgroup(self):
def solvable_radical(self):
def intersection(self, other):
def conjugacy_class(self, g):
def conjugacy_classes(self):
def conjugate(self, g):
def direct_product(self, other, maps=True):
def semidirect_product(self, N, mapping, check=True):
def holomorph(self):
def subgroup(self, gens=None, gap_group=None, domain=None, category=None, canonicalize=True, check=True):
def as_finitely_presented_group(self, reduced=False):
def quotient(self, N):
def commutator(self, other=None):
def cohomology(self, n, p = 0):
def cohomology_part(self, n, p = 0):
def homology(self, n, p = 0):
def homology_part(self, n, p = 0):
def character_table(self):
def irreducible_characters(self):
def trivial_character(self):
def character(self, values):
def conjugacy_classes_representatives(self):
def conjugacy_classes_subgroups(self):
def subgroups(self):
def _regular_subgroup_gap(self):
def has_regular_subgroup(self, return_group = False):
def blocks_all(self, representatives = True):
def cosets(self, S, side='right'):
def minimal_generating_set(self):
def normalizer(self, g):
def centralizer(self, g):
def isomorphism_type_info_simple_group(self):
def is_abelian(self):
def is_commutative(self):
def is_cyclic(self):
def is_elementary_abelian(self):
def isomorphism_to(self, right):
def is_isomorphic(self, right):
def is_monomial(self):
def is_nilpotent(self):
def is_normal(self, other):
def is_perfect(self):
def is_pgroup(self):
def is_polycyclic(self):
def is_simple(self):
def is_solvable(self):
def is_subgroup(self, other):
def is_supersolvable(self):
def non_fixed_points(self):
def fixed_points(self):
def is_transitive(self, domain=None):
def is_primitive(self, domain=None):
def is_semi_regular(self, domain=None):
def is_regular(self, domain=None):
def normalizes(self, other):
def composition_series(self):
def derived_series(self):
def lower_central_series(self):
def molien_series(self):
def normal_subgroups(self):
def poincare_series(self, p=2, n=10):
def sylow_subgroup(self, p):
def upper_central_series(self):
</code></pre></div></div>
<p>Here are things a semi-intelligent mathematician can deduce from this fleshed-out skeleton, and that we might be able to export automatically:</p>
<ul>
<li>The arity of all those functions is useful. Unfortunately, this being python (2.x), the type cannot simply be read. Michael Kohlhase has some interesting ideas regarding mathematicians, types and the modeling necessary for MMT. I think he is right, partly, and there is much to look forward to in the services MMT can provide around type inference. Note that it will be core to this process that MMT allows for flexiformalisation as well!</li>
<li>I omitted docstrings in this export, but of course this is also useful for semantic information in natural language. Often the docstring contains structured information too, for instance some typing information (see above).</li>
<li>There is, as often, a method called <code class="language-plaintext highlighter-rouge">__init__</code> that specifies a constructor. In other words, some combination of maps from some parameter space into the object modeled by <code class="language-plaintext highlighter-rouge">PermutationGroup_generic</code>. That relationship is messy though, most of the time. Note that the GAP team took the opportunity over last summer to have an intern refactor/regularize the way they did constructors into a more “semantic” way”: essentially instead of using the elementary <code class="language-plaintext highlighter-rouge">__init__</code>, they made a <code class="language-plaintext highlighter-rouge">defconstructor</code> and gave it documentation, type information,… as parameters. Of course <code class="language-plaintext highlighter-rouge">defconstructor</code> elaborates to a call to <code class="language-plaintext highlighter-rouge">__init__</code> but the parameters can be used in the CD generation (and for static type-based optimizations later; ask Markus Pfeiffer @ St. Andrews if you are interested in the details).</li>
<li><code class="language-plaintext highlighter-rouge">_gap_xxxx</code> and <code class="language-plaintext highlighter-rouge">_magma_xxxx</code> indicate that the relevant “stuff” exists in the corresponding CASes. This is thus indicating a good place to bootstrap the alignment process between <code class="language-plaintext highlighter-rouge">gap</code> and <code class="language-plaintext highlighter-rouge">sage</code>, and therefore extract KPIs and generally optimize our progress. This would be best done by instrumenting at the <code class="language-plaintext highlighter-rouge">SageObject</code> level, since this is where all those <code class="language-plaintext highlighter-rouge">_other-computer-algebra-system_xxxx</code> methods are first located, as abstract methods.</li>
<li>the presence of magic methods <code class="language-plaintext highlighter-rouge">__xxxxxxx__</code> indicates the existence of a relation of some kind on the elements of <code class="language-plaintext highlighter-rouge">PermutationGroup_generic</code>, which is a Sage <code class="language-plaintext highlighter-rouge">Parent</code>. However, this information is best extracted from the categories export itself, presumably all(?) the time.</li>
<li><code class="language-plaintext highlighter-rouge">is_xxxx</code> methods indicate the existence of a test and thus a property.</li>
<li>after some very basic pruning, all the other methods indicate the existence of clear mathematical objects, often relatively simple maps.</li>
</ul>
<p>Many of the deductions made above will be done in the same way for all <code class="language-plaintext highlighter-rouge">Parent</code>s (at least if we go for the easiest information to grab), so that’s where the instrumentation should go. Most of that instrumentation actually makes sense to have in a CAS, beceause it exposes mathematically relevant concepts. It would simply be used by the exporter generating the Content Dictionary.</p>
<p><strong>Remark:</strong> Ultimately we want to extract information from live objects. It should not be lost, however, that what we are trying to do is partly a social process (the study of this process is itself the topic of <a href="https://github.com/OpenDreamKit//OpenDreamKit/tree/master/WP7">Work Package 7</a>). Humans have built the code from which we are trying to extract information, and now we want to communicate that with other humans so they can in turn code on top of that. Those other humans are familiar with different tools. For instance the KWARC team uses MMT related tools, like MathHub, but not Sage. Presumably other CAS developers or even “plain” mathematicians will just see Sage through an interface built on top of MMT. So I would advocate that we:</p>
<ol>
<li>Make sure to export <em>all</em> the information containing math from Sage into MMT, even that which is not readable beyond text by the system we export to;</li>
<li>Devise methods to make this informal export as addressable as possible from within MMT, but not necessarily runnable.</li>
</ol>
<p>Step 1. could be useful for instance if one is working in GAP and asking “How does Sage do that?”. We should be able to access Sage source code from within GAP, and it will be useful for automating some tasks.</p>
<p>Step 2. would be useful for students in the KWARC group, for instance, who would then be able to extract semantically richer information from a system like Sage with just verbal instructions from domain specific experts, because the data is now in MMT format. It splits the step in two: MMT extraction and semantic extraction, and requires different skills.</p>
<p>The process could be further accelerated, I bet, by exposing also deep sage introspection tools into MMT.</p>
<p>At this stage self-preservation instincts kick in and I don’t want to think deeper at this proposal from a logical standpoint.</p>
<p><br /> <em>I wish to thank Michael Kohlhase for suggestions that have improved the first draft of this post.</em></p>
Report on the WP6-WIAS Workshop on Math-in-the-Middle Content2017-03-24T00:00:00+00:00http://opendreamkit.org/2017/03/24/WIAS<p>WP6 participants JacU (Florian Rabe), FAU (Dennis Müller, Michael Kohlhase) and UZH (Paul
Olivier Dehaye) came together with members of the
<a href="http://wias-berlin.de/">Weierstrass Institute for Applied Analysis and Stochastics</a>
(WIAS: Thomas Koprucki and Carsten Tabelow) for a one-week code (20. 3. – 24. 3.) sprint
on the Math-in-the-Middle Content and Logic and the encoding of mathematical Models. The
result of this was a significant extension of the MitM ontology (in particular for the
meta-theories for Sage) and a
<a href="http://dx.doi.org/10.20347/WIAS.PREPRINT.2385">WIAS preprint on formalizations of Models</a>.</p>
Sage/GAP Days 85: packaging, portability, documentation tools Cernay (France) 2017-03-13 to 172017-03-13T00:00:00+00:00http://opendreamkit.org/2017/03/13/SD85<p>## Main goals</p>
<p>This developer meeting was focused on
initiating long term work on OpenDreamKit tasks related to packaging,
portability and documentation tools for GAP and SageMath.</p>
<p>## OpenDreamKit implication</p>
<p>This event was organized and funded by
OpenDreamKit (Paris Sud).</p>
<p>## Event summary</p>
<p>An intensive week with many brainstorms and coding sprints.</p>
<p>## Demographic</p>
<p>9 OpenDreamKit members from five sites together with
half a dozen other participants</p>
<p>## Results and impact</p>
<p>Proper packaging and distribution has
been a recurrent issue for large computational software like Sage
or GAP, and is a major task for OpenDreamKit
(\longtaskref{component-architecture}{mod-packaging}). This workshop
was a follow up to the highly successful Sage Days 77 organized in
2016 by OpenDreamKit where long term plans were designed. Here the workshop
focused on accelerating the implementation of the plans, with highly
productive collaborative coding sprints on topics such as GAP and
Sage packaging (Conda, …), Sage portability (Windows) and
documentation, GAP’s build system and release management, and how
to best integrate GAP and Sage together via a shared library.</p>
SageMathCloud for OpenDreamKit2017-03-08T00:00:00+00:00http://opendreamkit.org/2017/03/08/SMC<p>Part of OpenDreamKit’s mission is to work on <a href="https://github.com/OpenDreamKit//OpenDreamKit/tree/master/WP4">user interfaces</a>
for better collaboration and also <a href="https://github.com/OpenDreamKit//OpenDreamKit/tree/master/WP3">component architectures</a>.
This is why the <a href="http://cloud.sagemath.com/">SageMathCloud</a> platform is of special
interest for us. One of our tasks is even to <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/55">have a deeper look into its code base</a>.
In this post, as part of our <em>Review on emerging technologies</em>, we propose an overview
of the platform.</p>
<h2 id="what-is-sagemathcloud">What is SageMathCloud?</h2>
<p>SageMathCloud is an online platform which allows the creation of <strong>collaborative scientific projects</strong>
including many scientific softwares and tools like <a href="http://www.sagemath.org/">SageMath</a>,
<a href="http://jupyter.org/">Jupyter</a>, <a href="https://www.scipy.org/">SciPy</a>, <a href="http://julialang.org/">Julia</a>,
<a href="https://fr.wikipedia.org/wiki/LaTeX">Latex</a>, and more.</p>
<p>Its codebase is <a href="https://github.com/sagemathinc/smc">open-source</a>, distributed under the GNU General Public
License. The platform is run by a private company (SageMath Inc.) created by William Stein
who is also the initiator of the SageMath software. The platform offers both free
and paying premium accounts.</p>
<h3 id="projects">Projects</h3>
<p>The main tool of the SageMathCloud platform is the possibility to create <strong>projects</strong>
from which you can access the many features. A single user can create as many
projects as needed. Each project is an <strong>independant Linux virtual machine</strong>. It
thus comes with a full file system and an <strong>online terminal</strong> that allows you to
run Linux commands. The storage of each project is limited by default but can be
extended on premium accounts. You can access the files through the SageMathCloud
web interface or also through ssh.</p>
<p>One key feature is that each project can be <strong>shared by multiple users</strong>. This allows
sharing access to the files and also <strong>real time editing</strong> though the platform. Single
files or folders can also be made <strong>public</strong>. A link is then provided which allows
either viewing or downloading the files (even without a SageMathCloud account) and
also an easy way to copy onto a different SageMathCloud project owned by the viewer.</p>
<h3 id="softwares">Softwares</h3>
<p>When you create a SageMathCloud project, your Linux virtual machine comes with many
softwares and tools especially useful for mathematicians and scientists in general.
We list here the most important ones.</p>
<ul>
<li>
<p><strong>Sage</strong> and <strong>Sage worksheets</strong>. As the name indicates, the platform was primarily
developed as a replacement for the old Sage notebook server to allow collaborative
online work using Sage. The <strong>SageMath</strong> software is of course installed by default
on the virtual machine and one can run Sage through the online terminal. The platform
also offers its own <strong>Sage worksheet filetype</strong> to edit and run Sage code in a cell-type
system (as in the Jupyter notebook or the old Sage notebook) mixed with other cell types
like text and HTML. This is used to create interactive worksheets that can be easily
shared and copied.</p>
</li>
<li>
<p><strong>Jupyter</strong>. SageMathCloud includes a Jupyter notebook interface with many kernel
options (Python 2, Python 3, Anaconda, Sage, R, Julia, and more). On top of the usual
interface, SageMathCloud’s Jupyter offers <strong>real time</strong> synchronization among
multi users.</p>
</li>
<li>
<p><strong>Latex</strong>. The common document preparation system Latex is installed on the virtual
machine. It also offers a multi user editor with real time synchronization and
a dual view of both the Latex source code and pdf output.</p>
</li>
</ul>
<h3 id="notebooks-smc-sage-and-jupyter">Notebooks: SMC, Sage and Jupyter</h3>
<p>SageMathCloud offers very inovative features in terms of notebooks which should be studied
both on technichal and usability aspects.</p>
<ul>
<li>
<p><strong>Real time notebooks</strong>. Real time multi user synchronization
is a key aspect of SageMathCloud development. In particulatr, it has been a motivation
for the development of SageMathCloud homemade Sage worksheet. More recently, it has
also been added to the <strong>Jupyer notebook</strong> by <em>enhancing</em> the original
software. This enhancement is of particular interest for OpenDreamKit as this could
benefit all Jupyer users.</p>
</li>
<li>
<p><strong>Muli-kernel, multi-client</strong>. The multi-kernel philosophy is an inherent part
of Jupyer development. Indeed, Jupyer is a notebook interface that can be used
with many different language kernels (python, Sage, Julia, and more). SageMathCloud
follows the same spirit and offers a variety of kernels on the Sage worksheet. One advantage
of the Sage worksheet is that it allows for <strong>many kernels</strong> to be used in different
cells of one single worksheet (in Jupyer, the kernel has to be chosen once and for all
for the entire worksheet). Furthermore, SageMathCloud has developed what they call the <strong>Jupyer
bridge</strong>: allowing user to run a Jupyer kernel from within a Sage worksheet. In this
sense, the Sage worksheet could be seen as alternative client to Jupyer the same way one
can develop alternative kernels. More on this question can be read on the <a href="https://github.com/sagemathinc/smc/wiki/sagejupyter">github page</a> of SageMathInc.</p>
</li>
</ul>
<h2 id="sharing-and-teaching-with-sagemathcloud">Sharing and teaching with SageMathCloud</h2>
<h3 id="accessibility">Accessibility</h3>
<p>The great advantage of SageMathCloud is that it offers a <strong>complete scientific environment</strong>
without the usual setting up hassle. It makes the different software very easy to access
independently of the user personal system as long as there is an access to a good Internet
connexion. As an example, a mathematician can share a demo of code (in a Jupyter
or a Sage notebook) that could be used directly by its collaborators. Of course,
the Internet access is itself a limit. Given poor network access, for example but not only
in some developing countries where bandwidth is sometimes limited.</p>
<h3 id="teaching">Teaching</h3>
<p>When teaching is concerned, the sharing facilities of SageMathCloud come very useful.
Moreover, the platform offers a course managing system. The principle is as follows:
the teacher has acces to a “main project” containing the class material; every student
has its own project which is shared with the teacher. The course management system
allows for automatic actions like:</p>
<ul>
<li>Create all the student projects where the teacher is automatically added as a
collaborator.</li>
<li>Create assignments by copying some material from the main project to the students
projects.</li>
<li>Collecting, grading, and returning assignments by copying back and forth between
the students projects and the main project.</li>
</ul>
<p>An <em>assignment</em> is just a folder. It can have multiple content depending on the class.
Of course, the system is especially interesting when the assignment is given within
an <strong>interactive worksheet</strong> and can then be achieved by the student directly on the
interface. SageMathCloud then becomes a very good interface to initiate students
to the many scientific softwares it offers.</p>
<h2 id="sagemathcloud-and-opendreamkit">SageMathCloud and OpenDreamKit</h2>
<p>The many features of SageMathCloud make it a very interesting project for OpenDreamKit
to look at. Indeed, it offers one of the leading technologies for scientists in terms of cloud
project management, teaching and sharing facilities. In particular it showcases a collection of
<strong>features</strong> that have been selected and adopted by a wide
community.It also has some limits which
we would like to address through our project:</p>
<ul>
<li>
<p><strong>Accessibility</strong>. As previously mentioned, the cloud based interface can not be easily
accessed in places where the Internet connexion is not good enough. One solution
would be to have clear easy-to-follow instructions on how to install a SageMathCloud
platform in a local institution or on a personal machine. This is to be taken care of in D3.2 and D3.4.</p>
</li>
<li>
<p><strong>Interoperability and file formats</strong>. At the moment, the SageMathCloud platform
offers two file formats for interactive worksheet: the Jupyter one and a home-made
Sage worksheet one. It is not possible to run the Sage worksheets elsewhere than
on the platform. Especially, there is no way to run a Sage worksheet on a local
Sage installation. It is not yet clear what a long term unified worksheet solution
would be and it is part of the OpenDreamKit project to work on this question. The
technical choices made for the Sage worksheets are interesting to investigate in this
regard, as well as, file conversions and so on.</p>
</li>
</ul>
<h2 id="links">Links</h2>
<ul>
<li><a href="http://sagemath.blogspot.fr/2015/05/guiding-principles-for-sagemath-inc.html">Guiding principles for SageMath Inc.</a></li>
<li><a href="http://wstein.org/talks/2016-06-sage-bp/bp.pdf">The origin of SageMath</a> by William Stein</li>
<li><a href="http://sagemath.blogspot.fr/2014/08/what-is-sagemathcloud-lets-clear-some.html">What is SageMathCloud</a> by William Stein</li>
<li><a href="https://github.com/sagemathinc/smc/wiki/sagejupyter">Jupyter and Sage on SageMathCloud</a></li>
</ul>
Jupyter Notebooks Facilitating Productivity, Sustainability, and Accessibility of Data Science2017-03-01T00:00:00+00:00http://opendreamkit.org/2017/03/01/jupyter-cse<p>Min Ragan-Kelley presented a poster on Jupyter notebooks facilitating
productivity, sustainability, and accessibility of data science and computational science in general.
The poster included the role of OpenDreamKit-supported projects,
such as nbdime and nbval in facilitating reproducible science.</p>
<p><a href="https://figshare.com/articles/Jupyter_Notebooks_Facilitating_Productivity_Sustainability_and_Accessibility_of_Data_Science/4696414">PDF of Poster</a></p>
<p><a href="https://doi.org/10.6084/m9.figshare.4696414.v1">DOI: 10.6084/m9.figshare.4696414.v1</a></p>
SageMathCloud tutorial2017-02-28T00:00:00+00:00http://opendreamkit.org/2017/02/28/SMCTutorial<p>We just released <a href="https://mikecroucher.github.io/SMC_tutorial/">a tutorial for teachers using SageMathCloud</a>.</p>
Sage Days 84 Olot (ES), 2017-02-27 to 2018-03-102017-02-27T00:00:00+00:00http://opendreamkit.org/2017/02/27/SageDays84<h2 id="main-goals">Main goals</h2>
<p>The main goal of the Sage Days 84 were to
gather researcher specialists in polyhedral computations and advanced
SageMath developers.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit organizer: V. Delecroix.</p>
<p>OpenDreamKit participants: V. Klein</p>
<p>OpenDreamKit provided the funding source for the workshop (accommodation,
subsistence and travel expenses) for about 20k Euro.</p>
<h2 id="event-summary">Event summary</h2>
<p>The Sage Days 84 took place in Olot (Spain) from february 2nd to
march 10th. It was focused on polyhedral computations.</p>
<p>There were 12 registered participants. A typical day of the workshop
consisted in talks and hacking session.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was a development workshop dedicated to researcher polyhedral
computations. We obtain interesting feedback especialy from
the polymake developers Julian Pfeifle and Andreas Paffenholz. One
of the achievement was the integration in SageMath of an interface
to polymake
(see https://trac.sagemath.org/ticket/22452).</p>
Reports from "Computational Mathematics with Jupyter" workshop2017-02-15T00:00:00+00:00http://opendreamkit.org/2017/02/15/JupyterICMS<p>Jointly with the Collaborative Computational Project <a href="http://www.codima.ac.uk/">“CoDiMa - CCP
in the area of Computational Discrete Mathematics”</a>,
we have organised the workshop
<a href="http://opendreamkit.org/meetings/2017-01-16-ICMS/">“Computational Mathematics with Jupyter”</a>,
which took place at the <a href="http://www.icms.org.uk/">International Centre for Mathematical Sciences</a>
in Edinburgh on 16-20 January 2017. You can find some reports from the workshop here:</p>
<ul>
<li>
<p><a href="https://www.software.ac.uk/blog/2017-02-07-computational-mathematics-jupyter"><strong>Computational Mathematics with Jupyter</strong></a>
by Raniere Silva (Software Sustainability Institute) and Hans Fangohr (University of Southampton)</p>
</li>
<li>
<p><a href="https://www.software.ac.uk/blog/2017-02-13-computational-mathematics-jupyter-sprint"><strong>Computational Mathematics with Jupyter: Sprint</strong></a>
by Raniere Silva (Software Sustainability Institute)</p>
</li>
<li>
<p>Storify of <a href="https://storify.com/CIRCA_StAndrews/computational-mathematics-with-jupyter"><strong>#JupyterICMS</strong></a> tweets</p>
</li>
</ul>
Task based parallelization of recursive linear algebra routines using Kaapi2017-01-20T00:00:00+00:00http://opendreamkit.org/2017/01/20/Journee-Runtime<p>Clément Pernet gave a talk at the <a href="http://calcul.math.cnrs.fr/spip.php?article275">Journée Runtime</a>, on the work of the <a href="http://opendreamkit.org/partners/">UGA partner</a> (formerly UJF) on the parallelization of exact linear algebra using recursive tasks.</p>
<p><a href="http://calcul.math.cnrs.fr/Documents/Journees/janv2017/runtime_pernet.pdf">Pdf slides of Talk</a></p>
A case study of computational science in Jupyter notebooks: JOOMMF2017-01-20T00:00:00+00:00http://opendreamkit.org/2017/01/20/JOOMMF-intro<p>Hans Fangohr gave an introduction to computational micromagnetics and
the current workflow that is used by thousands of scientists across
the planet. He then introduced a new Python interface to the
computational tool (OOMMF), and demonstrate how this can be driven
from within a Jupyter Notebook. Through the notebook, work can be
carried out more effectively, and more reproducibly. A roadmap and
update for the Jupyter-OOMMF project was presented as well.</p>
<p><a href="http://joommf.github.io/assets/2017-01-20-Edinburgh-Jupyter-workshop-joommf.pdf">Pdf slides of Talk</a></p>
<p><a href="https://github.com/joommf/joommf.github.io/blob/master/assets/2017-01-20-Edinburgh-Jupyter-workshop-standard_problem3.ipynb">Notebook of micromagnetic simulation (standard problem 3)</a></p>
<p><a href="https://github.com/joommf/joommf.github.io/blob/master/assets/2017-01-20-Edinburgh-Jupyter-workshop-micromagneticmodel.ipynb">Notebook of micromagnetic model</a></p>
<p><a href="https://www.software.ac.uk/blog/2017-02-02-computational-mathematics-jupyter">Blog entry mentioning the presentation</a></p>
Biannual ODK Steering Committee meeting2017-01-19T00:00:00+00:00http://opendreamkit.org/2017/01/19/EdinburghSteeringCommittee<p>The biannual OpenDreamKit Steering Committee meeting is taking place in Edinburgh at the occasion of the <a href="http://opendreamkit.org/meetings/2017-01-16-ICMS/">Computational Mathematics with Jupyter</a> workshop.</p>
<ul>
<li>Date: 19/01/2017</li>
<li>Location: <a href="http://www.icms.org.uk/">ICMS</a>, Edinburgh, United-Kingdom</li>
<li>10am-12am (local time), lecture hall on the 4th room</li>
<li><a href="https://framadate.org/IsCZEQqctimhKiHU">Participants</a></li>
<li><a href="https://mensuel.framapad.org/p/odk-sc">Pad</a></li>
<li><a href="https://gitter.im/OpenDreamKit">Gitter chat</a></li>
<li><a href="https://appear.in/odk-sc">Appear.in chat room</a> (backup in case videoconference would fail)</li>
</ul>
<h2 id="brief-agenda">Brief agenda</h2>
<ul>
<li>Preparation for the formal review</li>
<li>Progress reports per site</li>
<li>Advisory Board and Quality Review Board</li>
<li>Amendment to the grant agreement</li>
<li>Key Performance Indicators</li>
<li>Deliverables due Month 18</li>
<li>WP7 topics in view of personnel changes</li>
<li>Future funding</li>
</ul>
<h2 id="detailed-agenda-and-minutes">Detailed <a href="/meetings/2017-01-19-EdinburghSteeringCommittee/steering-minutes/">agenda and minutes</a></h2>
<h2 id="progress-reports-per-site">Progress reports per site</h2>
<ul>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/ParisSud">Université Paris Sud</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/CNRS">CNRS</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/JacU">Jacobs University Bremen</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/UJF">Université Joseph Fourier</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/Kaiserslautern">University of Kaiserslautern</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/Oxford">University of Oxford</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/Silesia">University of Silesia</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/Sheffield">University of Sheffield</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/southampton.pdf">University of Southampton</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/StAndrews">University of St Andrews</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/UVersailles">Université Versailles Saint-Quentin</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/Warwick">University of Warwick</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/Zurich">Universität Zürich</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/Logilab">Logilab</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/Simula">Simula Research Laboratory</a></li>
<li><a href="/meetings/2017-01-19-EdinburghSteeringCommittee/ProgressReports/UGent/ugent.pdf">Universiteit Gent</a></li>
</ul>
Workshop: Computational Mathematics with Jupyter2017-01-16T00:00:00+00:00http://opendreamkit.org/2017/01/16/Computational_Mathematics_with_Jupyter<p>Jointly with the Collaborative Computational Project <a href="http://www.codima.ac.uk/">“CoDiMa - CCP
in the area of Computational Discrete Mathematics”</a>,
we organised a workshop
<a href="http://opendreamkit.org/meetings/2017-01-16-ICMS/">“Computational Mathematics with Jupyter”</a>.</p>
<p>It took place at the <a href="http://www.icms.org.uk/">International Centre for Mathematical Sciences</a>
in Edinburgh on 16-20 January 2017. Please see the <a href="http://opendreamkit.org/meetings/2017-01-16-ICMS/">workshop website</a> for further details.</p>
<p>The <a href="https://www.software.ac.uk/blog/2017-02-02-computational-mathematics-jupyter">Software Sustainability Institute blog</a>
is hosting a summary of the event.</p>
nbdime released2017-01-13T00:00:00+00:00http://opendreamkit.org/2017/01/13/nbdime<p><a href="https://nbdime.readthedocs.io">nbdime</a> has had its first stable release.
nbdime provides tools for diffing and merging Jupyter notebooks,
and integrating notebooks into git workflow.
nbdime aims to alleviate some common difficulties when working with Jupyter notebooks.</p>
<p>Particular features of nbdime:</p>
<ul>
<li>recognizing binary outputs that cannot be reasonably interpreted in the terminal</li>
<li>recognizing transient fields and eliminating them from merge conflicts</li>
<li>ensuring that merged notebooks are always valid</li>
<li>integration with git as <code class="language-plaintext highlighter-rouge">drivers</code> for passive diff/merge integration and <code class="language-plaintext highlighter-rouge">tools</code> for interactive GUI integration</li>
</ul>
<p>Tools provided by nbdime:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">nbshow</code>: show a legible formatting of a notebook on the command-line</li>
<li><code class="language-plaintext highlighter-rouge">nbdiff</code>: command-line diff of notebooks, eliding outputs that are known to not be renderable in a terminal. <code class="language-plaintext highlighter-rouge">nbdiff</code> can be integrated into git as a diff driver</li>
<li><code class="language-plaintext highlighter-rouge">nbdiff-web</code>: create a rich, rendered web view of the changes between two notebooks</li>
<li><code class="language-plaintext highlighter-rouge">nbmerge</code>: three-way merge with automatic conflict resolution, which can be integrated into git as a merge driver, ensuring always-valid notebooks and eliminating merge conflicts on transient fields</li>
<li><code class="language-plaintext highlighter-rouge">nbmerge-web</code>: interactive three-way merge tool for manually resolving merge conflicts</li>
</ul>
<p>More info <a href="https://nbdime.readthedocs.io">on nbdime</a>.</p>
Computational Mathematics with Jupyter ICMS, Edinburgh, Jan. 16 -- 20, 20172017-01-07T00:00:00+00:00http://opendreamkit.org/2017/01/07/Edinburgh-Jupyter<h2 id="main-goals">Main goals</h2>
<p>The focus of the workshop was on various components from the Jupyter’s ecosystem
and related projects such as, for example, Jupyter notebooks kernels for mathematical software systems,
and their applications in research, training and teaching.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The workshop has been organised by Olexandr Konovalov and Markus
Pfeiffer (USTAN). OpenDreamKit speakers included Marijan Beg, Mike Croucher, Jeroen Demeyer,
Hans Fangohr, Vidar Fauske, Olexandr Konovalov and Markus Pfeiffer. Many other OpenDreamKit members
were attending and took part in various activities taking place during the workshop.
The full list of participants is available at
https://opendreamkit.org/meetings/2017-01-16-ICMS/participants/.
The costs primarily included catering during the workshop and travel expenses.
Some expenses were covered by our partner project CoDiMa which supports computational
discrete mathematics community in the UK.</p>
<h2 id="event-summary">Event summary</h2>
<p>The workshop combined presentations and tutorials (mainly during morning sessions)
with concurrent coding and documentation sprints, which were advertised to the participants
to sign up. At the end of each day we heard brief reports from group activities.
The programme of the workshop is available
at https://opendreamkit.org/meetings/2017-01-16-ICMS/programme/. A very detailed
summary of the even is given by Raniere Silva (Software Sustainability Institute) and
Hans Fangohr at https://www.software.ac.uk/blog/2017-02-07-computational-mathematics-jupyter.</p>
<p><img src="/public/images/events/" alt="" />ICMS_Jan2017.jpg)
Participants of the workshop ``Computational Mathematics with Jupyter’’</p>
Atelier PARI/GP 2017 Grenoble (FR), 2017-01-09 to 2017-01-132017-01-05T00:00:00+00:00http://opendreamkit.org/2017/01/05/Atelier-PARI-GP<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: B. Allombert, K. Belabas, V. Delecroix, J. Demeyer,
J.-P. Flori, L. de Feo.</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and travel expenses), for about 13k Euro. The Lyon
institute of mathematics (Institut Camille Jordan) co-funded the event.</p>
<h2 id="event-summary">Event summary</h2>
<p>The 7th Atelier PARI/GP took place in Lyon (France) from january
9th to 13th.</p>
<p>There were 43 registered participants from 19 different institutions
(no registration fees).</p>
<p>A typical day of the workshop had introductory talks and tutorials
in the morning; afternoons allowed ample time for hacking sessions,
discussions and training.</p>
<p>The Atelier featured 10 morning talks on mathematical topics and
implementation projects including 4 talks by OpenDreamKit members</p>
<ul>
<li>Karim Belabas <code class="language-plaintext highlighter-rouge">Using GIT with PARI'',</code>$L$-functions’’ and
``Dirichlet characters’’</li>
<li>Bill Allombert ``New GP features’’</li>
</ul>
<p>Slides for all talks are available at
ghttp://pari.math.u-bordeaux.fr/Events/PARI2017/</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop was very productive and particularly beneficial to WP5
(high-performance computing). It also was a successful dissemination event: half
the participants had not come to a previous Atelier.</p>
Sage Days 82: Women in Sage Ris-Orangis (France), Jan. 9 -- Jan. 13, 20172017-01-01T00:00:00+00:00http://opendreamkit.org/2017/01/01/SD82<h2 id="main-goals">Main goals</h2>
<p>The main goal of the event was to initiate more women to the software Sage to reduce the gender gap in mathematics software
development. Each participant had to propose a mathematic development project to be carried out during the week.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The event was initiated by Viviane Pons from OpenDreamKit and co-organized with Jennifer Balakrishnan (Boston University) and Jessica Striker (North Dakota State University). It was funded solely by OpenDreamKit which covered: transportation for the organizers, lodging for the participants (rented house) and local food cost.</p>
<h2 id="event-summary">Event summary</h2>
<p>The opening event was a series of three lectures at Institut Henri-Poincaré on Sage and research given by the three organizers. This was followed by a one-week workshop in a rented house in Ris-Orangis. There, we organized short talk sessions to get to know our respective research fields and expectations for the week. After that, we were able to split into small groups to work on many different projects: STL export, Krummer surfaces, Kuznyechik cipher, Motzkin words, Shioda invariants, and more. We also had presentations on <code class="language-plaintext highlighter-rouge">How to contribute to Sage'' (with a crash course on git) and</code>How to write a Sage package’’. Every evening, we had a Status report session to share our progress with the group. We concluded the event with a joint coding afternoon in Paris with the PyLadies group.</p>
<h2 id="demographic">Demographic</h2>
<p>All participants were women coming from 7 different countries (France, US, Russia, Belgium, Greece, Austria, and Spain). About half of them could be considered Sage beginners. We had 7 PhD students, 4 postdoc or ATER, 8 <strong>Maîtresses de conférences</strong> or Assistant professor and 1 Emeritus professor.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>A full report on the impact of this
workshop can be read on our website:
\centerline{https://opendreamkit.org/2017/04/06/WomenInSage/}
The main goal was to make the participants more confident into their programming skills and more prone to become Sage contributors and attend classical Sage Days. It was a big success in that regard. Indeed, before the conference, only 18</p>
<p><img src="/public/images/events/pyladies-WIS.jpg" alt="" /></p>
<p>The Women in Sage at the PyLadies coding afternoon</p>
From pythran import typing2016-12-28T00:00:00+00:00http://opendreamkit.org/2016/12/28/pythran-typing<p>As part of its OpenDreamKit deliverable <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/117">D5.4 (Make Pythran typing better to improve error information)</a>, the Pythran team has written an in-depth <a href="http://serge-sans-paille.github.io/pythran-stories/from-pythran-import-typing.html">article about an unsound type checker</a>.</p>
Full-time open-ended research software engineer position2016-12-22T00:00:00+00:00http://opendreamkit.org/2016/12/22/RSE-position-warwick<p>We are seeking a full-time research software engineer at Warwick to
work with Professor John Cremona on those parts of the OpenDreamKit
project connected with the LMFDB in relation to Work Package 6. The
position is open-ended. The post holder will be employed within the
Scientific Computing Research Technology Platform (RTP) at Warwick,
initially under secondment to Professor Cremona for the duration of
the ODK project, together with longer-term responsibilities for the
development and support of research software across the University.</p>
<h1 id="job-advertisement">Job advertisement</h1>
<p>The full advertisement may be found <a href="https://atsv7.wcn.co.uk/search_engine/jobs.cgi?owner=5062452&ownertype=fair&jcode=1625541&vt_template=1457&adminview=1">here</a>.</p>
<h1 id="deadline">Deadline</h1>
<p>Deadline for applications: 26 January 2017.</p>
<h1 id="location">Location</h1>
<p>The developer will work at the <a href="http://www2.warwick.ac.uk/">University of Warwick</a>, UK.</p>
<h1 id="mission-and-activities">Mission and activities</h1>
<p>Initially, to work as part of the OpenDreamKit collaboration, on those
parts of WP6 related to the LMFDB project. Longer term, to play a
leading role in the development of research software support at
Warwick.</p>
<h1 id="skills-requirements">Skills requirements</h1>
<p>See the See the full <a href="https://atsv7.wcn.co.uk/search_engine/jobs.cgi?owner=5062452&ownertype=fair&jcode=1625541&vt_template=1457&adminview=1">job
advertisement</a>
for a detailed job description, and a list f essential and desirable
criteria for the person to be appointed.</p>
<h1 id="context">Context</h1>
<p>Until 31 August 2019, the position will be mainly funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<p>From 1 September 2019 the post will be funded by the University of Warwick.</p>
<h1 id="further-particulars">Further particulars</h1>
<p>See the full <a href="https://atsv7.wcn.co.uk/search_engine/jobs.cgi?owner=5062452&ownertype=fair&jcode=1625541&vt_template=1457&adminview=1">job
advertisement</a>.</p>
<h1 id="expressions-of-interest">Expressions of interest</h1>
<p>Interested candidates should send an email to Professor John Cremona
(j.e.cremona@warwick.ac.uk) for further information about the
position, as soon as possible.</p>
nbdime 0.1.02016-12-16T00:00:00+00:00http://opendreamkit.org/2016/12/16/nbdime-release-0.1.0<p>nbdime 0.1.0 has been released, implementing tools for diffing and merging Jupyter notebooks</p>
<p>Key features:</p>
<ul>
<li>nbdiff for diffing notebooks in the terminal</li>
<li>nbdiff-web for viewing a rich, rendered diff of two notebooks</li>
<li>nbmerge for merging three notebooks, with automatic conflict resolution that should always guarantee a valid notebook, even with unresolved conflicts</li>
<li>nbmerge-web for manually resolving conflicts when merging notebooks</li>
<li>nbshow for quickly viewing a notebook in the terminal</li>
<li>
<p>git integration for using the diff and merge tools on notebook files by default</p>
</li>
<li><a href="https://nbdime.readthedocs.io">Read the docs!</a></li>
<li><a href="https://github.com/jupyter/nbdime">Contribute!</a></li>
<li><a href="https://pypi.python.org/pypi/nbdime">Download!</a></li>
</ul>
Full-time mathematical software developer position at TU Kaiserslautern2016-12-15T00:00:00+00:00http://opendreamkit.org/2016/12/15/developer-position-kaiserslautern<p>We are seeking a full-time mathematical software developer at TU Kaiserslautern to work with Prof. Wolfram Decker on the Singular contribution to the OpenDreamKit project.</p>
<h1 id="deadline">Deadline</h1>
<p>Deadline for applications: TBA.</p>
<h1 id="location">Location</h1>
<p>The developer will work at TU Kaiserslautern in the city of Kaiserslautern Germany. Kaiserslautern is next to one of the largest contiguous forests in Europe.</p>
<h1 id="mission">Mission</h1>
<p>To work as part of the OpenDreamKit collaboration, to implement improvements via parallelisation of components of Singular.</p>
<h1 id="activities">Activities</h1>
<p>To implement parallel algorithms in Singular in C/C++.</p>
<p>Particular deliverables include:</p>
<ul>
<li>
<p>Improving the quadratic sieve for integer factorisation.</p>
</li>
<li>
<p>Parallelising the new polynomial arithmetic functionality in Singular.</p>
</li>
</ul>
<p>Depending on the skills of the applicant, the developer may also wish to contribute to other aspects of the Singular project and mathematical research in Kaiserslautern.</p>
<h1 id="skills-requirements">Skills requirements</h1>
<ul>
<li>
<p>C/C++ programming experience</p>
</li>
<li>
<p>Interest in either:</p>
<ul>
<li>algebra/number theory/algebraic geometry</li>
<li>fast arithmetic</li>
<li>the design and development of computer algebra systems</li>
</ul>
</li>
<li>
<p>Fluency in English</p>
</li>
<li>
<p>Must have an Masters degree in Mathematics fully certificated</p>
</li>
<li>
<p>Experience in Open Source development and tooling (GitHub)</p>
</li>
</ul>
<h1 id="context">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<h1 id="applications">Applications</h1>
<p>Interested candidates should send an email to both decker {at} mathematik dot uni-kl dot de and goodwillhart {at} googlemail dot com with a CV and short letter of application, as soon as possible.</p>
OpenDreamKit Virtual Research Environment2016-11-17T00:00:00+00:00http://opendreamkit.org/2016/11/17/OpenScience-conference<p><a href="http://joommf.github.io/assets/2016-11-17-openscience-ljubljana.pdf">Pdf slides of Talk</a></p>
Jupyter Day in Orsay:2016-11-04T00:00:00+00:00http://opendreamkit.org/2016/11/04/jupyterday-orsay<h1 id="-subscriptions-are-closed">!!! SUBSCRIPTIONS ARE CLOSED</h1>
<p>The LoOPS network, DevLog and OpenDreamKit are organising a day for the various tools available in the <a href="http://jupyter.org/">Jupyter environment</a>.
Notebooks are more and more used among research communities thanks to their ease of use and their interactivity. They allow
an easy access to class documentation and appealing practical exercises for students, to share ideas between colleagues and
to initiate a reflexion to allow reproducible research works.</p>
<ul>
<li>Where: Room 1-2-3 of the <a href="http://www.ias.u-psud.fr/">Institut d’Astrophysique Spatiale</a>, Orsay, France</li>
<li>When: 6th of December 2016</li>
<li>Who: four core developers of Jupyter tools will be present (S. Corlay, A, Darian, T. Kluyver, B. Ragan-Kelley) and V. Pons who is working on SageMathCloud.</li>
<li>Event organisation: Loïc Gouarin</li>
</ul>
<p><a href="https://indico.lal.in2p3.fr/event/3319/registration/register#/register">Subscriptions</a> are free but mandatory. Most of
talks and workshops will be given in English. You may need to bring your own training materials, in which case we will
warn you in advance.</p>
<h2 id="agenda">Agenda:</h2>
<ul>
<li>9h15-9h45: Welcome</li>
<li>9h45-12h45 Presentations
<ul>
<li>A. Darian et S. Corlay : JupyterLab and third-party extensions, featuring ipywidgets: the next generation of Jupyter notebooks.</li>
<li>B. Ragan-Kelley : JupyterHub: Deploying Jupyter Notebooks for students and researchers.</li>
<li>V Fauske : nbdime: diffing and merging notebooks.</li>
<li>V. Pons : the SageMathCloud platform</li>
</ul>
</li>
<li>12h45-14h00 Buffet</li>
<li>14h00-14h30 T. Kluyver: Nbconvert: make things from notebooks</li>
<li>14h30-17h30 Workshops run in parallel</li>
</ul>
<h2 id="presentations">Presentations:</h2>
<p>1) A. Darian et S. Corlay : JupyterLab and third-party extensions, featuring ipywidgets: the next generation of Jupyter notebooks</p>
<p>This talk will consist of an architectural overview and the current state of affairs of the new JupyterLab and ipywidgets.
It will feature demos of the master branch of these projects, reflecting the latest developments</p>
<p>2) B. Ragan-Kelley : JupyterHub: Deploying Jupyter Notebooks for students and researchers</p>
<p>Since the Jupyter notebook is a web-based environment, the notebook server can be run remotely, not just on your local
machine. JupyterHub is a multi-user server, aimed at helping research groups and instructors host notebook servers for
their users or students. By default, JupyterHub uses the local system users and PAM authentication, but it can be
customized to use any authentication system, including GitHub, CILogon, Shibboleth, and more. The way single-user
servers are spawned can also be customized to use services such as Docker, Kubernetes, or HPC cluster queuing systems.
The tutorial will cover a basic deployment of JupyterHub on a single machine, then extending it to use docker and GitHub
authentication, as well as general best practices for JupyterHub deployment.</p>
<p>3) V. Fauske : nbdime: diffing and merging notebooks</p>
<p>Jupyter notebooks are JSON documents containing a combination of code, prose, and output. These outputs may be rich
media, such as HTML or images. The use of JSON and including output can present challenges when working with version
control systems and code review. The JSON structure significantly impedes the readability of diffs, and simple line-based
merge tools can produce invalid results. nbdime aims to provide diff and merge tools specifically for notebooks. For diffs,
nbdime shows rendered diffs of notebooks, so that the content can be compared efficiently, rather than the raw JSON. Merges
performed with nbdime will guarantee a valid notebook as a result, even in the event of conflicts. nbdime integrates with
existing tools, such as git, so you shouldn’t need to change how you work.</p>
<p>4) V. Pons : the SageMathCloud platform</p>
<p>We will present the open-source interactive platform SageMathCloud and its many useful aspects for research collaboration
and teaching:</p>
<ul>
<li>creation of a collaborative project;</li>
<li>sharing files and worksheets;</li>
<li>using Jupyter in SageMathCloud;</li>
<li>multi-user real time editing;</li>
<li>course management with Jupyter and SageMathCloud</li>
</ul>
<p>5) T. Kluyver : Nbconvert: make things from notebooks</p>
<p>Nbconvert is a set of tools to convert notebooks to other file types, such as HTML,
Latex, or executable scripts. We’ll cover how to use it at the command line and in the
notebook interface, along with an overview of how it works. Nbconvert is also designed to be highly extensible,
and we’ll describe some of the things that can be done by building on nbconvert, such as extra converters, reports
based on input, and cross-linking between converted notebooks.</p>
OOMMF Python interface presentation2016-11-02T00:00:00+00:00http://opendreamkit.org/2016/11/02/oommf-python-mmm2016<p>Hans Fangohr presented the first prototype of the Python OOMMF interface at the 61st international meeting on magnetism and magnetic materials in New Orleans (US).</p>
<p><a href="http://joommf.github.io/assets/2016-11-02-MMM2016-CF-03-Python-OOMMF.pdf">Pdf slides of Talk</a></p>
Emerging Technologies: Windows Subsystem for Linux2016-10-04T00:00:00+00:00http://opendreamkit.org/2016/10/04/emerging-technologies-wsl<p>One of the goals of the OpenDreamKit project is to improve support for open
source mathematics software on a wider range of hardware platforms and
operating systems (see <a href="https://github.com/OpenDreamKit/OpenDreamKit/issues/50">Task 3.1:
Portability</a>). Among
the largest portability challenges is improving installation and operation of
such softwares on Microsoft Windows–still the dominant OS in many user
communities, especially on desktop and laptop computers. Despite there being
many large communities of Windows users, most open source software developers
have traditionally preferred UNIX-like software development environments. The
UNIX environment differs in many significant ways from Windows, such that
support for Windows has often been neglected by those developers.</p>
<p>As part of our series reviewing <em>emerging technologies</em>, we look at a new
development from Microsoft that has the potential to open up a smörgåsbord
of open source mathematical and scientific software to Windows users that
was never previously available to them.</p>
<h1 id="introducing-windows-subsystem-for-linux">Introducing Windows Subsystem for Linux</h1>
<p>In late March of 2016, at its annual developers’ conference, Microsoft
announced a surprising new technology. Dubbed <a href="https://msdn.microsoft.com/commandline/wsl/about">Windows Subsystem for
Linux</a> (WSL), this new
feature premiering in the Windows 10 “Anniversary Update” would add a Linux
system call compatibility layer to the Windows NT kernel, and a Windows-native
port of the popular “bash” shell. And furthermore, in partnership with
Canonical, creators of the popular <a href="http://www.ubuntu.com/">Ubuntu</a> Linux
distribution, the WSL supports Ubuntu’s “apt” package repository, giving
Windows users access to a large swath of open source software built for Ubuntu,
but running directly on Windows.</p>
<p>In short, what this means, is that Windows users will now have a
Microsoft-supported Unix-like shell environment, and the ability to run
Linux-based software directly on Windows, without a virtual machine. This
would have been unthinkable to most even a decade ago.</p>
<h1 id="why-porting-unix-software-to-windows-is-hard">Why porting UNIX software to Windows is hard</h1>
<p>Software that is compiled from languages like C and C++, often favored by
researchers, is generally built in such a way that the compiled <em>binaries</em>
support a specific operating system. Each OS has a particular <em>binary
format</em>–the way the program is organized on disk and copied into memory at
runtime. So any compiled software built for that OS has to be arranged
in the binary format for that OS in order for the OS to know how to interpret
and execute it. It is not typical for one OS to be able to understand
binaries for another OS. For example, software built for Linux uses the
<a href="https://en.wikipedia.org/wiki/Executable_and_Linkable_Format">ELF</a> binary
format; normally if one tried to run a program built for Linux on Windows,
which only understands the <a href="https://en.wikipedia.org/wiki/Portable_Executable">PE</a>
format, it will not be recognized as a valid executable.</p>
<p>An even deeper complication to writing portable software is the system calls–
software run by users interacts with the operating system to perform low-level
operations such as writing to disk, or making network connections, through
special functions provided by the operating system called “system calls”.
Modern UNIX-like operating systems follow, to an extent the <a href="https://en.wikipedia.org/wiki/POSIX">POSIX
standard</a> for system calls, allowing them
to be generally more interoperable. Windows, on the other hand, has its own
system call defitions that are not necessarily in one-to-one correspondence
with POSIX system calls. As such, a program built for Linux has no idea how to
communicate with a Windows operating system.</p>
<p>This can be a problem even on higher-level interpreted languages like Python.
Although code writing in Python abstracts away most operating system
differences, Python code <em>can</em> still access OS-specific features such as
system calls, and this is sometimes necessary to access more advanced OS
features needed by some scientific software. So Python code that uses Linux-
specific features, for example, can only run on a version of the Python
<em>interpreter</em> built for Linux.</p>
<p>A third difficulty has to do with minor differences in user interface
standards. For example, a common issue in Windows support is its different
standard for representing file paths. While Windows paths contain a “drive
letter” and uses the backslash (“<code class="language-plaintext highlighter-rouge">\</code>”) to separate between folders (e.g.
<code class="language-plaintext highlighter-rouge">C:\Windows\cmd.exe</code>), UNIX-like systems have no concept of a “drive letter”,
and use forward-slashes (“<code class="language-plaintext highlighter-rouge">/</code>”) (e.g. <code class="language-plaintext highlighter-rouge">/bin/bash</code>). Issues like this can cause
many small, but pervasive bugs when porting software between operating systems.</p>
<h1 id="how-wsl-gets-around-it">How WSL gets around it</h1>
<p>The Windows Subsystem for Linux does two main things:</p>
<ol>
<li>
<p>It enables with Windows NT kernel to understand the ELF binary format, and
<em>translate</em> it, as closely as possible, to the binary format used by
Windows.</p>
</li>
<li>
<p>It implements a sizeable subset of the POSIX system call standard on top of
Windows. Although Windows’ own system calls do not map directly the POSIX,
because Microsoft has access to how its underlying operating system is
implemented, they are able to implement the POSIX interface on top of the
lower-level details of their NT kernel.</p>
</li>
</ol>
<p>WSL also provides its own <em>bash</em> shell–a command-line interface favored by
many users of Linux. This provides a UNIX-like command-line interface within
Windows, also has an underlying system for transparently translating things
like file paths between the Windows and UNIX formats.</p>
<p>The ultimate goal is to be able to take a program compiled and built on a Linux
system, copy it over to Windows, and allow it to run without any modifications,
with all the system-level translations completely transparent to the user.
Targeting Linux software <em>specifically</em> makes this possible, because the system
interfaces it will use are well-specified <em>predictable</em> in most cases. This is
as opposed to running a virtual machine, in which an entire separate operating
system is run in order to run software on that OS, and which needs to be able
to run any arbitrary OS.</p>
<p>This is direct support for Linux software in Windows itself–there is no
virtualization.</p>
<p>This is also an improvement over previous efforts at supporting Linux software
on Windows, such as <a href="https://www.cygwin.com/">Cygwin</a>. Because Cygwin is
third-party software it cannot modify the Windows NT kernel itself. It does
not support ELF binaries–to run software with Cygwin it has to be <em>recompiled</em>
to the native PE binaries understood by Windows. It also does its best to
provide emulation of POSIX <em>system calls</em>, but it has to do this by building
them on to of the NT system calls which, as noted above, is not a one-to-one
mapping. WSL, on the other hand, provides support directly from the operating
system for POSIX and other Linux system calls.</p>
<h1 id="what-it-means-for-opendreamkit">What it means for OpenDreamKit</h1>
<p>Because WSL allows binaries built for Linux to run directly on Windows, it
makes much of the enormous repository of software built for Ubuntu (and
potentially other Linux distributions) immediately available to run on
Windows. No recompilation has to be performed or anything (at least, that
is the goal–as we’ll see below it is still not fully realized).</p>
<p>For example, Ubuntu’s software repository already includes builds of many of
the packages that are central to OpenDreamKit, such as
<a href="http://packages.ubuntu.com/trusty/gap">GAP</a>,
<a href="http://packages.ubuntu.com/trusty/pari-gp">PARI/GP</a>, and some smaller packages
including many of the dependencies of Sage. Sage itself has an unofficial
Ubuntu package–this has been found so far to nominally “work” on WSL, but
there have been found to be many bugs. That said, a great deal of other
mathematical software–especially that which is less dependent on OS-specific
features, should already work out of the box.</p>
<p>An additional potential advantage for WSL (indeed, one of the project’s goals
as detailed in <a href="http://arstechnica.com/information-technology/2016/04/why-microsoft-needed-to-make-Windows-run-linux-software/">this article at Ars
Technica</a>)
is to make the development tools and command-line interfaces favored by
UNIX-oriented developers available on Windows. This makes it possible,
in principle, to develop software like Sage the same way on both Windows
and Linux.</p>
<p>In some sense this could be an end-run around OpenDreamKit’s goal of better
supporting Windows–Microsoft has already done the lion’s share of the work for
us. But there is more to be done, and it may not be an end-all be-all
solution.</p>
<h1 id="caveats">Caveats</h1>
<p>As mentioned in the previous section, while some OpenDreamKit software has been
found to work in WSL, it is not without issues. Many bugs were found in
running Sage on WSL (and even more when trying to compile it). This is not
unexpected–the current release is marked
“<a href="https://msdn.microsoft.com/en-us/commandline/wsl/install_guide">beta</a>” by
Microsoft, and they fully acknowledge that it is buggy and incomplete.</p>
<p>Second, Microsoft has made it clear in several statements, such as in
<a href="https://blogs.Windows.com/buildingapps/2016/03/30/run-bash-on-ubuntu-on-Windows/">this blog post</a>
that the WSL and “Bash for Windows” are to be considered tools for developer
convenience <em>only</em>. It is not intended for use in a server infrastructure nor,
presumably, as a means of distributing/installing software for end-users (i.e.
who are agnostic about how the software is implemented). Although one could
take the cynical view that this just Microsoft’s way of protecting its own
server products, there are also some practical reasons for this:</p>
<ol>
<li>
<p>As a developer tool, the WSL + Bash for Windows are not easy for casual
users to install. First, it is only available on Windows 10 with the recent
(as of writing) “Anniversary Update”. Not all users are on Windows 10 yet.
It also requires having an account on Microsoft’s developer network, and for
their Windows to be configured to “developer mode” in order to receive
development-related updates, plus a few extra steps. This can also involve
some sizeable downloads. This is not especially onerous for a developer,
but is not a serious of steps that can or should be asked of the “casual”
or first-time user just to install some software.</p>
</li>
<li>
<p>Despite having support directly in the kernel, the WSL is something of a
walled garden. It is not possible to run native Windows applications from
within the Windows <em>bash</em> prompt. Nor is it possible (in any transparent
sense) to interact with Linux applications from native Windows applications.
This is probably required, on some level, to maintain a clean abstraction.</p>
</li>
</ol>
<p>Finally, it is not currently supported to run GUI applications on top of WSL,
in part because that requires a lot more than just system call compatibility.
While not supported officially by Microsoft, some hobbyists have made progress
on it though, by integrating with <a href="http://www.pcworld.com/article/3055403/Windows/Windows-10s-bash-shell-can-run-graphical-linux-applications-with-this-trick.html">existing X server implementations for
Windows</a>.
For many mathematical softwares this is a non-issue–they are text based:
numbers in; numbers out. Additionally, graphical interfaces for interactive
research environments are increasingly moving to the web (see for example
<a href="https://cloud.sagemath.com/">SageMathCloud</a>). In such
cases the GUI elements have been moved out to the web browser and the backend
typically runs “headlessly”–it has no reliance on the system’s desktop
interface.</p>
<h1 id="conclusion">Conclusion</h1>
<p>The Windows Subsystem for Linux represents a major step in the right direction
for Microsoft. It shows that they are listening to the needs of the broader
software developer community (not just those who work exclusively on Windows)
and that they have some interest in cooperating with the open source software
community (this has also been demonstrated in several other ways in recent
years).</p>
<p>For the purposes of OpenDreamKit, this work will make <em>development</em> of open
mathematical software more accessible to a wider community. Although this may
not improve accessiblity for casual end-users, many users of open research
software tend to become <em>de facto</em> developers as well, as the more they use
the software the more interested they become in modifying it for their own
purposes. Making it possible for Windows users to do development on otherwise
UNIX-oriented software, without leaving their personal desktop environments, is
appealing. Being able to compile one’s own software is also important for some
highly optimized numerical software, which tunes itself at compile time to the
computer it is being built on, sometimes with dramatic results.</p>
<p>Although this does not yet provide a fully reliable immediate solution for
porting OpenDreamKit software to Windows, we will continue to keep an eye
on WSL as it evolves.</p>
OpenDreamKit: Cloud needs and uses2016-09-27T00:00:00+00:00http://opendreamkit.org/2016/09/27/egi-workshop-krakow<section data-markdown="" data-separator="^---\n" data-separator-vertical="^--\n">
# OpenDreamKit: Cloud needs and uses
Dima Pasechnik
27 September 2016
-- [EGI](https://www.egi.eu/) workshop [Design your e-infrastructure](https://indico.egi.eu/indico/event/3025/), co-located with [DI4R](http://digitalinfrastructures.eu/)
---
## OpenDreamKit - background
***Open Digital Research Environment Toolkit***
***for the Advancement of Mathematics*** :
[OpenDreamKit.org](OpenDreamKit.org)
- [Horizon 2020](https://ec.europa.eu/programmes/horizon2020/)
**Work Programme**: [European Research Infrastructures](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures)
**Call**: Virtual Research Environments (VREs)
- **Budget**: 7.6 M€, 2015-2019 (and beyond?)
- **Consortium**: [15 sites, 50 participants](http://opendreamkit.org/partners)
Belgium, France, Germany, Norway, Poland, UK (mostly university and government labs)
--
## OpenDreamKit - background
- Objective: build a VRE for pure and applied mathematics from an ecosystem of **open source** mathematical software systems (cf. Matlab/Mathematica/Maple...)
- In particular:
- technical aspects (distribution, HPC, testing ...)
- interactive environments based on **Python** (**Jupyter** notebook, [SageMathCloud (SMC)](http://cloud.sagemath.com) ...)
- mostly based around [SageMath](http://sagemath.org), a Python library and computer algebra system, several hundred active devs, 500Mb codebase - Python, Cython, C/C++, Fortran, Lisp,...
--
## OpenDreamKit - background
- timeline: development started in 2006, ongoing
- continuous testing using a network of patch/build-bots (in-house design)
- some use of cloud resources (Google Compute Engine used by SMC, some use of Microsoft Azure), no (real) experience with EGI, EUDAT, GEANT, etc...
---
## OpenDreamKit - users
- interactive maths/CS books (about 10 published), online courses, maths databases
- used in research (pure and applied maths, CS), tens of thousands active users, running local installs, part small (10-100) mult-user systems, SMC (20000+ users), SageCell
- most current users are undergraduate students taking courses
- research users - line between users and devs is blurred (and this is good!)
---
## OpenDreamKit - current status
- existing components:
- web(browser)-based frontends (Jupyter notebooks (and kernels), SageMath
notebooks, SMC notebooks, )
- backends: SageMath and other systems (e.g. Anaconda); distributed as source, binary installs (wheels)
- online databases, such as [LMFDB](http://www.lmfdb.org/) (number-theoretic objects),...
- SMC service (run by a commercial company on Google Compute Engine, although completely with GPL software)
- development and testing frameworks
--
## OpenDreamKit - current status
- under development
- docker (and other?) containers for backends
- SMC installs made user-friendly
- semantic-aware interfaces between systems
- how e-infrastructures can help
- more integration, better testing/development plaforms?
- running SMC and similar services on federated clouds?
See our [Proposed use cases](https://github.com/OpenDreamKit/OpenDreamKit/blob/master/WP3/2016-09-27-DesignYourInfrastructureWorkshop-UseCase.md)
---
## OpenDreamKit - workshop plans
- to get an idea how federated clouds may be useful for us
- start on using some of it?
- social nets, can they help us?
---
# More reading
- [On OpenDreamKit's open and collaborative proposal writing](http://opendreamkit.org/2015/01/31/open-proposal-writing/)
- The [developer's perspective](/about-developers) on OpenDreamKit.
- [About OpenDreamKit](/about)
</section>
Emerging technologies: Running Docker Natively on Windows and Mac2016-09-16T00:00:00+00:00http://opendreamkit.org/2016/09/16/emerging-docker<p>As part of our series reviewing <em>emerging technologies</em>, we look at
the newly available <em>Docker for Windows</em> and <em>Docker for OSX</em>
technlogies produced by <a href="https://www.docker.com">Docker Inc.</a>.</p>
<p><a href="https://www.docker.com">Docker</a> is an emerging technology for packaging software in so-called containers that can run processes isolated from the rest of the operating system. It is the midway point between a complete virtual machine for a single system and a process running with limited user rights.</p>
<p>Docker provides so-called software images that package software. These images can then be executed inside a docker container. Furthermore, Docker provides a service called <a href="https://hub.docker.com/">DockerHub</a> that allows users to upload and publish their own images. This allows developers to bundle their software and distribute it easily. Furthermore, since each container runs isolated from the rest of the systems, the developers do not have to rely on any kind of other system configuration.</p>
<p>Within the OpenDreamKit project this allows us to bundle mathematical software systems (such as <a href="http://www.gap-system.org/">GAP</a>, <a href="http://www.sagemath.org/">Sage</a>, etc) and distribute them in a manner that is both accessible to users and easily maintainable for developers (see \longdelivref{component-architecture}{virtual-machines}). Furthermore the isolated aspect of docker containers allows us to easily integrate multiple systems together without having to make additional assumptions about the user’s specific setup – we can just run all systems in one docker container. This way users can install entire mathematical software stacks that the OpenDreamKit project aims to provide easily.</p>
<p>Docker was originally a Linux only application – it relied on a lot of functionality provided by the Linux kernel. To make it available on Windows and Mac the Docker developers provide a virtual machine, called <a href="https://www.docker.com/products/docker-toolbox">Docker Toolbox</a>, that runs a streamlined Linux system with Docker pre-installed. This makes it possible for Windows and Mac users to run docker containers; however it introduces an additional layer of abstraction that comes with some disadvantages. The additional virtualisation slows down docker containers and faces technical limitations when wanting to integrate with the host system. It also requires users to install virtualisation software before being able to run any kind of Docker Image. Even though Docker Toolbox automatically installs <a href="https://www.virtualbox.org/">VirtualBox</a>, this is a separate application that adds load to users machines.</p>
<p>Recently Docker started to build native versions for Windows and Mac. These versions do not rely on Linux functionality – instead they leverage functionality provided by Windows and OS X operating systems natively. In terms of the OpenDreamKit project these are a big step in terms of usability – they make it significantly easier for users of such systems to run a Docker container. Users can now install Docker just as they would install any other software on their machine. Among speed and resource advantages, these versions will make it easier for developers to create docker-powered applications and Docker containers because of better integration between host system and containers; for example the file system of the physical machine can be mounted inside containers more easily. As a side note, the Windows version of Docker only works on Windows 10 Professional and Enterprise editions and requires some manual configuration – a setting in the BIOS has to be changed (for more information see <a href="https://docs.docker.com/docker-for-Windows/#/what-to-know-before-you-install">Docker for Windows – Getting Started Documentation</a>). This is much less effort than was required previously, however it is not quite ready for adoption by inexperienced users yet. The Docker developers have stated clearly their intention to make the Docker experience in Windows and OSX as easy and streamlined as it is for Linux. When the users will be able to run Docker without the need for manual configuration or a high-end edition of Windows, we expect many components of OpenDreamKit to be avaible via Docker containers on Windows and OSX as easily as they are now for Linux.</p>
<p>Docker for Windows and Mac has been in a private Beta since March 2016 and has recently become available as a public Beta. Interested readers can find more information on the <a href="https://blog.docker.com/2016/06/docker-mac-Windows-public-beta/">Docker Blog</a>. A more detailed introduction to Docker can also be found in the <a href="https://docs.docker.com/engine/understanding-docker/">Docker Documentation</a>.</p>
nbdime - Notebook diffing and merging2016-08-26T00:00:00+00:00http://opendreamkit.org/2016/08/26/nbdime-euroscipy<p>Talk to be presented at EuroSciPy, August 2016.</p>
OpenDreamKit at ECCO 20162016-08-23T00:00:00+00:00http://opendreamkit.org/2016/08/23/ECCO<p>Last June, Viviane Pons participated to the Research School <a href="http://ecco2016.combinatoria.co/">ECCO 2016</a>.
She was there representing OpenDreamKit to lead two Sage sessions: introducing Sage
to the students in relations to the combinatorial courses they were following.</p>
<p>The school welcomed <strong>130 participants</strong> from very diverse mathematical background:
undergrads, grad students, postdocs, and professors. Each Sage sessions had <strong>50
students</strong> working on the School computers using <a href="http://www.sagemath.org/download-liveusb.html">Live USB keys</a>.
The keys were chosen to get a <strong>very quick setup</strong> (less than 10 minutes) allowing the
students to start working on the sage tutorials as soon as possible. They are a very good solutions
in countries where Internet connexion does not allow for massive cloud usage. As a result,
the students could spend their time working on the tutorials: from basic introduction
to more advanced combinatorics exercises. It was a real success! Many students brought
a copy of the live USB key home so that they would have their own copy of the software.</p>
<p>The conference as a whole was a very rewarding experience: doing a great job promoting
<strong>inclusivity</strong> and <strong>diversity</strong>. These are values we share at OpenDreamKit and we
were happy to be part of it. Viviane Pons wrote a <a href="http://openpyviv.com/2016/07/12/ECCO/">post on her blog</a>
describing her experience there. This was then re-pubiblished on the <a href="http://blogs.ams.org/matheducation/2016/08/22/an-inclusive-maths-conference-ecco-2016/">AMS Blog, On
teaching and learning Mathematics</a>.</p>
<p>The course material can be found on <a href="https://www.lri.fr/~pons/2016-06-13-conference-en.html">Viviane Pons’s webpage</a>.</p>
Jupyter-GAP now available as Docker image2016-08-03T00:00:00+00:00http://opendreamkit.org/2016/08/03/gap-docker-jupyter<p>Yesterday Olexandr Konovalov and Markus Pfeiffer created a <a href="https://docker.com">Docker</a> image containing
<a href="http://www.gap-system.org">GAP</a>, and an early version of the <a href="https://github.com/gap-packages/jupyter-gap">GAP-Jupyter</a>
interface for ease of access. It is available from <a href="https://hub.docker.com/r/gapsystem/gap-docker-jupyter/">DockerHub</a>.</p>
<p>If you have Docker already installed on your system, it should be fairly easy to get yourself up and running with the
following commands</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># docker pull gapsystem/gap-docker-jupyter
# docker run -t -i -p 8888:8888 --net="host" gapsystem/gap-docker-jupyter
# firefox http://localhost:8888
</code></pre></div></div>
<p>The above commands will start a Docker container with Jupyter and a complete GAP installation ready to use. Here’s an
example session:</p>
<p><img src="/public/img/gap-docker-jupyter-demo.png" alt="Jupyter/GAP/Docker" class="center-image" /></p>
<p>Note that this is an early version, and the roadmap for the next release is already quite
clearly laid out with many improvements. Contributions to <a href="https://github.com/gap-packages/jupyter-gap">GAP-Jupyter</a>,
and the <a href="https://github.com/gap-system/gap">GAP system</a> itself are very welcome.</p>
<p>Please contact us through the appropriate trackers on <a href="https://github.com">GitHub</a>.</p>
Sage Days 79 Jerusalem, Nov. 21 -- Nov. 24, 20162016-08-01T00:00:00+00:00http://opendreamkit.org/2016/08/01/SD79<h2 id="main-goals">Main goals</h2>
<p>This workshop was dedicated to the thematics ``geometric combinatorics and symbolic dynamics’’ in Sage. It was also a training program
for the local community in Israel with participants from the main universities (Herbrew University, Bar-Ilan University, Beer-Sheva, Ben-Gurion University, Tel-Aviv University, University of Haifa, Weizmann Institute)</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The workshop was mostly organized locally by Jean-Philippe Labbé and funded by the ERC Grant entitled ``Avenues in Probabilistic and Geometric Combinatorics’’. OpenDreamKit supported the travael costs of French participants.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event started by introduction talks and tutorials as well as installation sessions about Sage so that new comers could be initiated and guided. The rest of the week featured many specific tutorials about some mathematics features related to the thematic of the workshop. Lots of time was let open for coding and projects.</p>
<h2 id="demographic">Demographic</h2>
<p>29 participants out of the 46 came from local universities in Israel.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>As the first Sage Days in Israel, the main objective of this meeting was to introduce the software to local participants. Around half of the participants were beginners, we made around 15 installations of Sage on Windows, Mac and Linux. We allocated a lot of time for beginners to learn from tutorials but also directly from the more experienced users by personalized help. Having a flexible schedule allowed to have such help sessions.</p>
<p>Here is a summary of the accomplishments of the week:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>* Worked was carried out on 47 Sage tickets: https://trac.sagemath.org/query?keywords=~days79
* Around 15 installations on Windows, Mac and Linux.
* Creation of a sample Sage package to document the process.
* Around 10 participants got to know Sage better doing tutorials
* Experienced Sage users answered tons of questions surrounding usage of Sage
* We got 5-6 more experienced users to contribute to Sage by reviewing, reporting, writing tickets
* Many discussions and healthy debates about view, plot, show and polytope in Sage
* Some projects and discussions will be continued through future collaborations and meetings in 2017
</code></pre></div></div>
Sage Days 75: Coding theory Cernay la Ville (France), 2016-08-22 to 2016-08-262016-08-01T00:00:00+00:00http://opendreamkit.org/2016/08/01/SD75<h2 id="main-goals">Main goals</h2>
<p>The event was organized primarily by the Inria project
Actis, to celebrate the termination of its two year lifetime period. The purpose
of the project was a major redesign and implementation of the coding theory
features of SageMath. Hence the workshop gathered researchers from coding
theory, and related topics, including cryptography, group theory, combinatorics,
and linear algebra.
The goal was to expose the results of the project to the community ensure its
proper integration into the main frame of the software, and initiate new
projects, so that its development would carry over with, after the end of the
Actis engineer position</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>the event was co-organized by OpenDreamKit (through Clément
Pernet, UJF) and Inria’s Actis project. The event costed around 5000 Euro (including 300 Euro for OpenDreamKit).
A short presentation about OpenDreamKit was made during the conference to present the project to the participants.</p>
<h2 id="event-summary">Event summary</h2>
<p>We started the event by some introduction presentations and tutorials so that
the participants would familiarize themselves with Sage. Then the time was shared between lectures
and coding sprints.</p>
<p>The full program can be found on the <a href="https://wiki.sagemath.org/days75">website</a>.</p>
EuroSciPy Erlangen (Germany), 2016-08-24 to 2016-08-272016-08-01T00:00:00+00:00http://opendreamkit.org/2016/08/01/EuroSciPy<h2 id="main-goals">Main goals</h2>
<p>EuroSciPy is a gathering of the scientific Python community in Europe.
It brings together Scientific users and tools developers. Attending EuroSciPy
is a great way to confront ideas about the future of scientific development
in relations with OpenDreamKit work plan.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Benjamin Ragan-Kelley presented on the Jupyter project as a whole.
Thomas Kluyver discussed Jupyter notebooks as academic publications.
Vidar Fauske presented on nbdime, a deliverable in Work Package 4.
A sprint was organized, gathering some new contributors for Jupyter projects,
and useful discussion was had on the prospects of nbdime in the scientific software community.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The conference produced good conversations on the future of the Jupyter project and how Work Package 4 can improve scientific and open source work.
There were many discussions on the prospect of open source practices improving the scientific process,
which inform how OpenDreamKitQ can have the most impact moving the scientific community forward.</p>
Personal debriefing of CICM'20162016-08-01T00:00:00+00:00http://opendreamkit.org/2016/08/01/CICM<ul id="markdown-toc">
<li><a href="#about-this-blog-post" id="markdown-toc-about-this-blog-post">About this blog post</a></li>
<li><a href="#about-cicm" id="markdown-toc-about-cicm">About CICM</a></li>
<li><a href="#some-talks-i-particularly-enjoyed" id="markdown-toc-some-talks-i-particularly-enjoyed">Some talks I particularly enjoyed</a> <ul>
<li><a href="#invited-talk-of-john-harrison-on-hol-light-algebraic-combinatorics-at-cicm" id="markdown-toc-invited-talk-of-john-harrison-on-hol-light-algebraic-combinatorics-at-cicm">Invited talk of John Harrison on HOL Light: Algebraic Combinatorics at CICM!</a></li>
<li><a href="#michael-kohlhases-talk-on-opendreamkit-and-math-in-the-middle" id="markdown-toc-michael-kohlhases-talk-on-opendreamkit-and-math-in-the-middle">Michael Kohlhase’s talk on OpenDreamKit and Math-in-the-Middle</a></li>
<li><a href="#dennis-müllers-talk-on-alignments-rosetta-stones-on-steroids" id="markdown-toc-dennis-müllers-talk-on-alignments-rosetta-stones-on-steroids">Dennis Müller’s talk on alignments (Rosetta Stone’s on steroids)</a></li>
<li><a href="#florian-rabes-mmt-tutorial" id="markdown-toc-florian-rabes-mmt-tutorial">Florian Rabe’s MMT tutorial</a></li>
<li><a href="#jacques-carettes-talk-on-mathscheme-and-code-generation" id="markdown-toc-jacques-carettes-talk-on-mathscheme-and-code-generation">Jacques Carette’s talk on MathScheme and code generation</a></li>
<li><a href="#my-invited-talk-on-sage-category-infrastructure--" id="markdown-toc-my-invited-talk-on-sage-category-infrastructure--">My invited talk on Sage category infrastructure :-)</a></li>
</ul>
</li>
<li><a href="#knowledge-use-in-systems-stuff-i-learned-or-rethought-about" id="markdown-toc-knowledge-use-in-systems-stuff-i-learned-or-rethought-about">Knowledge use in systems: stuff I learned or rethought about</a> <ul>
<li><a href="#terminology" id="markdown-toc-terminology">Terminology</a></li>
<li><a href="#mixins" id="markdown-toc-mixins">“Mixins”</a></li>
<li><a href="#formally-defining-versus-implementing-a-realm" id="markdown-toc-formally-defining-versus-implementing-a-realm">Formally defining versus implementing a realm</a></li>
<li><a href="#inheritance-versus-composition-and-views" id="markdown-toc-inheritance-versus-composition-and-views">Inheritance versus composition and views</a></li>
<li><a href="#referencing-subcategories-from-super-categories" id="markdown-toc-referencing-subcategories-from-super-categories">Referencing subcategories from super categories</a></li>
</ul>
</li>
<li><a href="#using-views--mmt--alignments-for-sage-profit" id="markdown-toc-using-views--mmt--alignments-for-sage-profit">Using views + MMT + alignments for Sage profit?</a> <ul>
<li><a href="#context-code-duplication-in-sage" id="markdown-toc-context-code-duplication-in-sage">Context: code duplication in Sage</a></li>
<li><a href="#no-static-transliteration-in-python" id="markdown-toc-no-static-transliteration-in-python">No static transliteration in Python</a></li>
<li><a href="#but-we-have-adapters-for-dynamic-transliteration" id="markdown-toc-but-we-have-adapters-for-dynamic-transliteration">But we have <em>adapters</em> for dynamic transliteration</a></li>
<li><a href="#putting-things-together-the-explog-functors-for-free" id="markdown-toc-putting-things-together-the-explog-functors-for-free">Putting things together: the Exp/Log functors for free</a></li>
<li><a href="#the-additive-structures-hierarchy-from-its-multiplicative-counterpart" id="markdown-toc-the-additive-structures-hierarchy-from-its-multiplicative-counterpart">The additive structures hierarchy from its multiplicative counterpart</a></li>
<li><a href="#further-applications" id="markdown-toc-further-applications">Further applications</a></li>
<li><a href="#about-the-dynamic-vs-static-overhead" id="markdown-toc-about-the-dynamic-vs-static-overhead">About the dynamic vs static overhead</a></li>
</ul>
</li>
</ul>
<h2 id="about-this-blog-post">About this blog post</h2>
<p>For a number of years, I have been following the announcements of the
<a href="http://www.cicm-conference.org/">CICM (Conference in Intelligent Compupter
Mathematics)</a>, not quite daring to
hop in given that this is on the far fringe of my usual research topic
(Computational Algebraic Combinatorics). This year gave me the
occasion to attend for the first time, being honoured to deliver <a href="/2016/07/28/CICM-invited-talk/">an
invited talk</a> there about the
category infrastructure in Sage.</p>
<p>After the conference, I enjoyed two days of hiking and biking in the
nearby Bielowiesa national park which gave me plenty of time for
afterthoughts (well, when not busy dodging mosquitoes), hence this
blog post.</p>
<h2 id="about-cicm">About CICM</h2>
<p>Since a decade, CICM collocates several workshops: Calculemus
(automated reasoning systems and Computer algebra), DML (Digital Math
Library), MKM (Mathematical Knowledge Management), AISC (Artificial
Intelligence and Symbolic Computation) with the deliberate strategy to
bring together a diverse crowd of people. In particular, the Tetrapod
workshop is meant to exchange ideas between people involved in the
four main areas of <em>mechanized mathematics</em>: computation, data,
knowledge management, and deduction (proof systems).</p>
<p>I very much enjoyed the experience, discovering a nice and welcoming
community, and participating to many enlightening discussions. It was
a good opportunity to step back and enlarge my views.</p>
<h2 id="some-talks-i-particularly-enjoyed">Some talks I particularly enjoyed</h2>
<h3 id="invited-talk-of-john-harrison-on-hol-light-algebraic-combinatorics-at-cicm">Invited talk of John Harrison on HOL Light: Algebraic Combinatorics at CICM!</h3>
<p><a href="https://www.cl.cam.ac.uk/~jrh13/">John Harrison</a> is the lead
developer of <a href="https://www.cl.cam.ac.uk/~jrh13/hol-light/">HOL Light</a>,
a proof system based on higher order logic. I did not quite expect to
hear about algebraic combinatorics in this conference! So I much
enjoyed the part about the formalization of the Wilf-Zeilberger
algorithm for proving identities.</p>
<p>John’s reflections reminded me of Florent’s on his work on the
formalization of the Littlewood-Richardon’s rule: in algebraic
combinatorics, the nitty-gritty details can become very cumbersome
(lengthy case studies) to write down while being, most of the time,
irrelevant. Hence the natural – and actually vital – tendency in
this field to skip them in proofs. Here proof systems could contribute
added value, by mechanizing at least the proof verification, if not
parts of the proof of the nitty-gritty details.</p>
<p>In the case of WZ, the nitty gritty details all come from the behavior
differences between rational functions as algebraic versus analytic
functions (simplification, convergence).</p>
<h3 id="michael-kohlhases-talk-on-opendreamkit-and-math-in-the-middle">Michael Kohlhase’s talk on OpenDreamKit and Math-in-the-Middle</h3>
<p>Energetic as usual! The goal was to expose our ongoing work to the
audience, and this triggered quite some interesting exchanges.</p>
<h3 id="dennis-müllers-talk-on-alignments-rosetta-stones-on-steroids">Dennis Müller’s talk on alignments (Rosetta Stone’s on steroids)</h3>
<p>Dennis presented his ideas about a database of <em>alignments</em> between
systems, in MMT. Think the <a href="http://www.axiom-developer.org/axiom-website/rosetta.html">Rosetta
stone</a> dear
to Tim Daly’s heart, crowd-sourced and mechanized.</p>
<p>An alignment is a matching between two names for the same concept,
typically in two different systems, like:</p>
<ol>
<li>“cardinality” in Sage vs “Size” in GAP</li>
<li>“Groups” in GAP vs https://en.wikipedia.org/wiki/Groups</li>
<li>“Parent” in Sage vs “carrier” in formal logic systems</li>
<li>“Groups” vs “AdditiveGroups” in Sage</li>
</ol>
<p>An alignment can be formal as in 1 (exactly the same concept). Even
more formal would be a proven alignment between e.g. two proof
systems. On the other hand, 3 is informal (the alignment only occurs
when speaking about the elements of an algebraic structure; also the
carrier is the underlying set while the parent is the set endowed with
its structure), and 2 as well since the page content on wikipedia’ is.
The alignment can also be up to a “functor”, as in 3 where
multiplication is mapped to addition.</p>
<p>Alignments are at the core of our so-called Math-in-the-Middle
approach.</p>
<p>Dennis’s thesis:</p>
<ul>
<li>
<p>We should care both about formal and informal alignments, i.e. about
<em>flexiformality</em> as the <a href="http://kwarc.info/">KWARC group</a> would put
it.</p>
</li>
<li>
<p>The Rosetta stone should extend to proof systems, knowledge
management systems, …</p>
</li>
<li>
<p>We can achieve this by crowd sourcing & mechanizing: in short, have
system authors and users progressively contribute the alignments
they know of (manually or systematically), and compute adequate
transitive closures.</p>
<p>To bootstrap the social process, it’s important that each
contribution gives immediate added value for the contributor.</p>
</li>
</ul>
<h3 id="florian-rabes-mmt-tutorial">Florian Rabe’s MMT tutorial</h3>
<p><a href="https://kwarc.info/people/frabe/">Florian</a> led, with the help of the
other KWARC members, the first official introductory tutorial to
<a href="https://svn.kwarc.info/repos/MMT/doc/html/index.html">MMT</a>
(Meta-Meta-Theory). This was a good occasion for me to learn more
about this system for representation of mathematical knowledge, with a
strong emphasis on genericity (the meta-meta keyword): bring your own
logic, your own syntax, …</p>
<h3 id="jacques-carettes-talk-on-mathscheme-and-code-generation">Jacques Carette’s talk on MathScheme and code generation</h3>
<p>After years as core developer in the very informal world of Maple,
<a href="http://www.cas.mcmaster.ca/~carette/">Jacques Carette</a> is now
exploring the other extreme, and in particular how to best exploit the
mathematical knowledge in the design of mathematical systems (rings a
bell?).</p>
<p>Together with William Farmer, he has been developing
<a href="http://www.cas.mcmaster.ca/research/mathscheme/">MathScheme</a> “An
Integrated Framework For Computer Algebra And Computer Theorem
Proving”. A particular focus of his talk (or of my attention?) was on
automatic “code” generation, in particular using views (I’ll be using
“code”, but this includes as well proofs, …). For example, he
manages to avoid all the code duplication that we have in Sage / GAP
between the hierarchies for additive structures and multiplicative
structures by generating automatically one for the other (or maybe
both from yet another?); in practice he uses Haskell’s substitution
feature to transliterate statically all the operations appearing in
the code: 1->0, *->+, …</p>
<h3 id="my-invited-talk-on-sage-category-infrastructure--">My invited talk on Sage category infrastructure :-)</h3>
<p>I enjoyed delivering it! And there were quite some discussion during
and after the talk. Here are my <a href="http://www.cicm-conference.org/2016/slides/I3.pdf">slides</a>.</p>
<p>I first introduced my pet analogy that Sage categories are the
bookshelves of the Sage library: as mathematicians, to lookup features
of groups (e.g. definitions, theorems, proofs, algorithms, data), we
go to the library, and search through the bookshelf about group
theory. Similarly, when computing with groups, Sage searches through
the Sage category of groups to find code (and doc, and tests).</p>
<p>The bookshelves are organized hierarchically: sometimes the feature we
need is more specific, and we lookup the sub bookshelf about e.g. the
theory of finite groups. Sometimes it will be more general, and we
lookup the larger bookshelf about semigroup theory.</p>
<p>I then put some focus on code bloat taming by:</p>
<ul>
<li>
<p>Generating automatically the inheritance</p>
</li>
<li>
<p>Using the lattice structure of categories/theories to only implement
the non trivial categories (e.g. those containing code)</p>
</li>
</ul>
<h2 id="knowledge-use-in-systems-stuff-i-learned-or-rethought-about">Knowledge use in systems: stuff I learned or rethought about</h2>
<p>From the discussions at CICM, there is a consensus that, with the
growth of proof/computation/… mathematical systems, explicitly using
the mathematical knowledge is central in the design and implementation
of the former. Of course, given the themes of the conference, there
is a strong bias. Still I was glad to hear it was not just a fancy of
mine :-)</p>
<p>In practice, many systems have developed infrastructure to use such
knowledge:</p>
<ul>
<li>Computer algebra systems: categories in Axiom/Aldor/MuPAD/Sage</li>
<li>Proof systems: Gonthier’s team MathComp library for Coq</li>
<li>Knowledge management system: MMT</li>
</ul>
<p>In particular, the development of such infrastructure has always been
prompted by concrete scaling needs, and in particular code bloat
taming. The key ideas are of the same nature, but different systems
have different implementations and are more advanced in different
directions due to the variety of constraints:</p>
<ul>
<li>using an existing language or not</li>
<li>static vs dynamic</li>
<li>formal vs informal</li>
<li>current breath and depth of the system. E.g. mathcomp was originally
designed for the proof of a specific theorem which made it
relatively deep but narrow.</li>
</ul>
<p>So it’s good to be discussing across systems to learn from each other
what kind of issues our own systems may be facing next, and how such
issues have possibly been tamed elsewhere.</p>
<p>Below are notes about various things that were discussed and matured,
or that I learned at the occasion of the workshop.</p>
<h3 id="terminology">Terminology</h3>
<p>Of course, coming from different communities, we use different
terminologies. In fact terminology was the main problem I have had
when typing those notes. So one thing to discuss further would be the
standardization of terminology for all the concepts involved.</p>
<p>In the mean time, here are some more or less informal alignments:</p>
<table>
<thead>
<tr>
<th>Sage</th>
<th>logic world (e.g. MMT)</th>
</tr>
</thead>
<tbody>
<tr>
<td>Sage category</td>
<td>theory</td>
</tr>
<tr>
<td>subcategory</td>
<td>category extension</td>
</tr>
<tr>
<td>join or intersection of categories</td>
<td>push out of theories</td>
</tr>
<tr>
<td>meet of categories</td>
<td>pull back?</td>
</tr>
<tr>
<td>parent</td>
<td>carrier</td>
</tr>
<tr>
<td>functor</td>
<td>view</td>
</tr>
</tbody>
</table>
<p>Here are some other concepts for which we need to decide on common
terminology:</p>
<ul>
<li>realms, bookshelves, …</li>
<li>Mixins</li>
<li>functorial construction</li>
<li>category with axiom, category defining/implementing an axiom</li>
<li>stating that the implementation of a feature (e.g. a view, an axiom,
a construction) should be split into little chunks, one for each
category it is relevant for. By lack of something better, I’ll speak
for now of <em>split implementation</em> of a feature (better suggestions
welcome!). Paul suggests this has been called “small theory approach”
in the logic world.</li>
</ul>
<h3 id="mixins">“Mixins”</h3>
<p>Compared to other areas, in mathematics there tends to be relatively
few fundamental concepts (e.g. magmas, associativity, …); all the
richness comes from the many ways those concepts can be combined to
obtain, e.g. fields. More so, new concepts tend to appear very low in
the concept hierarchy (with exceptions; e.g. “connected” which appears
at the level of graded modules); there may be a bias in algebra
w.r.t. other areas (analysis?).</p>
<p>A key point is that knowledge enables splitting the features into
small feature sets (e.g. features of commutative groups that are not
necessarily valid for all groups) that are manageable, reusable, and
composable according to the hierarchy of realms, to cater for the
variety of combinations of the basic concepts.</p>
<p>In Sage, each such chunk is essentially a collection of “Mixin
classes”, together with some metadata.</p>
<p>Terminology: do we want to use the word “Mixin” to speak about those
feature sets?</p>
<h3 id="formally-defining-versus-implementing-a-realm">Formally defining versus implementing a realm</h3>
<p>Implementing “Groups” can have two rather distinct means:</p>
<ul>
<li>formally defining what groups are (operations / axioms)</li>
<li>providing the features that holds for groups</li>
</ul>
<p>Typically, in a book about group theory, the first pages will be about
the definition (and recap about semigroups), and all the rest about
the features of groups. In Sage, we focus mainly on the later. Jacques
liked the bookshelf analogy to express that.</p>
<p>Those two goals may be better achieved in two steps. Indeed, the
formal definitions tend to be tricky, in particular to avoid
definition loops: for example, to state that additive groups are
modules over the integer ring, one first need to define this ring
which is … an additive group itself. On the other hand, the amount
of material is much smaller, since one really only need to define the
basic concepts (magmas, associativity, …) and combine them by
trivial push outs.</p>
<p>Providing the features can then be done in a later step. So we would
have:</p>
<ul>
<li>a hierarchy of <em>defining theories</em>, with just the theories needed to
define new concepts, and possibly some trivial push outs for naming
purposes; e.g. we would have trivial push out theory that just
states that a “monoid” is a “magma” that is both “associative” and
“unital”.</li>
<li>a hierarchy of <em>feature theories</em></li>
</ul>
<p>the latter being typically ``conservative theory extensions’’ of the
former.</p>
<p>Somehow this is exactly what we are doing for Sage in the
Math-in-the-Middle approach: we use MMT’s latin1 as defining theories,
Sage categories as feature theories, and the semantic annotations of
Sage categories (aka Sage <-> MMT alignments) to produce the interface
theories aligning the two.</p>
<h3 id="inheritance-versus-composition-and-views">Inheritance versus composition and views</h3>
<p>There are several ways to express that a ring is a set endowed with
both a multiplicative monoid and an additive monoid structure. In
Sage, we use plain inheritance, having Ring inherit from
AdditiveMonoid and MultiplicativeMonoid. These two are implemented
independently (duplication!!!). If both define a function
“cayley_graph”, there is an ambiguity for rings: is the “cayley graph”
about the additive or multiplicative structure.</p>
<p>An alternative – used in MMT, MathComp, and … – is to use
composition and views, saying that Ring is the composition of
MultiplicativeMonoid and AdditiveMonoid, both being views on
Monoid. Then, the operation cayley_graph can be disambiguated by
accessing it through MultiplicativeMonoid.cayley_graph or
AdditiveMonoid.cayley_graph.</p>
<p>Using composition only would be very cumbersome: to access the zero of
the ring we would need to do use something like
<code class="language-plaintext highlighter-rouge">AdditiveCommutativeMonoid.AdditiveMonoid.UnitalMagma.unit</code>. This is
resolved in MMT by doing explicit imports/aliases: “Ring.zero” is an
alias for “AdditiveMonoid.zero” which itself is a view on
“Monoid.unit” which itself is a view on “UnitalMagma.unit”. That’s all
fine because this is all resolved statically, with no runtime overhead
(when it’s about computation).</p>
<p>Claudio mentioned that one particular reason for using composition in
MathComp is that Coq does not support multiple inheritance.</p>
<h3 id="referencing-subcategories-from-super-categories">Referencing subcategories from super categories</h3>
<p>In Sage, one can create the subcategory of a category (say Groups) by
applying axioms or constructions (say Finite) with the syntax:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Groups().Finite()
</code></pre></div></div>
<p>This is reflected in the code by having the class Groups holding an
attribute Finite that references the class FiniteGroups.</p>
<p>Jacques rightfully pointed out that this is an anti-pattern: a class
(in a rather large sense) should not need to be aware of its
subclasses, in particular so that adding new subclasses does not
require changing the code of preexisting classes.</p>
<p>That’s a valid concern which is mitigated by concrete added value:</p>
<ul>
<li>
<p>From a user interface point of view, it’s very handy to be able to
create new categories from basic ones: this enables having a single
entry points for groups, from which one can create all variants
(finite, finitely generated, compact, quotient, …).</p>
<p>An alternative syntax would be to use push outs only: Groups() &
FiniteSets(). This would not require a priori references to
subcategories. But from a Pythonic perspective, the former syntax
further enables the discovery by introspection of which
axioms/constructions can be applied.</p>
</li>
<li>
<p>This enables a fast algorithm for computing joins (roughly linear in
the size of the created hierarchy)</p>
</li>
<li>
<p>The reference Groups -> FiniteGroups could be set a posteriori by
the FiniteGroups source code, rather than the Groups source
code. This is not done so just to support lazy loading of the library.</p>
</li>
</ul>
<h2 id="using-views--mmt--alignments-for-sage-profit">Using views + MMT + alignments for Sage profit?</h2>
<p>Jacques’s talk was thought provoking, as not only he could use views
to reduce duplication in knowledge representation, but also in code,
thanks to Haskel’s code transliteration features.</p>
<p>Combined with the various other discussion points, I am now wondering
whether we could use <em>dynamic views</em> for producing uniformly, from
views and alignments represented in MMT, automatically generated split
implementations of</p>
<ul>
<li>
<p>the exp / log constructions between additive structures and multiplicative structures</p>
</li>
<li>
<p>the subquotient, quotient, subobjects and isomorphic objects constructions</p>
</li>
<li>
<p>a proper forgetful functor</p>
</li>
<li>
<p>the semantic handle Sage-XXX interface</p>
</li>
<li>
<p>the additive structures hierarchy from its multiplicative counterpart</p>
</li>
<li>
<p>facade from non facade parents (aka the OO vs modular programming disease)</p>
</li>
<li>
<p>…</p>
</li>
</ul>
<h3 id="context-code-duplication-in-sage">Context: code duplication in Sage</h3>
<p>It’s been a while that the code duplication between additive and
multiplicative structures in Sage (and most if not all other
computational systems I know of) has been itching.</p>
<p>I should say that it’s not yet critical though: we currently live with
only a small subset of features for additive structures (basically
those really needed for rings) and having those being duplicates of
the equivalent features of multiplicative structures. That’s
manageable and a plausible trade-off for some simplicity (using plain
OO).</p>
<p>Still interesting to scratch further, if not just out of curiosity.</p>
<h3 id="no-static-transliteration-in-python">No static transliteration in Python</h3>
<p>The first thing that had stopped me from scratching this issue so far
is that, unlike Haskel, Python has no transliteration feature. At
least not that I know of. Is this intrinsic? After all Python allows
for full read-write reflection on the code AST, so nothing prevents
code rewriting (what about Cython?).</p>
<p>The main difficulty however is that there is little type information
to determine, in a piece of code, when a method call should be
transliterated or not. Let’s take the following method:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def f(self, i):
return self+self, i+i
</code></pre></div></div>
<p>If we want to transliterate from additive to multiplicative, it’s
clear that we want to replace <code class="language-plaintext highlighter-rouge">self+self</code> by <code class="language-plaintext highlighter-rouge">self*self</code>. However
should <code class="language-plaintext highlighter-rouge">i+i</code> be replaced by <code class="language-plaintext highlighter-rouge">i*i</code>? Maybe this can be derived by type
inference from the context, but this is a non trivial endeavour.</p>
<h3 id="but-we-have-adapters-for-dynamic-transliteration">But we have <em>adapters</em> for dynamic transliteration</h3>
<p>Let’s take the Sage-GAP interface. For the longer while we have had a
<em>semantic handle interface</em> which allows for manipulating a handle <code class="language-plaintext highlighter-rouge">h</code>
to a GAP group <code class="language-plaintext highlighter-rouge">G</code> as if it was a native Sage group. It works using
the classical <a href="https://en.wikipedia.org/wiki/Adapter_pattern">adapter</a>
design pattern: the handle <code class="language-plaintext highlighter-rouge">h</code> is wrapped in an <em>adapter object</em> <code class="language-plaintext highlighter-rouge">Gh</code>
with adapter methods like <code class="language-plaintext highlighter-rouge">cardinality</code> that delegates calls of the
form <code class="language-plaintext highlighter-rouge">Gh.cardinality()</code> to the equivalent GAP call <code class="language-plaintext highlighter-rouge">Size(G)</code> thanks to
an</p>
<p>It had been itching me for a long time that this was available only
for groups. Last january in Saint Andrews I started making a split
implementation of this. So now we had semantic handle interfaces for
semigroups, magmas, etc for free, and for rings and such for the price
of implementing just the additional adapter methods.</p>
<p>The next step I initiated in Saint Andrews was to generate
automatically this split implementation from alignements. Indeed, most
of the time all we need to know to generate an adapter method is the
name alignment and some basic signature information.</p>
<p>This is still work in progress with Samuel Lelièvre:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> https://github.com/nthiery/sage-gap-semantic-interface
</code></pre></div></div>
<h3 id="putting-things-together-the-explog-functors-for-free">Putting things together: the Exp/Log functors for free</h3>
<p>What I realized over last week-end is that putting together</p>
<ul>
<li>The interface theory for additive structures in Sage</li>
<li>The additive -> multiplicative view in MMT</li>
<li>The interface theory for multiplicative structures in Sage</li>
<li>Dennis’ service for doing the transitive closure of the above
alignments</li>
<li>An automatic generation of adapters as above</li>
</ul>
<p>we could generate automatically, in Sage, the Exp construction that
takes an additive monoid and adapts it into a multiplicative monoid.</p>
<p>We already have a crude implementation of the Exp construction in
Sage. We use it when we have an additive monoid and get frustrated by
the lack of features available. However this construction is manually
generated and thus very far from complete. The automatic generation
would also make for a much smaller code base which is thus easier to
optimize (e.g. compile with Cython).</p>
<p>Of course the same could be done for the Log functor.</p>
<h3 id="the-additive-structures-hierarchy-from-its-multiplicative-counterpart">The additive structures hierarchy from its multiplicative counterpart</h3>
<p>The hierarchy of categories for additive structures could be generated
from the multiplicative one. To compute an operation x on an additive
monoid M, one would apply the equivalent operation on the
corresponding multiplicative monoid.</p>
<p>When no alignment exist for an operation, one could provide one
automatically by adding an additive_ prefix to the operation
name. This trick could be used as well for disambiguation, so that a
ring would have two operations additive_cayley_graph and cayley_graph,
mimicking composition.</p>
<h3 id="further-applications">Further applications</h3>
<p>An adapter method for an alignment between f and g typically looks like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def f(x, y):
wrap( g( unwrap(x), unwrap(y) ) )
</code></pre></div></div>
<p>An other location in Sage where we use very similar methods are
quotients. Typically the product in a quotient A of an ambient space B
is computed using:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def mul(x,y):
retract( g(lift(x), lift(y)) )
</code></pre></div></div>
<p>where retract is the quotient map from B to A, and lift a
section. More generally the same adapter can work for subquotients,
assuming proper lift and retract maps.</p>
<p>Based on this principle, we currently have in Sage a manually written
split implementation of a hierarchy of categories for computing the
basic operations on A just from the lift and retract map to B. The
number of features available depend on whether A is a subquotient, a
subobject, a quotient of or is isomorphic to B.</p>
<p>Using the trivial alignment, this hierarchy of categories could be
automatically generated. This would require some additional semantic
information, to specify which operation actually pass down to
quotients, subquotients, or subobjects, and not just isomorphic
objects.</p>
<h3 id="about-the-dynamic-vs-static-overhead">About the dynamic vs static overhead</h3>
<p>Using dynamic adapter classes and methods rather than static code
transliteration has a cost. Let’s consider for example the <code class="language-plaintext highlighter-rouge">Log</code>
functor, and analyse the overhead of calling a method <code class="language-plaintext highlighter-rouge">g</code> through an
adapter method <code class="language-plaintext highlighter-rouge">f</code> on adapter objects <code class="language-plaintext highlighter-rouge">x</code> and <code class="language-plaintext highlighter-rouge">y</code>, e.g. as in
<code class="language-plaintext highlighter-rouge">x.f(y)</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def f(x, y):
wrap( g( unwrap(x), unwrap(y) ) )
</code></pre></div></div>
<p>We further assume that the adapter class and methods are written in
Cython. The adapter method will typically be a generic Cython
implementation shared between all adapter methods with a given
signature, and specialized through a closure.</p>
<p>Costs:</p>
<ul>
<li>
<p><code class="language-plaintext highlighter-rouge">unwrap</code> calls: each call to unwrap above costs a Cython method
calls and a C pointer dereferencing. In a static world both could be
done at compile time, in particular through inlining.</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">wrap</code> call: one Cython function call and the creation of a Cython
object. In a static world, whenever the wrapped object is a
temporary value, there are chances that the compiler could avoid
it’s creation.</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">g</code> call: usual cost of calling <code class="language-plaintext highlighter-rouge">g</code>, potentially saving the virtual
resolution if the adapter is specialized on a specific monoid.</p>
</li>
<li>
<p><code class="language-plaintext highlighter-rouge">f</code> call: a Cython method call; potentially saving the virtual
resolution?</p>
</li>
</ul>
<p>Altogether, one could tentatively hope that the overall cost would be
of the order of magniture of a Cython method call (with virtual
resolution). For very low granularity Cython methods like arithmetic
on small fields, there still will be a high overhead compared to what
could achieve with static code transliteration. On the other hand, the
overhead would be tentatively negligible at higher granularity and for
plain Python code. This remains of course to be confirmed by
benchmarks.</p>
Infrastructure for generic code in SageMath:2016-07-28T00:00:00+00:00http://opendreamkit.org/2016/07/28/CICM-invited-talk<p>Invited talk at the Conference in Intelligent Compupter Mathematics
<a href="http://www.cicm-conference.org/2016/cicm.php">CICM 2016</a>.
See also my <a href="http://www.cicm-conference.org/2016/slides/I3.pdf">slides</a> and my
<a href="http://opendreamkit.org/activities/2016-08-01-CICM/">blog post</a> about
this conference.</p>
Diffing and Merging Jupyter Notebooks with nbdime2016-07-15T00:00:00+00:00http://opendreamkit.org/2016/07/15/nbdime-scipy-us<p>Talk to be presented at SciPy US, July 2016. Presentation to be linked when available.</p>
Interoperability in the OpenDreamKit Project:2016-07-13T00:00:00+00:00http://opendreamkit.org/2016/07/13/odk-mitm-talk-ICMS2016<p><a href="https://gl.mathhub.info/MiKoMH/talks/blob/master/source/ODK/talks/icms16.pdf">Pdf slides of Talk</a></p>
Full-time developer position opening for 20172016-07-09T00:00:00+00:00http://opendreamkit.org/2016/07/09/position-grenoble<p>This is an announcement for a full-time developer (<a href="http://fr.wikipedia.org/wiki/Ing%C3%A9nieur_de_recherche">Ingénieur de
Recherche</a>)
position opening at Université Grenoble-Alpes, working on open source
software engineering for computational mathematics.</p>
<p>contact: clement.pernet@imag.fr and jean-guillaume.dumas@imag.fr</p>
<h1 id="duration">Duration</h1>
<p>For 2 years (one year renewable) starting early 2017. Interviews will be run in December 16 and
January 17.</p>
<h1 id="salary">Salary</h1>
<p>Approx. 2200 € of monthly <em>salaire net</em> (salary after non-wage labour cost but before
income tax).</p>
<h1 id="location">Location</h1>
<p>The developer will work at the <a href="http://www-ljk.imag.fr">LJK</a> lab. on the campus of Saint-Martin d’Heres, 15 minutes
from Grenoble city center by public transportation.</p>
<h1 id="context">Context</h1>
<p>Exact linear algebra, computing with arbitrary precision integer or rationals, and over finite field, is a core component of computer algebra software and also plays a central role in numerous computation intensive applications, ranging from algebraic cryptanalysis to experimental mathematics.
The <a href="http://github.com/linbox-team/linbox">LinBox</a> library and its components, the <a href="http://github.com/linbox-team/givaro">Givaro</a> and the <a href="http://github.com/linbox-team/fflas-ffpack">FFLAS-FFPACK</a> libraries, form a C++ library ecosystem offering high performance implementation of the most common exact linear algebra routines.
These open-source libraries are integrated in the <a href="http://sagemath.org">SageMath</a> open-source mathematics software.</p>
<p>Recently the FFLAS-FFPACK library has been parallized for multi-core architectures which opens way to two development directions:</p>
<ul>
<li>propose a framework and new implementations for distributed memory architectures and accelerators such as GPU and MIC;</li>
<li>make SageMath’s linear algebra’s routines parallel by properly exposing the new features in the interface</li>
</ul>
<h1 id="mission">Mission</h1>
<p>The mission of the engineer will be to</p>
<ul>
<li>explore, experiment and develop new parallel implementations for exact linear algebra routines over distributed memory plateforms and GPU</li>
<li>improve and maintain the interface between SageMath and LinBox;</li>
<li>Participate actively in regular European development and training meetings with the other OpenDreamKit participants.</li>
<li>maintain and improve the continuous integration system for the linbox ecosystem</li>
</ul>
<h1 id="skills-and-background-required">Skills and background required</h1>
<ul>
<li>
<p>Development in Linux-like environments;</p>
</li>
<li>
<p>Experience with git</p>
</li>
<li>
<p>Experience with code optimisation, parallelism (OpenMP, MPI), debugging (valgrind, gdb);</p>
</li>
<li>
<p>Fluency in several of C, C++, Python, Cython;</p>
</li>
<li>
<p>Fluency in English;</p>
</li>
</ul>
<p>The following are not prerequisites but would be very much appreciated:</p>
<ul>
<li>
<p>Experience in open-source development (collaborative development tools, interaction with the community, …);</p>
</li>
<li>
<p>Experience with computational mathematics software, in particular
<a href="http://sagemath.org">SageMath</a> or <a href="http://github.com/linbox-team/">LinBox</a>;</p>
</li>
<li>
<p>Mathematics background.</p>
</li>
</ul>
<h1 id="applications">Applications</h1>
<p>Applicants should send an email to clement.pernet@imag.fr and jean-guillaume.dumas@imag.fr with:</p>
<ul>
<li>
<p>A complete CV</p>
</li>
<li>
<p>A motivation letter</p>
</li>
</ul>
<p>Both documents must be in an open format (pdf, ps, plain text, …) either in
french or english.</p>
<h1 id="context-1">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Led by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<p>The developer will work within a team of developpers of SageMath and LinBox, and researchers in the field of high performance computer algebra
at the <a href="http://www-ljk.imag.fr/">Laboratoire Jean Kuntzmann</a>.</p>
pythran 0.7.62016-07-06T00:00:00+00:00http://opendreamkit.org/2016/07/06/pythran-release-0.7.6<p>Pythran 0.7.6 has been released. It’s a big step forward better handling of complex scientific code:</p>
<ul>
<li>type engine improvement</li>
<li>improved value binding analysis</li>
<li>constant global variable support</li>
</ul>
<p>Additionally, some notable improvements can benefit to the scientific community:</p>
<ul>
<li>better Jupyter notebook integration</li>
<li>restored and tested OpenMP support</li>
<li>broader support of Numpy broadcasting</li>
</ul>
<p>And as usual, several performance improvements, more Numpy API support, and
hopefully more bugs solved than the number of newly introduced ones!</p>
<ul>
<li><a href="http://pythonhosted.org/pythran">Read the doc!</a></li>
<li><a href="https://github.com/serge-sans-paille/pythran">Contribute!</a></li>
<li><a href="https://pypi.python.org/pypi/pythran">Download!</a></li>
</ul>
Sage Days 78: Combinatorics Vancouver (Canada), 2016-06-29 to 2016-07-012016-07-05T00:00:00+00:00http://opendreamkit.org/2016/07/05/CICM<h2 id="main-goals">Main goals</h2>
<p>The event was organized as a satellite event of the yearly international conference
in algebraic combinatorics <a href="https://sites.google.com/site/fpsac2016/">FPSAC</a>. The objective was to gather
the combinatorics community around Sage development, to introduce Sage to newcomers (especially graduate students) and
to bring new Sage contributions.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>the event was co-organized by OpenDreamKit (through Viviane Pons) and the
<a href="https://www.pims.math.ca/">Pacific Institute for the Mathematical Science</a> where
it was hosted. The event costed around 4000 CAD (2000 CAD from OpenDreamKit).
A short presentation about OpenDreamKit was made during the conference to present
the project to the participants.</p>
<h2 id="event-summary">Event summary</h2>
<p>We started the event by some introduction presentations and tutorials so that
the participants would familiarize themselves with Sage. Then the time was shared between lectures
and coding sprints. Here are some highlights:</p>
<ul>
<li>Our invited speaker ## Mike Zabrock</li>
</ul>
<p>(York Univ.) gave a lecture on <strong>Open Problems in Combinatorial Representation Theory</strong>.</p>
<ul>
<li>
<h2 id="emily-gunawa">Emily Gunawa</h2>
</li>
</ul>
<p>(Univ. of Minnesota) and ## Jessica Strike</p>
<p>(North Dakota State Univ.) gave respectively
a tutorial and a lecture on <strong>Research-based coding</strong> for Sage.</p>
<ul>
<li>An undergrad student ## Amit Jamadagn</li>
</ul>
<p>gave a presentation of the extensive package on <strong>Knot Theory</strong> that
he developed during a Google Summer of code project.</p>
<p>The full program can be found on the href{https://wiki.sagemath.org/days78}{website}. We planned lots of time for
participants to work on development projects such as: Plane partitions, plotting functions for combinatorics
objects, Lie algebras, Rook placements, …</p>
<h2 id="demographic">Demographic</h2>
<p>The participants were required to fill out a demographic survey. We had 29 participants (24 males and
5 females), 27 identified as academics: 7 professors, 6 postdoc, 11 graduate students, and 3 undergrads. 19 participants
were from North America (10 from Canada and 9 from the US), 8 were from Europe (France, Austria, and Switzerland), and 3 from
Asia (South Korea and India).</p>
<h2 id="results-and-impact">Results and impact</h2>
<ul>
<li>Newcomers got to use Sage for the first time around one third of the participants had zero or very little experience with Sage before the meeting. By the
end of the three days, everyone had a way to use Sage (either online or on their machines)
and had written a bit of code.</li>
<li>Newcomers got to contribute to Sage
a lecture was given on how to contribute to Sage
and groups were formed on different projects mixing more experienced people with newcomers so
that the code that was written could end up being merged to the software. In particular, a implementation
of Plane Partitions was put together by a participant who had never used Sage before.</li>
<li>New contributions were made in the combinatorics component of Sage
we used the keyword days7 on the trac server of Sage to track the contributions that were submitted during the workshop.
Altogether the participants
worked on 17 different tickets either reviewing
existing ticket, implementing, or creating new tickets. 6 of them already got positive reviews and are
on the process of being merged to the software.</li>
</ul>
<p>\begin{tabular}{cc}
<img src="/public/images/events/pictures/sd78-1.jpg =40%" alt="" />
<img src="/public/images/events/pictures/sd78-2.jpg =40%" alt="" />
Participants of Sage Days 78 making Sage demo</p>
Invited Seminar Talk at ``Universidade Nova de Lisboa'' Lisbon, Portugal), 2016-07-192016-07-01T00:00:00+00:00http://opendreamkit.org/2016/07/01/talk-lisbon<h2 id="main-goals">Main goals</h2>
<p>Advertise capabilities of OpenDreamKit with a research talk
that showcases the GAP Jupyter interface.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Markus Pfeiffer showcased the GAP Jupyter notebook
interface as part of a research seminar talk on search in permutation groups.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This seminar talk was a good opportunity to advertise OpenDreamKit outside of
our core developer or user groups.</p>
OpenMath in OpenDreamKit2016-06-30T00:00:00+00:00http://opendreamkit.org/2016/06/30/OM_in_ODK<p>The <a href="http://opendreamkit.org">OpenDreamKit</a> project aims to build a virtual research environment for
mathematics from an ecosystem of open source mathematical software systems. A crucial subgoal in this
is to enable the systems to communicate mathematical objects, e.g. concrete elliptic curves or groups.</p>
<p>Several approaches exist (direct c-level library calls, semantic
handles, …),, cattering for different needs, and OpenDreamKit
pursues them all.</p>
<p>Of particular interest is the use of <a href="http://openmath.org">OpenMath</a>, a format which is
designed for a content-oriented representation of mathematical concepts, objects, and models. To create
common meaning space, the OpenDreamKit project is pursuing the
<a href="https://github.com/OpenDreamKit/OpenDreamKit/raw/master/WP6/CICM2016/published.pdf">Math-in-the-Middle</a> (MitM)
paradigm to establish an ontology of mathematics that can act as a joint context for the communication.</p>
<p>In the OpenMath terminology, the MitM ontology acts as a set of content dictionaries (CDs) that anchor the OpenMath
objects semantically. For communication between the systems we only need to equip them with OpenMath phrasebooks
(I/O libraries for OpenMath objects). The MitM ontology will be semi-automatically curated and connected to
“system interface ontologies” (CDs for the system objects).</p>
<p>The MitM Ontology can enable multiple added value services including:</p>
<ul>
<li>remote evaluation of mathematical expressions via
<a href="http://www.sciencedirect.com/science/article/pii/S0747717111002124">SCSCP</a></li>
<li>MONET-style service discovery via the MitM CDs.</li>
</ul>
<p>The current state of play is that we have initial exports of system interface ontologies
for three systems (the exporters are under development still, so your mileage may vary).</p>
<ul>
<li><a href="http://www.sagemath.org/">SageMath</a> (265 CDs);
<a href="https://mathhub.info/mh/mmt/?http://www.sagemath.org/">see them on MathHub.info</a></li>
<li><a href="http://www.gap-system.org/">GAP</a> (210 CDs with 2996 symbols)
<a href="https://mathhub.info/mh/mmt/?http://www.gap-system.org/">see them on MathHub.info</a>, and</li>
<li>(partially) and the <a href="http://lmfdb.org">LMFDB</a>.</li>
</ul>
<p>The <a href="https://mathhub.info/mh/mmt/?http://mathhub.info/ODK/math/">MitM ontology</a> is still very much experimental.</p>
<p>All CDs are encoded in <a href="http://omdoc.org">OMDoc</a>/<a href="https://uniformal.github.io">MMT</a>, which
is legal by the OpenMath2 standard. Lossful coversions to standalone OMCDs are possible,
but have not been pursued at the moment.</p>
Full time position2016-06-28T00:00:00+00:00http://opendreamkit.org/2016/06/28/position_sheffield<p><a href="http://www.jobs.ac.uk/job/ANT812/research-software-engineer/">A full time research software engineer position</a> starting June/July 2016 for two years.</p>
Full-time developer position opening for Fall 2016 or Fall 20172016-06-28T00:00:00+00:00http://opendreamkit.org/2016/06/28/developer-position2-bordeaux<p>This is an announcement for several full-time developer (<a href="http://fr.wikipedia.org/wiki/Ing%C3%A9nieur_de_recherche">Ingénieur de
Recherche</a>)
position opening at Université de Bordeaux, working on open source
software engineering for computational mathematics.</p>
<p>contact: vincent.delecroix-AT-labri.fr</p>
<h1 id="duration">Duration</h1>
<p>1 to 3 years starting in Fall 2016 or 2017</p>
<h1 id="salary">Salary</h1>
<p>Depending on the applicant’s past experience, 1600 € to 2900 €
of monthly <em>salaire net</em> (salary after non-wage labour cost but before
income tax). Equivalently, this is a yearly <em>salaire brut</em> of
23600 € to 40000 €.</p>
<h1 id="location">Location</h1>
<p>The developer will work at the
<a href="http://labri.fr">LaBRI</a> in the Talence campus, 10 minutes
from Bordeaux city center by public transport.</p>
<h1 id="mission">Mission</h1>
<p>To support the maintenance and development of the OpenDreamKit
components, and in particular of the <a href="http://sagemath.org">SageMath</a> and
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a> projects.</p>
<h1 id="activities">Activities</h1>
<p>According to his or her specific skills, the developer will take on, in close
collaboration with the community, a selection of the software engineering tasks
defined in the OpenDreamKit project. Among these:</p>
<ul>
<li>
<p>Improve the existing software components (code optimisation and
parallelisation);</p>
</li>
<li>
<p>Improve communication between the software components;</p>
</li>
<li>
<p>Improve the graphic interfaces;</p>
</li>
<li>
<p>Participate actively in regular European development and training meetings
with the other OpenDreamKit participants.</p>
</li>
</ul>
<h1 id="skills-and-background-required">Skills and background required</h1>
<ul>
<li>
<p>Development in Unix-like environments;</p>
</li>
<li>
<p>Experience with code optimisation, parallelism (pthread, OpenMP, MPI),
and debugging tools (valgrind, gdb);</p>
</li>
<li>
<p>Fluency in several of C, C++, Python, Cython, assemblers;</p>
</li>
<li>
<p>Fluency in English;</p>
</li>
</ul>
<p>The following are not prerequisites but would be very much appreciated:</p>
<ul>
<li>
<p>Experience in open-source development (collaborative
development tools, interaction with the community, …);</p>
</li>
<li>
<p>Experience with computational mathematics software, in particular
<a href="http://sagemath.org">SageMath</a> or <a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>;</p>
</li>
<li>
<p>Mathematics background.</p>
</li>
<li>
<p>System administration.</p>
</li>
</ul>
<h1 id="applications">Applications</h1>
<p>Applicants should send an email to vincent-DOT-delecroix-AT-labri-DOT-fr with:</p>
<ul>
<li>
<p>A complete CV</p>
</li>
<li>
<p>A motivation letter</p>
</li>
</ul>
<p>Both documents must be in an open format (pdf, ps, plain text, …) either in
french or english.</p>
<h1 id="context">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Led by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<p>The developer will work within one of the largest teams of SageMath
and PARI/GP developers, composed essentially of researchers in
mathematics and computer science, at the <a href="http://www.labri.fr/">Laboratoire Bordelais de Recherche en
Informatique</a> and in nearby institutions
such as the <a href="http://www.math.u-bordeaux1.fr/imb/spip.php">Institut de Mathématiques de Bordeaux</a>
and <a href="https://www.inria.fr/centre/bordeaux">INRIA Bordeaux</a>.</p>
<h1 id="comments">Comments</h1>
<p>:warning: This is <em>not</em> a postdoc position. While side research will
be welcome, and a few tasks may possibly lead to some research
problems in computer science, the core tasks will be pure
development. Candidates wishing to pursue an academic research career
in the long run should consider twice whether this opportunity is
adequate for them.</p>
Project meetings, review and workshop2016-06-27T00:00:00+00:00http://opendreamkit.org/2016/06/27/ODK_project_meetings_review_and_workshop_in_Jacobsuni<p><a href="/meetings/2016-06-27-Bremen/">OpenDreamKit’s annual project meeting</a>
took place in Bremen, from Monday the 27th of June to Friday the
1st of July. The beginning of the week was dedicated to ODK’s open
and internal meetings, including an informal review with our Project
Officer and the Commission reviewers. The rest of the week was
dedicated to joint work sessions on WP4 (User Interfaces) &
WP6 (Data/Knowledge/Software bases, aka Math-in-the-middle)
activities.</p>
OpenDreamKit annual meeting Bremen (Germany) 2016-06-27 to 2016-07-012016-06-27T00:00:00+00:00http://opendreamkit.org/2016/06/27/Bremen<p>## Main goals</p>
<p>Annual project meeting, interim review and workshops</p>
<p>## OpenDreamKit implication</p>
<p>JacobsUni (Bremen) hosted this event which
was coorganized by Paris Sud and fully funded by OpenDreamKit.</p>
<p>## Event summary</p>
<p>The beginning of the week was dedicated to
OpenDreamKit’s open and internal meetings, including an interim review with
our Project Officer and three EU Commission reviewers. The rest of
the week was dedicated to joint work sessions on WP4 (User
Interfaces) and WP6 (Data/Knowledge/Software bases, aka
Math-in-the-middle) activities.</p>
<p>## Demographic</p>
<p>21 OpenDreamKit participants together with OpenDreamKit’s
project officer and three reviewers from the EU Commission.</p>
<p>## Results and impact</p>
<p>This meeting was the occasion to build
a common overview of what was achieved during the first ten months,
and plan together work on the upcoming tasks and deliverables. The
project review was enormously helpful to get early feedback
and start preparing for the upcoming review at Month 18
(March 2017).</p>
5th Encuentro Colombiano de Combinatoria Medellin (Colombia), 2016-06-13 to 2016-06-242016-06-01T00:00:00+00:00http://opendreamkit.org/2016/06/01/ECCO<h2 id="main-goals">Main goals</h2>
<p>ECCO is a combinatorics summer school organized every other year in Colombia. It welcomes
students from all over the world of all levels: from undergraduates to postdocs. It is known to be a very
interesting event and to have a great impact for combinatorics in Colombia and South America in general.
The Sage community in combinatorics being very active, it was a great occasion to introduce Sage
to a new generation of researchers.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Viviane Pons was sent by OpenDreamKit to give two Sage interventions
during the school.</p>
<h2 id="event-summary">Event summary</h2>
<p>Each intervention was 2 hours long with around 50 participants each time.
Most participants were using the university computers. 50 USB sticks were bought previous
to the conference and set up with bootable Linux and sage, allowing for a very quick setup
during the two sessions without relying on on-line options. The students worked on some
introduction tutorials and also specifically made tutorials in relation with the on-going
classes.</p>
<h2 id="demographic">Demographic</h2>
<p>66 participants came from South and Central America, 34 from North America
and 30 from Europe.</p>
<h2 id="results-and-impact">Results and impact</h2>
<ul>
<li>
<p>The organizers were very happy that OpenDreamKit would propose to send someone at the school.
They did not have anyone who could carry such an intervention which requires both Sage and
technical skills.</p>
</li>
<li>
<p>It was quite a challenge to get Sage to work in approximatively 10 minutes for 50
computers all together. The solution of the bootable USB sticks has been developed by Thierry
Monteil but is not well known nor well documented. This was an occasion to test this solution
in this particular setup and improve our experience for future events.</p>
</li>
<li>
<p>The bootable USB keys were very successful and many students brought their own sticks to
get a copy of the software. During the sessions, we could also help students install
Sage on their own computers.</p>
</li>
<li>
<p>Many of the students, especially the younger ones from South America, had never used
Sage before. We proposed many different tutorials so that everyone could have something to
work on and we created exercises related to the class content of the two weeks. We received
enthusiastic feedbacks for the sessions.</p>
</li>
<li>
<p>Some of the introduction tutorials of the Sage documentation were translated into
Spanish for the sessions and will eventually be added to Sage.</p>
</li>
<li>
<p>The conference in general was a very rewarding event. It has been growing and successful
for the past ten years with a strong focus on inclusivity and impact. It was a great occasion
to be part of the event and learn from their experience. A blog post from Viviane Pons was
published by the <strong>AMS Blog, On Teaching and learning mathematics</strong>\cite{16PonsECCO}.</p>
</li>
</ul>
<p>\begin{figure}[ht]
\caption*{Participants of ECCO at the Sage sessions}
\includegraphics[scale=0.5]{pictures/ECCO-1.jpg}</p>
<p>\includegraphics[scale=0.5]{pictures/ECCO-2.jpg}
\end{figure}</p>
Sage days 732016-05-07T00:00:00+00:00http://opendreamkit.org/2016/05/07/sage-days-oaxaca<p>Sage days 73 in Oaxaca (Mexico) just ended. Participants intensively used</p>
<ul>
<li><a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a> (Weierstrass function)</li>
<li><a href="https://perso.ens-lyon.fr/nathalie.revol/software.html#MPFI_outline">interval arithmetic</a> (proven floating point computations)</li>
<li><a href="http://www.gap-system.org/">GAP</a> (finite groups representation)</li>
<li><a href="http://jedi.ks.uiuc.edu/~johns/raytracer/">tachyon</a> (ray tracing for animation)</li>
<li><a href="http://sagemath.org">Sage</a> for interacting with all of these…</li>
</ul>
<p>A detailed report is available on <a href="https://wiki.sagemath.org/days73">the dedicated wiki page</a>.</p>
Full-time developer position at TU Kaiserslautern for six months beginning August 20162016-05-02T00:00:00+00:00http://opendreamkit.org/2016/05/02/developer-position3-kaiserslautern<p>We are seeking a full-time developer at TU Kaiserslautern to work with Dr. William Hart on the MPIR <a href="http://mpir.org/">http://mpir.org/</a> bignum library.</p>
<h1 id="location">Location</h1>
<p>The developer will work at TU Kaiserslautern in the city of Kaiserslautern Germany. Kaiserslautern is next to one of the largest contiguous forests in Europe.</p>
<h1 id="mission">Mission</h1>
<p>To help maintain the MPIR library as part of the OpenDreamKit collaboration, in particular to enhance the performance and portability of the library.</p>
<h1 id="activities">Activities</h1>
<p>The successful applicant will help support recent Intel and AMD processors in MPIR by usng an existing superoptimiser on assembly language routines in MPIR to speed them up.</p>
<p>The applicant will also help parallelise the large integer multiplication routine in MPIR by parallelising the existing FFT code (written in C) to speed up multiplication of very large integers.</p>
<p>There is also an oppotunity to help add the double and triple large prime variants to our existing quadratic sieve, for factoring large integers.</p>
<h1 id="skills-requirements">Skills requirements</h1>
<ul>
<li>
<p>x86_64 assembly language experience or a strong willingness to learn</p>
</li>
<li>
<p>Experience in low-level optimisation</p>
</li>
<li>
<p>Fluency in English</p>
</li>
<li>
<p>Must have a Masters degree in Mathematics, Computer Science, Computer Engineering or equivalent.</p>
</li>
<li>
<p>Experience in Open Source development and tooling</p>
</li>
</ul>
<h1 id="context">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<h1 id="expressions-of-interest">Expressions of interest</h1>
<p>Interested candidates should send an email to goodwillhart at googlemail.com for further information about the position.</p>
<h1 id="deadline-for-applications">Deadline for applications</h1>
<p>The deadline for applications for this position is 9am GMT 26th May 2016. Applications should consist of an email to goodwillhart at googlemail.com and decker at mathematik.uni-kl.de with a CV, statement that you wish to apply for the position, a cover letter and a link to some code you have written, preferably on GitHub.</p>
<p>The application process also requires us to do a short interview, which can be conducted by telephone or Skype if candidates are not in Germany at the time.</p>
WP6 Workshop (Bremen) Bremen, Germany, 2016-05-30 to 2016-06-032016-05-01T00:00:00+00:00http://opendreamkit.org/2016/05/01/wp6-workshop<h2 id="main-goals">Main goals</h2>
<p>Work meeting to understand the type systems of GAP and
Sage, and to develop a first interface between MMT, GAPs!, and Sage.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit through Jacobs Uni Bremen was the main
organizer of this event to work on WP6. PS participated remotely.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event featured a talk about the GAP type
system, and many discussions between the researchers in Bremen and
Markus Pfeiffer. We developed a substantial piece of software to
enable GAP to interface with MMT. Meanwhile PS, with the
support of SA and JU, made further progress on
\delivref{component-architecture}{semantic-interface-sage-gap} and on
the export of the math knowledge embedded in Sage.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This workshop was essential to OpenDreamKit WP6,
in particular for \longtaskref{dksbases}{data-design} and
\delivref{dksbases}{design}.</p>
International Workshop on Software Engineering for Science Austin (TX, USA), 2016-05-162016-05-01T00:00:00+00:00http://opendreamkit.org/2016/05/01/se4science<p>## Main goals</p>
<p>Spread recommendations to support better
science in the area of software engineering for computational research.</p>
<p>## OpenDreamKit implication</p>
<p>The work presented has been created with
the upcoming Jupyter OOMMF integration in mind, and is of wider
interest to the OpenDreamKit partners and users. The conference
attendance was paid from a different grant.</p>
<p>## Event summary</p>
<p>Hans Fangohr delivered a talk on Software
Engineering for Computational Science, in particular reviewing
technical and social aspects of a computational science code that
was developed about 10 years ago. The presentation, and associated
publication \cite{16FangohrSE4Science} extracted lessons learned from the past and with the aim
to enable the community to identify potential mistakes sooner. The
presentation and work provides recommendations to enable better
science in the field of computational science and engineering; in
particular focusing on software engineering for computational
science and research codes.</p>
<p>The talk was the keynote presentation of the morning session in the
workshop on Software Engineering for Science (30 minutes).</p>
<p>## Demographic</p>
<p>About 15 people were present, 3 female.</p>
<p>## Results and impact</p>
<p>We reported evidence from the
effectiveness of particular sofware engineering techniques and
provided recommendations for future projects (including user
interface, testing, version control, documentation,
installation).</p>
Sage Days 74: Differential geometry and topology Observatoire de Paris, Meudon, France, 30 May - 2 June 20162016-05-01T00:00:00+00:00http://opendreamkit.org/2016/05/01/SD74<h2 id="main-goals">Main goals</h2>
<p>This workshop was dedicated to the implementation of some
topology and differential geometry in SageMath, partly in connection with the
SageManifolds project http://sagemanifolds.obspm.fr/. 3D visualisation
in the Jupyter notebook was also discussed.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit, via its Orsay and Bordeaux nodes, supported the travel and living expenses of 7 speakers:</p>
<ul>
<li>Marck Bell (U. Illinois, Urbana-Champaign)</li>
<li>Marck Culler (U. Illinois, Chicago)</li>
<li>Nathan Dunfield (U. Illinois, Urbana-Champaign)</li>
<li>Patrick Hooper (City College of New York)</li>
<li>Vincent Delecroix (U. Bordeaux)</li>
<li>Jeremy L. Martin (U. Kansas, Lawrence)</li>
<li>John Palmieri (U. Washington, Seattle)</li>
</ul>
<h2 id="event-summary">Event summary</h2>
<p>Morning sessions were devoted to talks on various
topics relevant to the workshop theme, some of them involving codes that are
not part of SageMath (SnapPy, Flipper, Gyoto).
Afternoon sessions were devoted to working groups and coding sprints.</p>
<h2 id="demographic">Demographic</h2>
<p>26 persons took part in these Sage Days: 5 females and 21 males, originating
from the following countries: France (11), USA (8), Poland (3), Germany (2), Russia (1) and UK (1).</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>41 SageMath tickets have been written or reviewed during the workshop; the list
of them is available at https://trac.sagemath.org/query?keywords=~sd74&or&keywords=~days74
Progresses on the K3D-jupyter visualisation are reported at
https://wiki.sagemath.org/K3D-tools.</p>
Sage Days 73 Oaxaca, Mexico, 04 - 07 May 20162016-05-01T00:00:00+00:00http://opendreamkit.org/2016/05/01/SD73<h2 id="main-goals">Main goals</h2>
<p>This Sage workshop was a satellite of the conference Flat Surfaces and
Dynamics of Moduli Space that happened in Oaxaca May 08-13. The aim was to
introduce participants to SageMath and share code and knowledge.</p>
<p>On the first day, we also had two participants from the University of
Oaxaca.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit, via its Bordeaux node, supported the expenses
of participants. Vincent Delecroix made introductory and advanced talks about
SageMath and Python.</p>
<h2 id="event-summary">Event summary</h2>
<p>The first day was dedicated to a SageMath introduction.
Each day in the afternoon, we had a demonstration from a participant. The rest
of the time was dedicated to programming.</p>
<h2 id="demographic">Demographic</h2>
<p>11 persons took part in these Sage Days: 2 females and 9 males, originating
from the following countries: Mexico (3), Canada (3), France (3), USA (2)</p>
<h2 id="results-and-impact">Results and impact</h2>
<ul>
<li>A step toward the convergence of the IPython and SageMath notebooks with https://trac.sagemath.org/ticket/20562} (\delivref{UI}{ipython-kernel-sage)</li>
<li>A fix in SageMath for a problem discovered Maxime Fortier-Bourque during the workshop (https://trac.sagemath.org/ticket/20566)</li>
<li>Charles Fougeron’s code about Lyapunov exponents gets integrated in https://github.com/videlec/sage-flatsurf.</li>
<li>New visualization tools for geometry of translation surfaces at https://github.com/videlec/flatsurf-package.</li>
<li>And several experimentations by the other participants.</li>
</ul>
PyCon Portland (Oregon), 2016-05-28 to 2016-06-022016-05-01T00:00:00+00:00http://opendreamkit.org/2016/05/01/PyCon<h2 id="main-goals">Main goals</h2>
<p>PyCon is the biggest Python conference in the world. It is the best place to learn
about the python community, open-source tools, new technologies, etc. It is also a good place
to grow a network in the software development community.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Viviane Pons was present at the meeting for the third time in a row,
consolidating her effort to build links between Sage and Python communities. In 2015, she had given
a talk and organized a parallel Sage Days event. It was not possible to do so this year but a
Sage sprint was still maintained.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The conference itself was very instructive as usual in thematics such as:
efficient programming, parallel computing, open-source community building, teaching, inclusivity. It
was a good occasion to discuss with other python programmers and introduce the OpenDreamKit project. The academic
community did not seem as present as it had been in the previous years. In the future, we might want to
target smaller events such as SciPy and EuroScipy.</p>
Pythran aliasing2016-04-17T00:00:00+00:00http://opendreamkit.org/2016/04/17/pythran-aliasing<p>As part of its OpenDreamKit deliverable, the Pythran team has written an in-depth <a href="http://serge-sans-paille.github.io/pythran-stories/identifier-binding-computation.html">article about identifier binding computation within the Pythran compiler</a>.</p>
Sage Days 77: packaging, portability, documentation tools Cernay (France) 2016-04-04 to 2016-04-042016-04-04T00:00:00+00:00http://opendreamkit.org/2016/04/04/SD77<p>## Main goals</p>
<p>This developer meeting was focused on
initiating long term work on OpenDreamKit tasks related to packaging,
portability and documentation tools for SageMath.</p>
<p>## OpenDreamKit implication</p>
<p>This event was organized and funded by
OpenDreamKit (Paris Sud).</p>
<p>## Event summary</p>
<p>An intensive week with some short informal
presentations, and many brainstorms and coding sprints.</p>
<p>## Demographic</p>
<p>9 OpenDreamKit participants from three sites together
with half a dozen other Sage, Sphinx, Guix, Gentoo, and Debian
experts.</p>
<p>## Results and impact</p>
<p>Proper packaging and distribution has
been a recurrent issue for SageMath, and is a major task for OpenDreamKit
(\longtaskref{component-architecture}{mod-packaging}). Major brainstorms occurred during
the week to clarify the needs, isolate the core difficulties, and
explore potential approaches to tackle them. The outcome was posted
on the <a href="https://wiki.sagemath.org/days77/packaging">Sage Wiki</a>,
to be shared and further edited by the community. This fostered
tighter collaboration between the packaging efforts for various
Linux distribution, and triggered major progress on the Debian
packaging side.</p>
<p>Similar brainstorms and coding sprints occurred around tasks
\longtaskref{component-architecture}{portability},
\longtaskref{UI}{sage-sphinx}, \longtaskref{UI}{structdocs}</p>
<p>Altogether
<a href="https://trac.sagemath.org/query?keywords=~days77&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority">20 Sage tickets</a> were actively worked on during the week.</p>
WP6 paper art CICM 20162016-03-21T00:00:00+00:00http://opendreamkit.org/2016/03/21/WP6-paper<p>We have just submitted a paper to the
<a href="http://www.cicm-conference.org/2016/cicm.php?event=surveys&menu=general">Projects&Surveys track of CICM 2016</a>. <a href="http://arxiv.org/abs/1603.06424">The
paper</a>
(<a href="https://github.com/OpenDreamKit/OpenDreamKit/blob/master/publications.bib">bibTeX</a>) presents the ODK project to the CICM community and then presents the
“Math-in-the-Middle” approach to interoperability we have developed in the
<a href="http://opendreamkit.org/2015/12/08/WP6StAndrewsMeeting/">St. Andrews WP6 workshop</a>.</p>
School in Lebanon University Lebanese University, HADAT -- 05-11 Mars 20162016-03-01T00:00:00+00:00http://opendreamkit.org/2016/03/01/HADAT<h2 id="main-goals">Main goals</h2>
<p>The goal was to organise a class on Polya theory and then
to deliver training sessions on Sage Math to implement Polya Theory.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>A. Boussicault was funded by OpenDreamKit for the travel.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event was one-week long, with a 4 hour-long intervention per day</p>
<h2 id="demographic">Demographic</h2>
<p>7 Students in their second year of Master’s degree in mathematics at the Lebanese University.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This was the occasion of a first contact with Amine Sahili, the students’ professor.</p>
Conference Combinatoire, Algèbre et Théorie des Nombres Monastir, Tunisia -- 24-28 Mars 20162016-03-01T00:00:00+00:00http://opendreamkit.org/2016/03/01/CATN<h2 id="main-goals">Main goals</h2>
<p>CTAN 2016 was a mathematical conference that took place in Tunisia.
The first goal of OpenDreamKit was to make an initiation on SageMath.
The second goal was to create a Sage developer team in Tunisia.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>A. Boussicault was sent by OpenDreamKit to deliver training sessions on
SageMath. OpenDreamKit paid the travel and accommodation of A. Boussicault.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event wasone-week long, with a 2-hour long intervention per day.
Trainings were given on participants’ computers.
We worked with Sage by using a Debian live USB stick.</p>
<p>A Linux installfest on participants’ laptops took place every evening.
The evening sessions were beginning at 20:30 and ended at midnight.</p>
<p>The session was conducted by A. Boussicault (University of Bordeaux) and
Imad Eddine Bousbaa (University of USTHB of Algeria), who was met during a
previous conference in Algeria (\ref{event-2015DIMACOS}).
The purpose was to present and make mathematical calculus with Sage.
We also gave classes on Python and on Combinatorics.</p>
<p>During that event, we discussed with CTAN organizers and some researchers
to organize some important mathematical events in Tunisia, Algeria, Morocco and
Lebanon.
Many researchers were present in the discussion. For example, there were
Professor H. Belbachir (University USTHB in Alger),
Professor I. Boudabbous (University of Sfax in Tunisia)
Professor O. Khadir (University Hassan II of Casablanca),
Professor M. Pouzet (University of Lyon1),
professor H. Kheddouci, (University of Lyon1).
We decided to organize, in the next year, an event with the mathematical
conference CTAN followed by mathematical schools and some Sage Days.
In the school, we could give some Sage classes and we could implement ideas
and mathematical tools during the Sage Days.</p>
<h2 id="demographic">Demographic</h2>
<p>30 participants. CTAN being an international conference, participants
came from Lebanon, Algeria, Tunisia, Morocco, etc.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The will to organize an event with a mathematical conference
followed by mathematical schools, and Sage Days in Morocco or in Algeria.</p>
<p>We made some Tunisian contacts to create a developer team in University of
Sfax.</p>
3D Visualization of simulation data with x3dom2016-02-16T00:00:00+00:00http://opendreamkit.org/2016/02/16/simulation-3d-data-viz<p>Logilab released a x3dom based software to visualize 3D simulation
mesh-based data in the browser.</p>
<p>A live demo page can be found
<a href="http://yuanxiangfranck.github.io/x3dom-plugins-API/">here</a>, a more in
depth article on <a href="https://www.logilab.org/blogentry/4386317">Logilab’s
blog</a> and the source code
on <a href="https://github.com/YuanxiangFranck/x3dom-plugins-API">this github
repository</a>.</p>
European Commission consultation on “mathematics in H2020”2016-02-11T00:00:00+00:00http://opendreamkit.org/2016/02/11/ec-consultation-h2020-mathematics<p>The European Commission has just launched an
<a href="https://ec.europa.eu/futurium/en/content/online-consultation-mathematics-excellence-science-horizon2020">open consultation on “mathematics in H2020”</a>.</p>
<p>``The objective is to collect a large quantity of concrete proposals
that can subsequently be used to inform and elaborate calls (in the
FET framework, for 2018-19-20) with explicit mathematical sciences
content. We urge you to contribute massively to this consultation
(deadline March 30th): We are of the opinion that mathematics should
not be reserved to ERC, but should be omnipresent in the future work
programmes!’’</p>
St. Andrews Workshop explored "Math-in-the-Middle" Paradigm for Integration2016-01-26T00:00:00+00:00http://opendreamkit.org/2016/01/26/kr-workshop<p>The OpenDreamKit Team held a workshop on knowledge representation to synchronize views and
start coding for Work Package WP6
… <a href="http://opendreamkit.org/2015/12/08/WP6StAndrewsMeeting/">details/program</a>.</p>
<p>The 15 participants explored the
<a href="/meetings/2016-01-25-DKS/Kohlhase_slides.pdf">Math-in-the-Middle Architecture</a>
(originally introduced as Knowledge-First-Strategy) introduced by the WP6 leader and they
started coding towards realizing it.</p>
<p>This information architecture uses a central mathematical ontology represented in
OMDoc/MMT as a central vocabulary for specifying system behaviors and capabilities, while
documenting the maths involved to users.</p>
WP6 Workshop: Knowledge representation in mathematical software and databases St Andrews (UK), 2016-01-25 to 2016-06-272016-01-25T00:00:00+00:00http://opendreamkit.org/2016/01/25/DKS<p>## Main goals</p>
<p>Semiannual OpenDreamKit project meeting joined with a WP6
``Data/Knowledge/Software bases’’ kickoff workshop dedicated to the exploration of how
mathematical knowledge could be better represented and exploited within systems and for
communicating between systems.</p>
<p>## OpenDreamKit implication</p>
<p>St Andrews hosted this event which was organized by Paris Sud
(project meeting) and Jacobs Uni (WP6 workshop).</p>
<p>## Event summary</p>
<p>Tuesday morning was dedicated to OpenDreamKit’s semiannual <a href="http://opendreamkit.org/meetings/2016-01-25-DKS/SteeringCommittee/minutes/">steering committee
meeting</a>,
including progress reports from all sites. A couple additional OpenDreamKit participants
joined physically or remotely for this meeting.</p>
<p>The rest of the week was dedicated to Work Package 6 activities. To bootstrap the
discussions, we started with a presentation by Michael Kohlhase of his ideas for a
Knowledge First strategy\footnote{This eventually became the <code class="language-plaintext highlighter-rouge">Math-in-the-Middle''
paradigm~\cite{DehKohKon:iop16}}, followed by presentations of how knowledge is
represented in various components of OpenDreamKit (FindStat: Viviane Pons, LMFDB: Paul-Olivier
Dehaye, GAP: Markus Pfeiffer, SageMath: Nicolas M. Thiéry), in preparation for
\longdelivref{dksbases}{design}. There was also a brief presentation of a
proof-of-concept Knowledge-aware Sage-GAP interface developed the week before at the
GAP/Sage Days. Current practices where discussed as well: commonalities in</code>dumping
math data on the web’’ process (FindStat, LMFDB,…). Then came a tutorial presentation of
MMT to explore how knowledge representation in MMT could serve as a generic knowledge
backbone for integrating the various systems. After these warm up activities, we moved
on to brainstorms and joint code sprints, developing proof-of-concepts formalizations of
the knowledge in the various components and exploring applications: detecting bugs and
inconsistencies in code and data, generating more complete documentation, supporting
generic handle interfaces, …</p>
<p>A fine grained report of the activities is available
<a href="http://opendreamkit.org/meetings/2016-01-25-DKS/report/">here</a>.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The Math-in-the-Middle approach was born
at this workshop. This led to a joint paper at the CICM
conference~\cite{DehKohKon:iop16} and kicked off and fueled much of
the activity on WP6 since then, in particular around
\longtaskref{dksbases}{data-design}, \delivref{dksbases}{design},
\delivref{dksbases}{dkstheories}, and
\delivref{component-architecture}{semantic-interface-sage-gap}.</p>
<p>\begin{figure}[ht]
\caption*{WP6 Workshop: Knowledge representation in mathematical software and databases}
\includegraphics[scale=0.5]{pictures/2016-01-25-DKS-group-picture.jpg}
\end{figure}</p>
OOMMF Python interface and Jupyter integration2016-01-12T00:00:00+00:00http://opendreamkit.org/2016/01/12/joommf-talk-MMM2016<p><a href="http://joommf.github.io/assets/2016-01-12-MMM2016-AD02-Jupyter-OOMMF.pdf">Pdf slides of Talk</a></p>
First Joint GAP-SageMath Days St Andrews (UK), 2016-01-18 to 2016-01-222016-01-08T00:00:00+00:00http://opendreamkit.org/2016/01/08/GAP-Sage-days<h2 id="main-goals">Main goals</h2>
<p>Both GAP and SageMath systems have traditions of regular developer meetings, where those
interested in these systems, from newcomers to contributors, are gathering together for
collaborative code writing, sharing best practices, advertising recent new features and
improvements, and discussing further developments. You can find the list of previous GAP
days at http://gapdays.de/} and of SageMath days at \url{https://wiki.sagemath.org/Workshops.</p>
<p>Following these traditions, it was decided to organise the 1st Joint GAP-SageMath Days, with the
focus on improving GAP-SageMath integration and interaction between these systems and between
their developers.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The 1st Joint GAP-SageMath Days were mainly supported by CoDiMa – Collaborative Computational
Project (CCP) in the area of Computational Discrete Mathematics (EPSRC grant EP/M022641/1,
http://www.codima.ac.uk/). It was immediately followed by the WP7 Workshop ``Knowledge
representation in mathematical software and databases’’ on January 25th-27th, 2016, therefore
OpenDreamKit participants involved in GAP and/or SageMath development could conveniently attend
both events. Accommodation, subsistence and travel expenses of partners from UPSud and UVSQ
were paid by the OpenDreamKit project, and those of partners from UNIKL,UOXF were reimbursed
by the CoDiMa project.</p>
<h2 id="event-summary">Event summary</h2>
<p>A typical day of the workshop had one or two introductory talks to facilitate subsequent
discussions and coding sprints, in particular:</p>
<ul>
<li><strong>Contributing to Sage</strong> by Nicolas M. Thiéry and Volker Braun</li>
<li>**Contributing to GAP ** by Max Horn, Olexandr Konovalov, and Markus Pfeiffer</li>
<li><strong>libGAP</strong> by Volker Braun</li>
<li><strong>GAP in the cloud</strong> by Markus Pfeiffer</li>
</ul>
<p>Other topics included, among others,
further integration of HPC-GAP into GAP;
working on the semantic-aware SageMath interface to GAP;
improving the installation of GAP in SageMath and in SageMath cloud;
creating and working with Docker containers;
development of GAP and SageMath teaching materials for Software Carpentry,
etc.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop was very productive. Only to the main GAP repository
(https://github.com/gap-system/gap/) there were 51 new pull
requests submitted, just 8 of which are still open; in addition,
28 new issues were created (9 of them are closed by now), and
there was also progress achieved with GAP packages developed
elsewhere; the work on converging GAP and HPC-GAP; discussing
development workflows, etc. It helped to both GAP and SageMath
teams to get further insights into each other’s systems and was
particularly beneficial to WP3 (component architecture), WP4
(user interfaces) and WP5 (high-performance computing).</p>
pythran 0.7.42016-01-07T00:00:00+00:00http://opendreamkit.org/2016/01/07/pythran-release-0.7.4<p>Pythran 0.7.4 has been released. It implements several prerequisites for Sage/Cython integration:</p>
<ul>
<li>less binary dependencies (dependency on <code class="language-plaintext highlighter-rouge">boost-python</code> and <code class="language-plaintext highlighter-rouge">cmake</code> removed)</li>
<li>easier installation (better dependency description)</li>
<li>partial Python3 port (only the cpp to native code part)</li>
<li>Windows portability (based on cygwin)</li>
</ul>
<p>And as usual, several performance improvements, more numpy API support, and
hopefully more bugs solved than the number of newly introduced ones!</p>
<ul>
<li><a href="http://pythonhosted.org/pythran">Read the doc!</a></li>
<li><a href="https://github.com/serge-sans-paille/pythran">Contribute!</a></li>
<li><a href="https://pypi.python.org/pypi/pythran">Download!</a></li>
</ul>
Full-time two year Postdoctoral Fellow position2016-01-07T00:00:00+00:00http://opendreamkit.org/2016/01/07/postdoc-position-simula<p><strong>This position has been filled</strong> by Dr. Vidar Tonaas Fauske</p>
<p>We are seeking a full-time postdoctoral fellow at
<a href="http://www.simula.no">Simula Research Laboratory</a> to work with
Dr. Min Ragan-Kelley and Dr. Martin S. Alnæs
on <a href="http://www.jupyter.org">Jupyter</a> related contributions to the OpenDreamKit project.</p>
<h1 id="deadline">Deadline</h1>
<p>Deadline for applications: Jan 30th 2016.</p>
<h1 id="location">Location</h1>
<p>The candidate will work at Simula Research Laboratory in
offices that are a 15 minutes drive or boat commute from
the centre of Oslo, Norway.</p>
<h1 id="mission">Mission</h1>
<p>To work as part of the OpenDreamKit collaboration on Work Package 4.
To design and develop extensions to <a href="http://jupyter.org/">IPython/Jupyter Notebook</a> in
collaboration with the OpenDreamKit partners and Jupyter team. Topics
include improved collaboration support, reproducible notebooks, test
automation, and 3D visualization. Work will be done in both Python
and JavaScript.</p>
<h1 id="your-profile">Your Profile</h1>
<p>Interested applicants should:</p>
<ul>
<li>
<p>Hold a Ph.D. degree or equivalent in Computer Science, Scientific Computing, or related fields.</p>
</li>
<li>
<p>Have solid programming skills and experience writing non-trivial programs in Python and/or JavaScript.</p>
</li>
<li>
<p>Demonstrate an excellent level of spoken and written English.</p>
</li>
<li>
<p>Possess good interpersonal skills and show willingness to work as part of an international team.</p>
</li>
<li>
<p>Be passionate about open science and open source.</p>
</li>
</ul>
<h1 id="context">Context</h1>
<p>The position will be funded by
<a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<h1 id="expressions-of-interest">Expressions of interest</h1>
<p>Please apply through <a href="https://www.simula.no/about/available-jobs">the Simula webpages</a>.</p>
<p>See also <a href="http://m.finn.no/job/fulltime/ad.html?finnkode=69302480&orgId=274233842&ref=fas">more details about the application</a>.</p>
<p>For requests of further information about the position, send an email to martinal@simula.no.</p>
13th Joint Magnetism and Magnetic Materials (MMM) - Intermag Conference San Diego (CA, USA), 2016-01-11 to 2016-01-152016-01-01T00:00:00+00:00http://opendreamkit.org/2016/01/01/MMM<p>## Main goals</p>
<p>The main goals of presenting the project at
this important international Joint Magnetism and Magnetic Materials
and Intermag conference was to introduce the Micromagnetic Virtual
Reasearch Environment (VRE) to our target user audience - the
micromagnetic scientific community.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>The OpenDreamKit project has sent the
speaker (Hans Fangohr), and paid expenses for the trip.</p>
<h2 id="event-summary">Event summary</h2>
<p>Hans Fangohr submitted a talk \cite{16FangohrOOMMF} that was peer
reviewed and accepted for presentation. In the talk, he outlined the
vision for the project and invited feedback from the community.</p>
<p>The OpenDreamKit project for computational micromagnetics was discussed with
various attendees informally throughout the conference.</p>
<p>In addition, we organised a meeting with the main developers of the
OOMMF micromagnetic simulation code, Dr M. Donahue and
Dr D. Porter, in order to discuss the project’s vision, our plans for
interfaces to get early feedback.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>We announced the project and its website
to the community and encouraged input to extend our vision, to make
sure the tool we develop can be as practical and efficient for as
large parts of the community as possible.</p>
Atelier PARI/GP 2016 Grenoble (FR), 2016-01-11 to 2016-01-252016-01-01T00:00:00+00:00http://opendreamkit.org/2016/01/01/Atelier-PARI-GP<h2 id="main-goals">Main goals</h2>
<p>The PARI/GP Ateliers were established in 2012 as a yearly meeting
between developers and users of the PARI/GP system.</p>
<p>The main goals are advertising new features and improvements,
discussing further developments, sharing best practices, and collaborative
code writing (hacking sessions, doc reviews, bug-squashing parties).</p>
<p>You can find the list of previous PARI Ateliers at
http://pari.math.u-bordeaux.fr/ateliers.html</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit participants: B. Allombert, K. Belabas, J. Demeyer, J.-P. Flori,
L. de Feo, as well as Aurel Page from the Warwick group (LMFDB).</p>
<p>OpenDreamKit provided the main funding source for the workshop (accommodation,
subsistence and travel expenses), for about 15k Euro. ERC Starting Grant
ANTICS, and the LabEx PERSYVAL-Lab co-funded the event.</p>
<h2 id="event-summary">Event summary</h2>
<p>The 6th Atelier PARI/GP took place in Grenoble (France) from january
11th to 15th.</p>
<p>There were 36 registered participants from 16 different institutions
(no registration fees).</p>
<p>A typical day of the workshop had introductory talks and tutorials
in the morning; afternoons allowed ample time for hacking sessions,
discussions and training.</p>
<p>The Atelier featured 10 morning talks on</p>
<ul>
<li>
<p>mathematical topics and implementation projects : modular forms,
L-functions, polylogs \& multizeta values,</p>
</li>
<li>
<p>packages and interfaces : PARI Jupyter notebook, a number field database,
an elliptic curve library for cryptography, CADO-NFS, GIAC/XCAS,
parallel programming with GP2C.</p>
</li>
</ul>
<p>Slides and videos for all talks are available at
https://www.youtube.com/playlist?list=PL0E0n75oNCDnWuydCHepxxSRc4UbtQQ</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The workshop was very productive and was particularly beneficial to WP4 (user
interfaces) and WP5 (high-performance computing):</p>
<ul>
<li>
<p>it was a major boost to PARI/GP development; feebackd received allowed
the release of PARI/GP-2.8 in august 2016, a major release after two years of
development; (\longtaskref{dissem}{devel-workshops})</p>
</li>
<li>
<p>issues related to the new PARI Jupyter notebook (\delivref{UI}{ipython-kernels-basic}, \delivref{UI}{ipython-kernels}) and
Sage/PARI interaction were ironed out during the meeting; discussions related
to PARI parallelisation engine (\delivref{hpc}{pari-hpc1})</p>
</li>
<li>
<p>the PARI developers learnt about technologies and created resources for
online GP deployment during the meeting using the <strong>emscripten</strong> compiler,
see e.g.~http://pari.math.u-bordeaux.fr/gp.html.</p>
</li>
</ul>
Knowledge representation in mathematical software and databases & Steering Committee2015-12-08T00:00:00+00:00http://opendreamkit.org/2015/12/08/WP6StAndrewsMeeting<p>This workshop is taking place at the University of St Andrews, UK. It will be preceded by the <a href="http://gapdays.de/gap-sage-days2016/">First Joint GAP-Sage Days</a> (GAP Coding sprint on Jan 13-16, 2016 & GAP-SageMath Workshop on Jan 18-22, 2016) which are taking place in St Andrews as well, but require separate registration.</p>
<p>This OpenDreamKit consortium workshop is a part of workpackage on Data/Knowledge/Software-Bases (also known as WP6), and its theme is “Knowledge representation in mathematical software and databases”.</p>
<h1 id="workshop-program"><a href="/meetings/2016-01-25-DKS/program">Workshop Program</a></h1>
<p><a href="/meetings/2016-01-25-DKS/group_picture.jpg">Group Picture</a></p>
<h1 id="steering-committee">Steering Committee</h1>
<ul>
<li><a href="/meetings/2016-01-25-DKS/SteeringCommittee/agenda">Agenda</a></li>
<li><a href="/meetings/2016-01-25-DKS/SteeringCommittee/SC-StAndrews_presentation_final.pdf">Presentation slides</a></li>
<li><a href="/meetings/2016-01-25-DKS/SteeringCommittee/minutes">Minutes</a> and yearly progress reports for all sites</li>
</ul>
<h1 id="registration-for-the-workshop">Registration for the workshop</h1>
<p>Please register via Eventbrite using <a href="https://www.eventbrite.com/e/opendreamkit-wp6-workshop-registration-19907319328">this link</a>.</p>
<h1 id="travel">Travel</h1>
<p>On Monday and Wednesday the workshop will take place in the Gateway Building (see <a href="http://www.openstreetmap.org/way/27225074#map=19/56.34119/-2.80943">OpenStreetMap</a>), located
in the vicinity of the <a href="http://www.cs.st-andrews.ac.uk/">School of Computer Science</a> (see <a href="http://www.openstreetmap.org/way/155223867#map=19/56.34030/-2.80871">OpenStreetMap</a>).</p>
<p>On Tuesday the workshop will take place in the <a href="http://byretheatre.com/">Byre Theatre</a>
(see <a href="http://www.openstreetmap.org/way/315991170#map=19/56.33891/-2.79156">OpenStreetMap</a>).</p>
<p>For further travel information about getting to St Andrews by air, rail or road, see <a href="http://www.st-andrews.ac.uk/about/visiting/">Visiting St Andrews</a> page.</p>
<p>UPDATE (2015-12-23): The <a href="https://www.forthroadbridge.org">Forth Road Bridge</a> has been
reopened for all traffic except Heavy Goods Vehicles. In particular all bus services
to St Andrews will run as normal again.</p>
<p>UPDATE: The <a href="https://www.forthroadbridge.org/">Forth Road Bridge</a> is currently closed
<a href="http://www.bbc.co.uk/news/uk-scotland-35001277">at least until new year</a>.
If you travel via Edinburgh, we recommend to take a train to Leuchars, but please
allow extra time for your journey as trains are expected to be busier than usually.
See <a href="http://www.scotrail.co.uk/frb">this ScotRail page</a> for further updates.
At the moment it is anticipated that the bridge
<a href="http://www.bbc.co.uk/news/uk-scotland-scotland-politics-35096668">will reopen before the workshop</a>.</p>
<h1 id="accommodation">Accommodation</h1>
<p>One of the options is to try the brand new
<a href="http://www.premierinn.com/en/hotel/STALAR/st-andrews">St Andrews Premier Inn Hotel</a>
just opened in September 2015. At the moment it offers rooms at £39 per night in
Premier Saver tariff (Pre-pay, no amends or refunds) and at £49-£55 per night in
Premier Flexible tariff (Amend up to 1pm on arrival days). This hotel is located
in the southern part of St Andrews and is in 25 minutes walk from the workshop venue.
If you would like to stay there, please make your own arrangements for the booking.</p>
<p>Alternatively, you may ask the organisers to find a place in some B&B. Prices may
vary between different B&B. For example, we have offers for single rooms with
shared downstairs shower facilities at £35 per night; single en suite at £45 per
night; double/twin room at £85 per night (£65 per night for single occupancy).
Please contact local organisers if you would like to use this option.</p>
<h1 id="contact-details">Contact details</h1>
<p>For more information please contact:</p>
<ul>
<li>Michael Kohlhase (m.kohlhase@jacobs-university.de) for science related matters</li>
<li>Benoît Pilorget (benoit.pilorget@lri.fr)</li>
<li>Olexandr Konovalov (alexander.konovalov@st-andrews.ac.uk) - local organiser</li>
</ul>
<h1 id="participants">Participants</h1>
<ul>
<li>Olexandr Konovalov (Local organiser - St Andrews)</li>
<li>Michael Kohlhase (WP6 leader - JacobsUni), steering committee member</li>
<li>Nicolas Thiéry (Project coordinator - UPSud), steering committee member</li>
<li>Benoît Pilorget (Project Manager - UPSud), steering committee member</li>
<li>Viviane Pons (UPSud), steering committee member</li>
<li>Ursula Martin (UOXF), replacing D. Pasechnik to the steering committee meeting</li>
<li>John Cremona, (UWarwick), steering committee member</li>
<li>Luca de Feo (UVSQ), steering committee member</li>
<li>Paul-Olivier Dehaye (UZH), steering committee member</li>
<li>Steve Linton (StAndrews), steering committee member</li>
<li>Dennis Müller (JacobsUni)</li>
<li>Florian Rabe (JacobsUni)</li>
<li>Mihnea Iancu (JacobsUni)</li>
<li>Markus Pfeiffer (StAndrews)</li>
<li>Tom Wiesing (JacobsUni)</li>
<li>Samuel Lelièvre (UPSud)</li>
</ul>
HPC en combinatoire, application du vol de tâches2015-12-08T00:00:00+00:00http://opendreamkit.org/2015/12/08/LaMHA<p><a href="https://github.com/OpenDreamKit/OpenDreamKit/raw/master/WP5/T5.6/HPC-Combi.pdf">PDF slides</a></p>
Retour d'expérience: licences pour les logiciels de maths pures2015-11-17T00:00:00+00:00http://opendreamkit.org/2015/11/17/LOOPS-Licences<section data-markdown="" data-separator="^---\n" data-separator-vertical="^--\n">
# Retour d'expérience: licences pour les logiciels de maths pures
Nicolas M. Thiéry
17 novembre 2015
Journée LOOPS: «Je code, je diffuse, oui mais comment ?»
---
# Logiciels pour les maths pures
- Algèbre
- Théorie des nombres
- Géométrie
- Calcul symbolique
- Mathématiques discrètes
- ...
--
## Des logiciels, pour quoi faire?
- Exploration informatique
- Vérification de conjectures
- Preuves mécanisées
- Assistants de preuves & preuves certifiées
- Travail collaboratif
- Enseignement
- Applications
--
## Micro historique
### 50's-80's
- Les mathématiciens écrivent du code ad-hoc pour leur recherche
- 1976: démonstration mécanisée du théorème des quatre couleurs
--
### 80's - 90's
- Systèmes généralistes commerciaux:
Maple, Mathematica, ...
- Systèmes libres spécialisés:
Maxima, GAP, PARI/GP, Singular, Macaulay, ...
- OVNIS:
Axiom, MuPAD, ...
### Début des années 2000
La frustration gagne la communauté
--
## 2005-2015: SageMath
- Une distribution des logiciels libres spécialisés
- Une bibliothèque Python:
- 1.5 M lignes code/doc/tests
- Interfaces et intégration
- Fonctionalités natives
- Un bloc-note :
- Source d'inspiration pour IPython/Jupyter
- En cours de migration vers celui-ci
- Un Environnement Virtuel de Recherche ([SageMathCloud](cloud.sagemath.org))
- 300 contributeurs
---
# Étude de cas: Sage-Combinat
## Constat dans mon domaine
- 95% des efforts de développement sont génériques
- Opportunité de mutualisation
## Mission de Sage-Combinat (2000-)
«Améliorer MuPAD/Sage comme boîte à outils extensible pour
l'exploration informatique en combinatoire algébrique, en fédérant et
mutualisant les efforts de développements des chercheurs»
http://combinat.sagemath.org
--
## Quelques éléments de stratégie
- Développé par des chercheurs pour des chercheurs
*Avec un usage plus large en vue*
- Développement décentralisé et international
*Garantie d'indépendance vis-à-vis des tutelles*
- Chaque ligne de code justifiée par un projet de recherche
Avec une vision à long terme (développement agile)
- Coeur du développement par des permanents
*Les doctorants se concentrent sur leurs propres besoins*
- Concepts et méthodologie de développement modernes
--
## Choix de la licence
- Licence libre:
- Choix éthique et pragmatique
- Mutualiser et diffuser
- Se protéger des tutelles
- MuPAD-Combinat: LGPL
- Sage-Combinat: GPL
---
# Licences dans SageMath
- sagemath: GPL
- Paquets standards: GPL-Compatible
- Paquets optionnels: toutes licenses
- Interfaces avec des logiciels privateurs
--
## Quelques problèmes rencontrés
- Logiciels sans licence
- Logiciels sous licence non standard:
- GAP3: moralement libre, mais développeurs dispersés voire décédés
- Nauty: interdit pour les militaires; donc non libre
- sagemathCloud: libérer ou pas le code?
--
# Évolution récente
- Reconnaissance des valeurs éthiques et pragmatiques du libre, par
les tutelles et les financeurs
- Projets Européens H2020 e-Infrastructure:
quasi-obligation de données, code, et publications libres
- Exemple: [OpenDreamKit](http://opendreamkit.org/)
--
# Résumé
## Spécificités du contexte
- Objectif principal:
mutualisation d'efforts de développements
- Code à forte composante métier
- Tradition de partage
- Opinions éthiques fortes
## Choix de licence
- Libre; typiquement GPL
</section>
Finite Simple Groups: Thirty Years of the Atlas and Beyond Princeton (US California), 2015-11-02 to 2015-11-052015-11-01T00:00:00+00:00http://opendreamkit.org/2015/11/01/thirty-years-atlas<h2 id="main-goals">Main goals</h2>
<p>Gather contributors to and users of the ``Atlas of Finite
Simple Groups’’ and other mathematical databases to learn about past, presence,
and future.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>OpenDreamKit was not the main organizer of this event, it was used to fund
one project member (Markus Pfeiffer).</p>
<h2 id="event-summary">Event summary</h2>
<p>The event featured talks by high-profile mathematicians,
such as John H. Conway, John Thompson, Michael Aschbacher, and many more.</p>
<p>Discussion sessions highlighted the need for mathematical knowledge
stored in databases. Some major examples that were discussed are</p>
<ul>
<li>The ``ATLAS of Finite Group Representations - Version 3’’
http://brauer.maths.qmul.ac.uk/Atlas/v3/</li>
<li>The ``Online Encyclopedia of Integer Sequences (OEIS)’’
http://oeis.org</li>
<li>The ``Modular forms and L-functions database (LMFDB)’’
http://lmfdb.org</li>
<li>The ``Small Groups Database’’ (small)</li>
</ul>
<p>Most of these databases share common issues such as <strong>reliability</strong> of the
data, the <strong>reliability</strong> and <strong>longevity</strong> of the storage,
<strong>maintenance</strong>, and <strong>managing contributions</strong>.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>The attendance of this conference shed light on how some mathematicians view
mathematical databases, and what issues they see. This is an important
contribution for WP6. It also contributed to the attendee’s understanding of the
needs of our potential users (WP4).</p>
<ul>
<li>\longtaskref{dksbases}{data-assessment} Survey of existing DKS bases, Formulation of Requirements</li>
<li>\longtaskref{dksbases}{mws} Math Search Engine</li>
</ul>
<p>Further discussions with GAP users and developers about HPC-GAP were a
side-effect of the attendance of this event.</p>
Sage Days 70 Berkeley (US California), 2015-11-08 to 2015-11-142015-11-01T00:00:00+00:00http://opendreamkit.org/2015/11/01/SD70<h2 id="main-goals">Main goals</h2>
<p>Gather developers from Sage, SageMathCloud and Jupyter together to learn the
inner machineries of the different projects and code together towards common goals.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>This event was coorganized by OpenDreamKit which
cofunded the participation of two OpenDreamKit members and another European
associate.</p>
<h2 id="event-summary">Event summary</h2>
<p>The event featured many interesting talks on the inner mechanics of
both SageMathCloud and Jupyter, in particular:</p>
<ul>
<li><a href="https://youtu.be/GOuy07Kift4">How to contribute to SageMathCloud</a> by William Stein</li>
<li><strong>The PARI Jupyter kernel</strong> by Jeroen Demeyer</li>
<li><strong>Jupyter Notebook development</strong> by Jason Grout.</li>
</ul>
<p>Lots of time was devoted to projects and code such as: installing a development version of SageMathCloud,
following tutorials on SageMathCloud development, working toward the integration of the Jupyter notebook
in Sage.</p>
<p>Furthermore a Jupyter interface for HPC-GAP was developed, and the Jupyter
interface for GAP was improved. A talk <strong>The current status of (HPC-)GAP</strong>
was contributed.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This workshop was essential to some OpenDreamKit planned tasks. This was especially related to WP3 and WP4. Here are some tasks that
were started during the Sage Days:</p>
<ul>
<li>
<p>\longtaskref{component-architecture}{extract-smc} Document and modularize SageMathCloud’s codebase. This task was started during the workshop using the
knowledge of the main developer of SageMathClod, William Stein.</p>
</li>
<li>
<p>\longtaskref{UI}{ipython-kernels} Uniform notebook interface for all interactive components. This is a major task of WP4. This workshop
was an occasion to share first hand information between Sage, GAP, and Jupyter developers.</p>
</li>
</ul>
<p>The knowledge we gathered during presentations was relevant to all tasks including notebook interfaces and cloud
systems.</p>
Conference-school on Discrete Mathematics and Computer Science 2015 University of Sidi Bel Abbès, Algeria -- 15-19 November 20152015-11-01T00:00:00+00:00http://opendreamkit.org/2015/11/01/DIMACOS<h2 id="main-goals">Main goals</h2>
<p>DIMACOS 2015 is a mathematical conference that took place in
Algeria. The first goal of OpenDreamKit was to make an initiation on SageMath.
The second goal was to create a team of sage developers in Algeria.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>A. Boussicault was sent by OpenDreamKit to deliver
training sessions on
Sage Math. OpenDreamKit paid the travel and accommodation of A. Boussicault.</p>
<h2 id="event-summary">Event summary</h2>
<p>This was a one-week event, with a two hour session per day.
The sessions were conducted and prepared by
an Algerian researcher (Professor A. Belahcene).
A Linux installfest on laptops took place every evening.
The evening sessions were beginning at 20h30 and ended at midnight.</p>
<p>The sessions were presented by A. Boussicault (University of Bordeaux) and
Z. Chemli (University of Paris-Est). The purpose was to present and make
mathematical calculus with Sage. The sessions covered also background
in Python and Combinatorics.</p>
<p>During that event, we discussed with professor H. Belbachir
(University USTHB in Alger) and professor I. Boudabbous (University of
SFAX in Tunisia) to plan another Sage event in the conference
“Combinatoire, Algèbre et Théorie des Nombres” in Monastir - Tunisia.</p>
<h2 id="demographic">Demographic</h2>
<p>30 participants were present at the sessions.
DIMACOS being an international conference, the people present came from Lebanon, Algeria, Tunisia, Morocco, etc.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This event allowed us to work with Imad Eddine Bousbaa, a PhD student.
He helped us during the Sage sessions.
It was the starting point of a collaboration that allowed us to recruit him in
the OpenDreamKit project.</p>
<p>His recruitment is part of the will to build a team of Sage developers in Algeria.</p>
<p>We could use this event to prepare the next conference : “Combinatoire, Algèbre et Théorie des Nombres”
in Tunisia (event \ref{event-2016CATN}).</p>
OpenDreamKit: bref retour d'expérience sur la soumission d'un projet H20202015-10-01T00:00:00+00:00http://opendreamkit.org/2015/10/01/systematic-gt-ll<section data-markdown="" data-separator="^---\n" data-separator-vertical="^--\n">
# OpenDreamKit
## Bref retour d'expérience sur la soumission d'un projet H2020
Nicolas M. Thiéry
1er octobre 2015
Groupe de travail [Logiciel Libre](http://www.systematic-paris-region.org/fr/logiciel-libre), pôle de compétitivité [Systématic](http://www.systematic-paris-region.org/)
---
# OpenDreamKit (2015-2019)
***Open Digital Research Environment Toolkit***
***for the Advancement of Mathematics***
[OpenDreamKit.org](OpenDreamKit.org)
- [Horizon 2020](https://ec.europa.eu/programmes/horizon2020/)
**Work Programme**: [European Research Infrastructures](https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures)
**Appel**: Environnements Virtuels de Recherche
- **Budget**: 7.6 M€
- **Consortium**: [15 sites, 50 participants](http://opendreamkit.org/partners)
Dont: Paris Sud, UVSQ, Logilab
En collaboration avec la communauté internationale!
--
# OpenDreamKit (2015-2019)
***Open Digital Research Environment Toolkit***
***for the Advancement of Mathematics***
[OpenDreamKit.org](OpenDreamKit.org)
- Objectif: soutien à l'écosystème des logiciels libres pour les
«**mathématiques pures et applications**»
- En particulier:
- aspects techniques (distribution, HPC, ...)
- environnements interactifs basés sur **Python** (notebook **Jupyter**, ...)
- Collaboration avec Logilab. Et vous?
---
# Historique du projet
## Avant 2014
Idée de base: chercher des financements pour:
- Ingénieurs de recherche
- Animation de la communauté
Différentes tentatives non fructueuses (ANR, ...)
## De janvier 2014 à Août 2014
- Recherche d'information sur le programme H2020
- Contacts avec la communauté élargie
- Premier jet de 5 pages
- Choix d'un appel à projet potentiel
--
## Septembre 2014
- Rencontre avec 12 des participants potentiels
- Analyse des besoin
- Première liste de tâche
- Première liste de workpackages
- Reconfiguration du consortium
## D'octobre 2014 à janvier 2015
- Rédaction
- Reconfiguration des work packages
- Contacts avec la communauté élargie
- Extension du consortium
- Rédaction, rédaction, rédaction, ...
[video](https://www.youtube.com/watch?v=kM9zcfRtOqo)
--
## De mai 2015 à août 2015
- Acceptation
- Préparation
- Lancement
## Septembre 2015
- Début officiel
---
# Investissement
## Préparation
- Trois mois à temps plein pour le coordonateur
- Quelques mois répartis sur les autres
- 3000 €
## Lancement
- Deux mois à temps plein pour le coordonateur
- 1000 €
---
# Notre approche
## Suivre ses rêves
- Être ouvert
- Partir de la base:
- Analyser et collecter les besoins d'une communauté
- Choisir un appel en conséquence
- Trouver une histoire convaincante connectant les deux
- Encourager les participants à:
- Définir la vision du projet
- Faire ce qu'ils jugent être juste
--
## Engager la communauté
- Être ouvert
- Invitation large à participer à la définition du projet
- Écriture publique du projet (sur github)
- Développer et partager une **vision** sur le projet
1. Qui sommes nous?
2. Quel est notre objectif?
3. Quelle est notre stratégie?
4. D'où partons nous?
5. Comment ce projet s'articule avec les autres projets?
6. Pourquoi sommes nous «excellents»?
--
## Outils collaboratifs: essentiel!!!
### Échelle
- 100 pages
- 20 auteurs
- 3000 mails; 400 les deux derniers jours
À oublier: Word, Dropbox, ...
### Quelques recommandations
- Gestion de version: par ex. github
- Automatisation: tables, graphiques, calcul budget, ...
- Fichiers source en texte: par ex. Latex + [proposal](http://www.ctan.org/tex-archive/macros/latex/contrib/proposal)
- Vidéoconférence: appear.in + framapad
- Chat: gitter?
- Former, former, former
--
## Aide extérieure
### Déléguer à une boîte spécialisée?
- Avantageusement remplacé par des bons outils collaboratifs
### Avoir une ou deux personnes de bon conseil
- Très important!
- SAIC (Université Paris Sud)
- Nécessaire aussi pour le lancement du projet
---
# More reading
- [On OpenDreamKit's open and collaborative proposal writing](http://opendreamkit.org/2015/01/31/open-proposal-writing/)
- The [developer's perspective](/about-developers) on OpenDreamKit.
- [About OpenDreamKit](/about)
</section>
PyConFr Pau (France), 2015-10-17 to 2015-10-182015-10-01T00:00:00+00:00http://opendreamkit.org/2015/10/01/PyConFr<h2 id="main-goals">Main goals</h2>
<p>PyConFr is the main gathering of the python community in France. It is a good place to meet the
French open source Python community and to talk and learn about projects.</p>
<h2 id="opendreamkit-implication">OpenDreamKit implication</h2>
<p>Viviane Pons was present at the meeting and she gave a talk
on her teaching experience using SageMathCloud \cite{15PonsSMC} (peer-reviewed submission). She was
also part of a panel on diversity.</p>
<h2 id="results-and-impact">Results and impact</h2>
<p>This event was an occasion to introduce OpenDreamKit to the larger python community in France as well as to
keep active the link between Sage development and other Python open source projects. It is always a great occasion to discuss subjects
such as user interfaces, cloud servers, best practices, communities, etc. The diversity panel in particular was a great success bringing
together most of participants of the event. As this is a great concern for OpenDreamKit, we were happy to be part of it.</p>
Kickoff meeting2015-09-02T00:00:00+00:00http://opendreamkit.org/2015/09/02/KickoffMeeting<p>This meeting gathered 30 of the 50 OpenDreamKit participants, together
with a couple external persons. See the <a href="/meetings/2015-09-02-Kickoff/participants">Participants
list</a>.</p>
<p>Many of them had never met in person before. The objective was to build
a joint vision by giving each participant an overview of the
consortium and its wide variety of expertise, and of the project’s
aims and specific tasks, as well as to expose them with key software
components, web platforms and technologies in the ecosystem. For
details as well as presentation slides, see the
<a href="/meetings/2015-09-02-Kickoff/program">Program</a>.</p>
<p>We also discussed the management structure, technical infrastructure,
and planning. And we got to work – finally! – on some collaborative
tasks, through brainstorms and coding sprints. For notes on the
discussions and sprints, see the <a href="/meetings/2015-09-02-Kickoff/projects">Projects
page</a>.</p>
<p>For the registration and local information, see the <a href="/meetings/2015-09-02-Kickoff/logistics">Logistics
page</a>.</p>
Full-time developer position at TU Kaiserslautern for 1 year beginning March 20162015-07-01T00:00:00+00:00http://opendreamkit.org/2015/07/01/developer-position2-kaiserslautern<p>We are seeking a full-time developer at TU Kaiserslautern to work with Dr. William Hart on the MPIR <a href="http://mpir.org/">http://mpir.org/</a> bignum library.</p>
<h1 id="location">Location</h1>
<p>The developer will work at TU Kaiserslautern in the city of Kaiserslautern Germany. Kaiserslautern is next to one of the largest contiguous forests in Europe.</p>
<h1 id="mission">Mission</h1>
<p>To help maintain the MPIR library as part of the OpenDreamKit collaboration, in particular to enhance the performance and portability of the library.</p>
<h1 id="activities">Activities</h1>
<p>The successful applicant will develop an assembly superoptimiser for x86_64 architectures, develop assembly improvements of low-level routines which make use of the AVX instruction sets and help with maintenance of the MPIR build system, especially to support recent processors.</p>
<p>Depending on the skills of the applicant, the developer may also wish to contribute to higher level C implementations of bignum operations.</p>
<h1 id="skills-requirements">Skills requirements</h1>
<ul>
<li>
<p>Assembly language experience</p>
</li>
<li>
<p>Experience in low-level optimisation</p>
</li>
<li>
<p>Fluency in English</p>
</li>
<li>
<p>Must have an undergraduate degree in Mathematics, Computer Science, Computer Engineering or equivalent.</p>
</li>
<li>
<p>Experience in Open Source development and tooling</p>
</li>
</ul>
<h1 id="context">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<h1 id="expressions-of-interest">Expressions of interest</h1>
<p>Interested candidates should send an email to goodwillhart at googlemail.com for further information about the position.</p>
<h1 id="deadline-for-applications">Deadline for applications</h1>
<p>The deadline for applications for this position will be 29 Nov 2015. Applications should consist of an email to goodwillhart at googlemail.com and decker at mathematik.uni-kl.de with a CV, statement that you wish to apply for the position, a cover letter and a link to some code you have written, preferably on GitHub.</p>
<p>The application process also requires us to do a short interview, which can be conducted by telephone if candidates are not in Germany at the time.</p>
<p>We hope to make final decisions in early December.</p>
Full-time mathematical researcher and developer position at TU Kaiserslautern for up to four years beginning November 20152015-07-01T00:00:00+00:00http://opendreamkit.org/2015/07/01/developer-position-kaiserslautern<p>We are seeking a full-time mathematical researcher and developer at TU Kaiserslautern to work with Prof. Wolfram Decker on the Singular contribution to the OpenDreamKit project.</p>
<h1 id="deadline">Deadline</h1>
<p>Deadline for applications: Sept 23 2015.</p>
<h1 id="location">Location</h1>
<p>The developer will work at TU Kaiserslautern in the city of Kaiserslautern Germany. Kaiserslautern is next to one of the largest contiguous forests in Europe.</p>
<h1 id="mission">Mission</h1>
<p>To work as part of the OpenDreamKit collaboration, to do research on and implement improvements via parallelisation of components of Singular.</p>
<h1 id="activities">Activities</h1>
<p>To research and implement parallel algorithms in Singular in C/C++.</p>
<p>Depending on the skills of the applicant, the developer may also wish to contribute to other aspects of the Singular project and mathematical research in Kaiserslautern.</p>
<h1 id="skills-requirements">Skills requirements</h1>
<ul>
<li>
<p>C/C++ programming experience</p>
</li>
<li>
<p>Experience in computer algebra</p>
</li>
<li>
<p>Fluency in English</p>
</li>
<li>
<p>Must have an graduate degree in Mathematics.</p>
</li>
<li>
<p>Experience in Open Source development and tooling</p>
</li>
</ul>
<h1 id="context">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<h1 id="expressions-of-interest">Expressions of interest</h1>
<p>Interested candidates should send an email to decker {at} mathematik dot uni-kl dot de for further information about the position.</p>
Full-time developer position opening at Université de Bordeaux for Fall 20152015-05-29T00:00:00+00:00http://opendreamkit.org/2015/05/29/developer-position-bordeaux<p>This is an announcement for several full-time developer (<a href="http://fr.wikipedia.org/wiki/Ing%C3%A9nieur_de_recherche">Ingénieur de
Recherche</a>)
position opening at Université de Bordeaux, working on open source
software engineering for computational mathematics.</p>
<p>contact: vincent.delecroix-AT-labri.fr</p>
<h1 id="duration">Duration</h1>
<p>From 2 to 4 years starting as soon as possible.</p>
<h1 id="salary">Salary</h1>
<p>Depending on the applicant’s past experience, 1600 € to 2900 €
of monthly <em>salaire net</em> (salary after non-wage labour cost but before
income tax). Equivalently, this is a yearly <em>salaire brut</em> of
23600 € to 40000 €.</p>
<h1 id="location">Location</h1>
<p>The developer will work at the
<a href="http://labri.fr">LaBRI</a> in the Talence campus, 10 minutes
from Bordeaux city center by public transport.</p>
<h1 id="mission">Mission</h1>
<p>To support the maintenance and development of the OpenDreamKit
components, and in particular of the <a href="http://sagemath.org">SageMath</a> and
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a> projects.</p>
<h1 id="activities">Activities</h1>
<p>According to his or her specific skills, the developer will take on, in close
collaboration with the community, a selection of the software engineering tasks
defined in the OpenDreamKit project. Among these:</p>
<ul>
<li>
<p>Improve the existing software components (code optimisation and
parallelisation);</p>
</li>
<li>
<p>Improve communication between the software components;</p>
</li>
<li>
<p>Improve the graphic interfaces;</p>
</li>
<li>
<p>Participate actively in regular European development and training meetings
with the other OpenDreamKit participants.</p>
</li>
</ul>
<h1 id="skills-and-background-required">Skills and background required</h1>
<ul>
<li>
<p>Development in Unix-like environments;</p>
</li>
<li>
<p>Experience with code optimisation, parallelism (pthread, OpenMP, MPI),
and debugging tools (valgrind, gdb);</p>
</li>
<li>
<p>Fluency in several of C, C++, Python, Cython, assemblers;</p>
</li>
<li>
<p>Fluency in English;</p>
</li>
</ul>
<p>The following are not prerequisites but would be very much appreciated:</p>
<ul>
<li>
<p>Experience in open-source development (collaborative
development tools, interaction with the community, …);</p>
</li>
<li>
<p>Experience with computational mathematics software, in particular
<a href="http://sagemath.org">SageMath</a> or <a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>;</p>
</li>
<li>
<p>Mathematics background.</p>
</li>
<li>
<p>System administration.</p>
</li>
</ul>
<h1 id="applications">Applications</h1>
<p>Applicants should send an email to vincent-DOT-delecroix-AT-labri-DOT-fr with:</p>
<ul>
<li>
<p>A complete CV</p>
</li>
<li>
<p>A motivation letter</p>
</li>
</ul>
<p>Both documents must be in an open format (pdf, ps, plain text, …) either in
french or english.</p>
<h1 id="context">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Led by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<p>The developer will work within one of the largest teams of SageMath
and PARI/GP developers, composed essentially of researchers in
mathematics and computer science, at the <a href="http://www.labri.fr/">Laboratoire Bordelais de Recherche en
Informatique</a> and in nearby institutions
such as the <a href="http://www.math.u-bordeaux1.fr/imb/spip.php">Institut de Mathématiques de Bordeaux</a>
and <a href="https://www.inria.fr/centre/bordeaux">INRIA Bordeaux</a>.</p>
<h1 id="comments">Comments</h1>
<p>:warning: This is <em>not</em> a postdoc position. While side research will
be welcome, and a few tasks may possibly lead to some research
problems in computer science, the core tasks will be pure
development. Candidates wishing to pursue an academic research career
in the long run should consider twice whether this opportunity is
adequate for them.</p>
Full-time developer position opening at Université Paris-Sud for Fall 2015 (filled)2015-05-22T00:00:00+00:00http://opendreamkit.org/2015/05/22/developer-position-paris-sud<p>This is an announcement for a full-time developer (<a href="http://fr.wikipedia.org/wiki/Ing%C3%A9nieur_de_recherche">Ingénieur de
Recherche</a>)
position opening at Université Paris-Sud, working on open source
software engineering for computational mathematics.</p>
<h1 id="time-line">Time line</h1>
<p>For now we hope to run interviews in early July, for a recruitment in
early Fall 2015, and for up to four years (presumably in the form of a
one year contract, renewable three times).</p>
<p>A second full-time developer will be hired later on, presumably in Fall 2016.</p>
<h1 id="salary">Salary</h1>
<p>Depending on the applicant’s past experience, between 2000€ and 3000€
of monthly “salaire net” (salary after non-wage labour cost but before
income tax). Equivalently, this is a “salaire brut” of up to 46200€
yearly, or a “salaire brut chargé” (total cost for the project) of up
to 5500€ monthly.</p>
<h1 id="location">Location</h1>
<p>The developer will work at the
<a href="http://www.lri.fr">Laboratoire de Recherche en Informatique</a> of
<a href="http://www.u-psud.fr">Université Paris Sud</a>, in the
Orsay-Bures-Gif-Saclay campus, 25 km South-West of Paris city centre.</p>
<h1 id="mission">Mission</h1>
<p>To support the maintenance and development of the
<a href="http://opendreamkit.org">OpenDreamKit</a> components, and in particular
of the <a href="http://www.sagemath.org">SageMath</a> project, by enhancing their
software infrastructure, in close collaboration with the community.</p>
<h1 id="activities">Activities</h1>
<p>A successful candidate will be expected to do significant progress, in
close collaboration with the community, on some of the engineering
tasks defined in the <a href="http://opendreamkit.org">OpenDreamKit</a>
project. Among these:</p>
<ul>
<li>
<p>Improve the build, packaging, test, and distribution infrastructure;</p>
</li>
<li>
<p>Improve the portability of many of the components, in particular on
the Windows+Cygwin platform;</p>
</li>
<li>
<p>Refactor the static Sphinx-based documentation system, and
design and develop dynamic documentation tools;</p>
</li>
<li>
<p>Participate in designing and implementing a component architecture
for the SageMath ecosystem, as a foundation for Virtual Research
Environments such as SageMathCloud;</p>
</li>
<li>
<p>Participate actively in regular international development and
training meetings with the other OpenDreamKit participants and the
community at large.</p>
</li>
</ul>
<p>For details, see the technical task lead by UPSud and UVSQ in
Section 3.1.6 ``Workpackage Description’’ of the
<a href="https://github.com/OpenDreamKit/OpenDreamKit/raw/master/Proposal/proposal-www.pdf">OpenDreamKit Proposal</a>.</p>
<h1 id="skills-and-background-requirements">Skills and background requirements</h1>
<ul>
<li>
<p>Fluency in C, C++, Python, …</p>
</li>
<li>
<p>Strong experience with development and system administration in
Unix-like environments;</p>
</li>
<li>
<p>Experience with Windows development (Cygwin) desirable;</p>
</li>
<li>
<p>Strong experience with software build systems (e.g. scons, cmake, or automake);</p>
</li>
<li>
<p>Strong experience in open-source development (collaborative
development tools, interaction with the community, …);</p>
</li>
<li>
<p>Experience with code optimization, parallelism, etc, appreciated but
not a prerequisite.</p>
</li>
<li>
<p>Experience with computational mathematics software, in particular
<a href="http://sagemath.org">SageMath</a>, desirable but not a prerequisite;</p>
</li>
<li>
<p>Mathematics background and/or holding a PhD appreciated but not a
prerequisite;</p>
</li>
<li>
<p>Strong communication skills;</p>
</li>
<li>
<p>Perfect fluency in oral and written English;</p>
</li>
<li>
<p>Speaking French is appreciated but not a prerequisite.</p>
</li>
</ul>
<h1 id="context">Context</h1>
<p>The position will be funded by</p>
<p><a href="http://opendreamkit.org">OpenDreamKit</a>, a
<a href="https://ec.europa.eu/programmes/horizon2020/">Horizon 2020</a>
European <a href="https://ec.europa.eu/programmes/horizon2020/en/h2020-section/european-research-infrastructures-including-e-infrastructures">Research Infrastructure</a>
project that will run for four years, starting from September</p>
<ol>
<li>This project brings together the open-source computational
mathematics ecosystem – and in particular
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>,
<a href="https://mathhub.info/">MathHub</a>,
and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.
– toward building a
flexible toolkit for
<a href="http://www.2020-horizon.com/e-Infrastructures-for-virtual-research-environments-%28VRE%29-i1490.html">Virtual Research Environments</a>
for mathematics. Lead by Université Paris-Sud, this project involves
about 50 people spread over 15 sites in Europe, with a total budget of
about 7.6 million euros.</li>
</ol>
<p>Within this ecosystem, the developer will work primarily on the free
open-source mathematics software system
<a href="http://sagemath.org">Sagemath</a>. Based on the
<a href="http://www.python.org">Python</a> language and many existing open-source
math libraries, SageMath is developed since 10 years by a worldwide
community of 300 researchers, teachers and engineers, and has reached
1.5M lines of code.</p>
<p>The developer will work within one of the largest teams of SageMath
developers, composed essentially of researchers in mathematics and
computer science, at the <a href="http://www.lri.fr/">Laboratoire de Recherche en
Informatique</a> and in nearby institutions.</p>
<h1 id="comments">Comments</h1>
<p>:warning: This is <em>not</em> a postdoc position. While side research will
be welcome, and a few tasks may possibly lead to some research
problems in computer science, the core tasks will be pure
development. Candidates wishing to pursue an academic research career
in the long run should consider twice whether this opportunity is
adequate for them.</p>
<h1 id="applications">Applications</h1>
<p>To apply for this position, please send an e-mail to
upsud-recruitment-research-engineer at opendreamkit.org before July
1st, with the following documents attached:</p>
<ul>
<li>
<p>cover_letter.pdf: a cover letter, in English (why are you interested in this particular position);</p>
</li>
<li>
<p>CV.pdf: a CV, highlighting among other things your skills and
background and your contributions to open source software;</p>
</li>
<li>
<p>phd_reports.pdf: PhD reports (when applicable);</p>
</li>
<li>
<p>reference letters (each named reference_letter_<contactname>.pdf),
or alternatively reference contact information.</contactname></p>
</li>
</ul>
<p>Applications sent after July 1st will be considered until the position
is filled.</p>
Proposal accepted2015-05-15T00:00:00+00:00http://opendreamkit.org/2015/05/15/OpenDreamKit-accepted<p>We are delighted to announce that the Horizon 2020 research proposal
<a href="http://opendreamkit.org/">OpenDreamKit</a> was accepted by the European
commission.</p>
<p>Starting next Fall and for four years, this project will provide
substantial funding to the open source computational mathematics
ecosystem, and in particular popular tools such as
<a href="http://linalg.org/">LinBox</a>,
<a href="http://mpir.org">MPIR</a>,
<a href="http://sagemath.org/">SageMath</a>,
<a href="http://www.gap-system.org/">GAP</a>,
<a href="http://pari.math.u-bordeaux.fr/">PARI/GP</a>,
<a href="http://lmfdb.org/">LMFDB</a>,
<a href="http://www.singular.uni-kl.de/">Singular</a>, and the
<a href="http://jupyter.org/">IPython/Jupyter</a> interactive computing
environment.</p>
<p>The total budget is about 7.6 million euros. The largest portion of
that will be devoted to employing an average of 11 researchers and
developers working full time on the project. We will announce job
openings in the coming weeks.</p>
<p>Additionally, the participants will contribute the equivalent of six
other people working full time. Altogether the project involves about
50 people spread over 15 sites in Europe.</p>
<p>This is a formidable recognition of the strength and maturity of this
ecosystem, of the power of open source development models, and of the
amazing hard work of many communities over the last decades.</p>
<p>The writing of the proposal itself was
<a href="http://opendreamkit.org/2015/01/31/open-proposal-writing/">open and collaborative</a>.
It grew out of a reflection on the long term needs of the community.
It benefited considerably from the feedback of many; we would like to
thank all those who helped shape this proposal and make it happen.</p>
<p>It is our hope that this financial support will help push forward
critical technical tasks. We tried hard in the proposal to make a
worthwhile selection of such tasks, within some constraints imposed by
the specific call. We are now legally committed to treat those tasks
in priority. This kind of long term prediction work is tough: one of
them has actually already been completed by the community in the mean
time! This is great; whenever this will happen we will be able
reprioritize the resources to whatever emerging needs that will arise.</p>
<p>Ultimately, this project belongs to the community. <a href="http://opendreamkit.org/">Get involved</a>!</p>
On open collaborative proposal writing2015-01-31T00:00:00+00:00http://opendreamkit.org/2015/01/31/open-proposal-writing<p>This is a debriefing rant about the open collaborative writing of the
<a href="http://opendreamkit.org/">OpenDreamKit</a> proposal, mostly as notes for
self (=Nicolas) for future occasions: how to approach it, what tools
to use, some tricks and pitfalls to avoid, …</p>
<h2 id="context">Context</h2>
<p>The OpenDreamKit proposal was written collaboratively by a diverse team
across Europe and science. Have fun watching this
<a href="https://www.youtube.com/watch?v=kM9zcfRtOqo">video</a>, produced with
<a href="https://code.google.com/p/gource/">Gource</a>, of the activity on our
<a href="https://github.com/sagemath/grant-europe/">collaborative work space on github</a>.
A pretty good illustration – especially during the final sprint – of
how a dispersed but dedicated team can collaborate efficiently with
modern tools! You may enjoy as well reading this fun <a href="http://inverseprobability.com/2015/01/14/open-collaborative-grant-writing/">blog
post</a>.</p>
<h2 id="stand-by-your-dreams">Stand by your dreams</h2>
<p>Write a proposal that you and all participants believe is right.</p>
<ul>
<li>
<p>Go open.</p>
<p>Then you won’t have a choice than being proud of what you write :-)</p>
<p>What we kept private: personal info like salary, the mailing list
for discussions (see below), the nity grity details of the budget
that are not so relevant anyway, some TODO lists.</p>
</li>
<li>
<p>Use a bottom up approach.</p>
<p>Start by analyzing the needs of the community. Choose the call
accordingly. And then, and only then, think about a great selling
story that will make the call and the needs match naturally
together.</p>
</li>
<li>
<p>Encourage the participants to shape the proposal to feel included,
and to base all their decisions upon what they believe in.</p>
</li>
</ul>
<h2 id="engage-the-community">Engage the community</h2>
<p>The main point of writing this proposal in the open was to get feedback
from the community. This allowed for example to rework early on some
phrasing that could have been misinterpreted.</p>
<p>Yet we did not get as much feedback as I was hoping for. This is
probably because a proposal is a massive amount of information. So it’s
hard for an outsider to make one’s way through it. In particular because
quite some of the information is not directly relevant to the community,
and because the overall structure will tend to remain messy for a long
time. Here are some tips that might help next time.</p>
<ul>
<li>
<p>Build a shared vision:</p>
<p>Write early on – and keep updating – a short introduction page on
the web site with tentative answers to the following questions:</p>
<ul>
<li>Who are we?</li>
<li>What is our goal</li>
<li>What is our strategy?</li>
<li>From where do we start?</li>
<li>How do we connect or differ from other projects?</li>
<li>Why are we excellent?</li>
</ul>
<p>Furthermore, add direct links to the main tasks.</p>
</li>
<li>
<p>Whenever relevant, write a draft of the tasks as independent tickets
(typically, for us, on trac.sagemath.org). People will then stumble
naturally on those that are of interest to them, and react.
Encourage them to take an active role to shape the tasks in the most
useful way for the community.</p>
</li>
</ul>
<h2 id="engage-your-team">Engage your team</h2>
<p>Easier to say than to do :-)</p>
<p>In short: run forward and trust your team mates: they have talent.</p>
<p>This <a href="http://inverseprobability.com/2015/01/14/open-collaborative-grant-writing/">blog
post</a>
gave me an interesting perspective from the participant side. The key
thing is that it’s both hard and intimidating to get involved.</p>
<p>For the technical side, see the next section.</p>
<h2 id="collaborative-tools">Collaborative tools</h2>
<p>Using the right tools is critical for productive collaborative proposal
writing. In case you have a doubt, watch the
<a href="https://www.youtube.com/watch?v=kM9zcfRtOqo">video</a> of the activity on
our repository.</p>
<p>Some participants won’t be familiar with the tools, and may push toward
basic solutions like dropbox+word. This is <em>NOT</em> an option when 20
people hack simultaneously on the same files. Stay firm, train them, and
work hard to keep the technical barrier low.</p>
<p>(for this particular proposal, having some technical barrier was
actually not such a bad thing: I wanted to select tech savvy people
anyway).</p>
<h3 id="use-a-good-version-control-system-eg-git">Use a good version control system (e.g. git)</h3>
<p>A decentralized version control system (like git or mercurial) has a key
advantage of offering massive backups.</p>
<p>Having a web interface to the repository is nice for people to easily
browse the information and do small edits.</p>
<p>github did the job to host our repository. It was ok since we had only
public information (no private info on the cloud, please!). There are of
course alternatives (e.g. gitlab based).</p>
<h3 id="use-a-public-repository">Use a public repository</h3>
<p>Be liberal in giving write access. People tend to be shy by default
anyway; and if accidently someone would go to far it’s always possible
to revert.</p>
<h3 id="use-a-good-document-writing-infrastructure-eg-latex">Use a good document writing infrastructure (e.g. LaTeX)</h3>
<p>The proposal document contains a <em>lot</em> of redundant information (e.g.
gantt chart, lists of tasks/deliverables/…, involvement measures,
budget, …). It is <em>vital</em>, especially for the final rush, to have some
appropriate infrastructure that builds automatically all of those from
duplication free and well located semantic information.</p>
<p>LaTeX and the
<a href="http://www.ctan.org/tex-archive/macros/latex/contrib/proposal">proposal</a>
style file is a good technical option.</p>
<p>Choose such an infrastructure, make the compilation procedure completely
automatic (<code class="language-plaintext highlighter-rouge">make</code>, with typically two variants to build either the draft
version with comment or the final version). Get an expert on board to
help, and push the infrastructure to its limits.</p>
<p>We went quite far in this direction. Next time I would want to go even
further and also build the external budget files automatically from the
semantic information in the proposal (not yet implemented).</p>
<p>There is one difficulty here: building the budget files requires to
store some information that is private, like the salary of the
participants. A good approach would be to have a secondary private git
repository with read-write access granted only to, e.g., site leaders
and staff. The private info probably could take the form of a single
table like latex file associating info like salary to participants. The
infrastructure shall allow to build the proposal itself with or without
the private info.</p>
<h3 id="document-building-and-version-control">Document building and version control</h3>
<p>Usual tip: don’t put automatically produced files (e.g. pdf’s) under
version control to avoid conflicts.</p>
<p>Exception: if you use like us the web repository browser as web site for
the proposal, you may want to commit from time to time the proposal’s
pdf to make it accessible to everybody. That’s alright, but do it under
a specific name, so that people don’t conflict with it when they build
the pdf locally. See e.g. the <code class="language-plaintext highlighter-rouge">final</code> dependency in our
<a href="https://github.com/OpenDreamKit/OpenDreamKit/blob/master/Proposal/Makefile">Makefile</a>.</p>
<h2 id="finding-a-nice-acronym">Finding a nice acronym</h2>
<p>There are many acronym generators online that can be quite helpful in
the process. But at the end of the day it’s all about creativity; get
everyone involved. In our case we were rather stuck until Paul came up
with the serendipitous idea to use “Digital” rather than “Virtual”
(Dream sounds better than Vream :-) ).</p>
<h2 id="communication">Communication</h2>
<h3 id="meetings">Meetings</h3>
<p>The meeting we had in September was fundamental to discuss the needs and
shape the proposal. A bit earlier in the process (e.g. 6 months before)
would have been good. A bit longer (3 days instead of 2) could have
helped too. At the end of the meeting, there should be:</p>
<ul>
<li>A first draft of answers to the questions above</li>
<li>Some idea of what the selling story will be (how do we fit the
call?)</li>
<li>A tentative list of aims and objectives</li>
<li>A tentative list of work packages, with designated leaders for each
of them</li>
<li>A tentative list of sites, with leaders for each of them</li>
</ul>
<p>Having a second meeting one or two months before the submission would
have been helpful to launch the final rush, brainstorm on the texts. We
mitigated this with online chats.</p>
<h3 id="online-video-chats">Online video chats</h3>
<p>Online video chats were extremely useful, especially for brainstorming
with 2-3 people on sensitive pieces of text (intro to the proposal, work
package descriptions, …), for getting to know each other, and for fast
distribution of tasks at the end. Use them early on and regularly.</p>
<p>We used intensively <a href="http://appear.in">appear.in</a> for this. It can’t get
simpler: just share a URL (we used <a href="http://appear.in/vre-math">http://appear.in/vre-math</a>) with
your collaborators and you are up and running. It worked for us with
chrome, safari, firefox. It’s peer to peer, so in term of privacy it
should be alright.</p>
<p>A limitation: if one of the person had a low bandwidth, this seemed to
affect everybody in the chat room; turning of the video usually helped.</p>
<p>When brainstorming on chunks of text we wanted to see the evolving text
simultaneously. Git was not enough for this. So we used temporary google
documents. Not very satisfying. I am still looking for a good
peer-to-peer collaborative plain text editor.</p>
<h3 id="mailing-list">Mailing list</h3>
<p>We used a private mailing list. I believe that this helped people speak
openly on more sensitive matters. But that’s debatable. Still we were
liberal in including interested external people to get feedback.</p>
<p>Most of the communication went on this mailing list; roughly ~3000
e-mails; with 400 of them in the last two days; crazy …</p>
<p>The good thing was that everybody could see what was going on. That’s
good in particular at the beginning, when there is little action and
it’s important to attract attention and to show off that the project is
alive. Having an archive is important too so that anyone can dig for
information. You can’t get this with private discussions.</p>
<p>However this went <em>really</em> overboard at the end: for all the
participants that were partially involved, sorting through all the
information to find the one that was relevant to them was hard. Many of
them spent as much time reading through the e-mails as really acting.
This is totally unproductive.</p>
<p>I don’t have a good solution. I was trying to make it clear at the top
of each e-mail who was directly involved (dear XXX); however this was
clearly not sufficient. A step could be to setup some official keyword
conventions for the subject line, to let people sort them efficiently.
Something like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>H2020: WP leaders: ...
H2020: WP HPC: ...
H2020: Site leaders: ...
H2020: All participants: ...
</code></pre></div></div>
<p>Or maybe use some other communication media than mailing lists.</p>
<h3 id="todo-list">TODO list</h3>
<p>It must be trivial for each participant to find out what (s)he could do
for the project at any given point. The <code class="language-plaintext highlighter-rouge">\TOWRITE{XXX}{...}</code> lines in
the latex sources are natural spots to write this information.</p>
<p>However extracting and synthesizing this information is necessary to
really engage people (they won’t <code class="language-plaintext highlighter-rouge">git pull</code> the latest version and run
<code class="language-plaintext highlighter-rouge">grep</code> by themselves unless they are already seriously engaged).</p>
<p>Sending the synthesis list by e-mail did not work that well. Instead it
should be posted on the web. We were using initials (and keywords for
tasks that could be handled by several persons in a group). First names
probably would have worked better. In any cases, the list of names /
keywords should be standardized.</p>
Proposal submitted2015-01-15T00:00:00+00:00http://opendreamkit.org/2015/01/15/proposal-submitted<p>The OpenDreamKit proposal was submitted on January 15th of 2015.
It was written collaboratively by a diverse team
across Europe and science. Have fun watching this
<a href="https://www.youtube.com/watch?v=kM9zcfRtOqo">video</a>, produced with
<a href="https://code.google.com/p/gource/">Gource</a>, of the activity on our
collaborative work space on github. A pretty good illustration –
especially during the final sprint – of how a dispersed but dedicated
team can collaborate efficiently with modern tools!</p>
<p>You may enjoy as well reading this fun <a href="http://inverseprobability.com/2015/01/14/open-collaborative-grant-writing/">blog
post</a>.</p>