OpenDreamKit is a project that brings together a range of projects and associate software to create and strengthen virtual research environments. The most widely used research environment is the Jupyter Notebook from which computational research and data processing can be directed. The OpenDreamKit project provides interfaces to well established research codes and tools so that they can be used seamlessly and combined from within a Jupyter Notebook.
OpenDreamKit also supports open source research codes directly by investing into structural improvements and new features to not only connect all of these tools but also enrich them, and make them more sustainable.
More concretely, the tools brought together in the OpenDreamKit project include mathematical software packages such as SageMath, GAP, PARI, Singular, but also simulation tools from materials science such as OOMMF. Visit our Software page to know more.
OpenDreamKit has also advanced the Jupyter Notebook Ecosystem, for example through providing tools to support reproducibility in computational science, effectively comparing and merging notebooks, and providing interactive 3D visualisation within the notebook.
Want to know more?
The project involves about 50 people spread over 15 sites in Europe, with a total budget of about 7.4 million euros. The largest portion of that will be devoted to employing an average of 11 researchers and developers working full time on the project. Additionally, the participants will contribute the equivalent of six other people working full time.
OpenDreamKit is a Horizon 2020 European Research Infrastructure project (#676541) that will run from Sept. 2015 to August 2019. Here are the documents form the binding contract between the consortium and the European Union. 3 amendments to the Grant were necessary since September 2015 to follow the evolution of partners forming the consortium and of the workplan.
- Part 1: Description of Action: Workpackages, tasks, deliverables
- Part 2: Objectives, workplan, description of participants, budgets
The github repository for the project shows the advancements of the work accomplished in regards with the Grant agreement
Abstract of the project
OpenDreamKit will deliver a flexible toolkit enabling research groups to set up Virtual Research Environments, customised to meet the varied needs of research projects in pure mathematics and applications, and supporting the full research life-cycle from exploration, through proof and publication, to archival and sharing of data and code.
OpenDreamKit will be built out of a sustainable ecosystem of community-developed open software, databases, and services, including popular tools such as LinBox, MPIR, Sage(sagemath.org), GAP, PARI/GP, LMFDB, and Singular. We will extend the Jupyter Notebook environment to provide a flexible UI. By improving and unifying existing building blocks, OpenDreamKit will maximise both sustainability and impact, with beneficiaries extending to scientific computing, physics, chemistry, biology and more, and including researchers, teachers, and industrial practitioners.
We will define a novel component-based VRE architecture and adapt existing mathematical software, databases, and UI components to work well within it on varied platforms. Interfaces to standard HPC and grid services will be built in. Our architecture will be informed by recent research into the sociology of mathematical collaboration, so as to properly support actual research practice. The ease of set up, adaptability and global impact will be demonstrated in a variety of demonstrator VREs.
We will ourselves study the social challenges associated with large-scale open source code development and publications based on executable documents, to ensure sustainability.
OpenDreamKit will be conducted by a Europe-wide steered by demand collaboration, including leading mathematicians, computational researchers, and software developers with a long track record of delivering innovative open source software solutions for their respective communities. All produced code and tools will be open source.
This proposal grew out of a reflection on the needs of the (pure) mathematics community in terms of computational software and databases. The highly successful development in the last decades of systems such as GAP, LinBox, LMFDB, PARI, Sage, or Singular, has proven the viability and power of collaborative open source development models, by users and for users, even for delivering general purpose systems targeting a large public (researchers, teachers, engineers, amateurs, …).
Yet some critical long term investments, in particular on the technical side, are in order to boost the productivity and lower the entry barrier:
- Streamline access, distribution, portability on a wide range of platforms, including High Performance Computers or cloud services.
- Improve user interfaces, in particular in the promising area of collaborative workspaces as those provided by CoCalc (previously called SageMathCloud).
- Lower barriers between research communities and promote dissemination. For example make it easy for a specialist of scientific computing to use tools from pure mathematics, and reciprocally.
- Bring together the developers communities to promote tighter collaboration and symbiosis, accelerate joint development, and share best practices.
- Outsource as much of the development as possible to larger communities to focus the work forces on their core specialty: the implementation of mathematical algorithms and databases.
Many people in the community have been working really hard on the above items but lack crucially manpower or funding; the purpose is to supply them with such.
The European H2020 call EINFRA-9: e-Infrastructure for Virtual Research Environment was a natural fit: putting the emphasis on Virtual Research Environments nicely wraps up all the above needs in a single aim.
A great opportunity is the rapid emergence of key technologies, and in particular the Jupyter (previously IPython) platform for interactive and exploratory computing which targets all areas of science.
We built the consortium by gathering core European developers of the aforementioned systems for pure mathematics, and reaching toward the numerical community, and in particular the Jupyter community, to work together on joint needs.
By definition this project will be mostly funding actions in Europe; however those actions will be carried out, as usual, in close collaborations with the worldwide community.