Key Performance Indicators Report, RP1
Benoît Pilorget
---
## Defining Key Performance Indicators
Very difficult for an opensource project:
- VRE toolkit, not VRE deployments themselves
- Citability in collaborative software is much harder than academic papers
- Difficulty in claiming credit for 100% of the impact
---
## Types of Key Performance Indicators
- OpenDreamKit's website statistics
- Downloads/installs (but parts of larger systems, different from new users, etc)
- Breakdown workshop attendance counts
- Contribution measurements on software
- **success stories**
---
## Disclaimer
### The variation of indicators between RP1 and RP2 will need to be interpreted
---
## KPI 1: More effective collaborations between researchers
---
### Number of Jupyter kernels
- September 2015: 49
- April 2017: [87 kernels](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels)
### Nr of Social Media followers (Twitter): 271 followers on April 2017
--
### Number of questions on Stackoverflow in March 2017 that concern:
- Sage: 650
- Gap: 309
- Singular:71
- Pari-GP:75
- MPIR:20
- IPython: 6897
- Jupyter: 3436
--
### Webstats of ODK website: 1200 unique visitors since mid-March 2017
--
### Attendees to workshops/conferences:
- 6 OpenDreamKit workshops: 105
- 6 Non-ODK workshops: 102
- 10 Conferences for: 3655
---
## KPI 2: Higher efficiency and creativity in research, higher productivity of researchers thanks to reliable and easy access to discovery, access and re-use of data
---
### Number of users of Sagemathcloud: Feb. 2016 to Feb 2017
- about 85000 new accounts
- 83000 active users on the period
--
### Number of downloads of SageMath
--
#### PARI/GP, for RP1
- 771 downloads for Debian (package pari-gp)
- 4446 downloads for Ubuntu (package pari-gp)
- 597 articles (91 since 2015)
--
#### Citations on Google scholar for GAP:
--
### Testimony of the University of Sheffield for the [Jupyter NoteBook Trial](https://www.sheffield.ac.uk/bms/teaching/stories/bms353), by Mark Quinn
Students' improvements using Sage through Jupyter Notebook
---
## KPI 3: Accelerated innovation in research via an integrated access to digital research resources, tools and services across disciplines and user communities
---
### Nr of contributors to ODK components (April 2017)
- Sage:[358 contributors for 57,973 commits](https://github.com/sagemath/sage/)
- Gap: [25 contributors for 4,483 commits](https://github.com/gap-system/gap)
- IPython: [485 contributors for 22,565 commits](https://github.com/ipython/ipython)
- Jupyterhub: [65 contributors for 1,670 commits](https://github.com/jupyterhub/jupyterhub)
- Singular: [39 contributors for 19,549 commits](https://github.com/Singular/Sources)
- Pari-GP: [32 contributers, including 22 since Sept. 2015](http://pari.math.u-bordeaux.fr/git/pari.git)
### Number of SageMath downloads on Windows
- 0 for now -> due Month 24
--
### User-contributed packages
- Sage: 0 for now
- Gap:
---
## KPI 4: Researchers able to process structured and qualitative data in virtual and/or ubiquitous workspaces
---
### Number of datasets in LMFDB:
- 50 in production database, 84 in beta database
### Number of components that support SCSCP
- Presumably 0
### Nr of data queries on LMFDB (and knowledge produced via MMT):
- 1357 sessions
- 885 users (split 50/50 between new and returning)
- 6950 page views
- 4m21s average session duration
---
## KPI 5: Increased take-up of collaborative research and data sharing by new disciplines, research communities and institutions
---
### Publications stats: 15 publications acknowledging OpenDreamKit's support on Openaire for RP1
--
#### Exemple of a high impact publication
[The L-Functions and Modular Forms Database Project](https://www.openaire.eu/en/search/publication?articleId=core_ac_uk__::bac82a8fe3e313ae79db60e152a36c54), by J. Cremona, 2016
*Attention score of 33: in the top 5% of research outputs scored by Altmetric*
### Nr of publications citing ODK or one of its components:
- Altmetric and Openaire support contacted in January 2017: no answer yet
--
### Nr of tutorial attendees in RP1
- 4 tutorials organised
- 160 people targeted
---
## KPI process: some internal feedback for the 1st Reporting Period
- Dificulty in choosing the most meaningful indicators
- Documenting needs to be done to repeat measurements for RP2
