Researchers have limited means to easily provide evidence about the quality of the software tools they produce. The OpenEBench platform supports scientific community-led benchmarking efforts across different domains in Life Sciences. It is the ELIXIR benchmarking and technical monitoring platform for bioinformatics tools, web servers and workflows. The service includes a data element and provides a benchmarking tool for evaluating the quality of the results of research tools. Currently, we have more than 20 engaged communities, 9 of them have make public their benchmarking results through the OpenEBench web portal. Related to technical monitoring, the portal includes an automatic research tools evaluator measuring for more than 35000 tools, including concrete metrics evaluating several properties like accessibility, distribution, licenses, publications, etc. Lately, the platform included a new functionality, a Life Sciences Observatory, considering more than 43.000 sources including trends on licensing versioning, use of version control systems.
Addressing target audiences and expressing needs
- Help in technical expertise
- Collaboration
Research and technology organizations, as well as universities and academia, can collaborate in two ways. Firstly, they can collaborate as users, life sciences research groups can generate their benchmarking results and make them accessible through the web portal. Users can also contribute identifying issues and potential improvements. Secondly, they can be involved in the framework development, for example, we just released a re-designed web portal improving usability and we included the life science observatory as part of the tools. Currently, we are working in a new data model design.
- Research and Technology Organisations
- Academia/ Universities
R&D, Technology and Innovation aspects
During this year we have deployed a new redesigned web portal improving its usability and included the life sciences observatory functionality. We are currently working in two directions, we are designing a new data model (v2.0) including some issues that arise from the previous version and we are working in a new intranet functionality to automate some of the processes to manage communities.
OpenEBench software components are released under Open Source Software (OSS) licences, the source code is publicly available in GitHub and GitLab public tools. In the openebench-hub repository in GitHub (https://github.com/inab/openebench-hub ) include the list of software components and a link to their corresponding repositories. OSS allows other researcher to understand, reuse and replicate the results. The data model used internally by OpenEBench is also publicly described providing a deeper knowledge of the data needed by the tool (https://github.com/inab/benchmarking-data-model and https://raw.githubusercontent.com/inab/benchmarking-data-model/1.0.3/json-schemas/openebench-bdm-1.0.x.dot.png ). There is also extensible documentation from the technical point of view (https://openebench.bsc.es/docs/) and from research point of view (https://openebench.readthedocs.io/en/latest/ ).
