Data Repository Infrastructures for Trustworthy Research Data Management

Abstract:

Having access to current and high-quality data is one of the key elements in data-driven research. Specifically, in the area of process analysis, the management of highly dynamic data constitutes as significant challenge. Building upon previous cooperation in the context of the TU Wien Pilotfabrik in cooperation with UTeM’s Teaching Factory, the need to ingest, manage and provisioning live sensor data streams has become evident. This project aims at adapting database repository infrastructure to ingest live sensor data streams, document their provenance and establish a solid data management processes to ensure data quality, trustworthiness and reproducibility of the research based on such data.

Implementation Period:

08/2024 – 10/2024

Project:

Project Objectives and Impact

The primary goal of this project was to adapt the database repository infrastructure DBRepo to support the ingest of sensor data streams and evaluate its suitability for the setting in the UTeM teaching factory. This goal could be successfully demonstrated by setting up an instance of DBRepo in a virtual machine. First, static data from X-ray Crystallography analyses was extracted, transformed and fed into the system to establish an initial demonstrator. Subsequently, the team succeeded in connecting it to the data feed emerging from a milling simulation running in the Siemens Tecnomatix Plant Simulation system. The live data feed could be processed, fed into DBRepo and subsequently be made available for further processing. This demonstrates that the infrastructure is able to meet the processing needs of the selected application domain. This offers a promising basis to explore its wider adoption across different domains, making it a candidate for institution-wide service roll-out.

Results and Contributions

Melaka (Outgoing from TU Wien):

The workshop in Melaka lasted from 02.09.-07.09.2024 and was located at Universiti Teknikal Malaysia Melaka (UTeM). To highlight the benefits of trustworthy, institutional data repositories, Prof. Dr. Andreas Rauber presented the different repositories used for managing research artifacts at TU Wien. Martin Weise deployed the inhouse-developed Database Repository (DBRepo) that provides centrally managed relational databases to researchers. It stores structured research datasets and makes it accessible with multiple APIs (HTTP, AMQP, JDBC) that are used by data scientists and data providers, as well as a Python library. Moritz Staudinger gathered and processed several datasets provided by different research groups at UTeM to allow for their import in DBRepo. This process included also the parsing from proprietary systems such as the Siemens Plant Simulator or X-Rays to open, machine processable formats as CSV.

After these steps were completed, we had an interactive workshop session with Dr-Ing. Azrul Azwan and Prof. Dr. MD Nizam on how to properly integrate file- and code-based simulation data into InvenioRDM[1] (such as TUWRD[2]) and GitLab. As a use case for this, Azrul Azwan presented their Reconfigurable Manufacturing System (RMS) that challenges the students as part of the teaching process of Industry 4.0. We imported the simulation model into TUWRD https://test.researchdata.tuwien.ac.at/records/mk4nz-4n637 and introduced his team to Git repositories. For this task we uploaded the simulation package (containing source code) to our curated code repository https://gitlab.tuwien.ac.at/fairdata/utem.

We further received an array of X-ray crystallography (XRD) data collected by UTeM as text files which we extracted using automated scripts. Then we imported the structured raw data into the UTeM DBRepo instance https://dbrepo.utem.edu.my/pid/1 and TUWRD https://test.researchdata.tuwien.ac.at/records/td04r-b3084. These could subsequently be accessed via Jupyter and the cloud-environment thereof which was installed as part of the workshop. Users can now use Jupyter from the browser and access data from UTeM DBRepo to perform Data Science tasks on it.

To demonstrate the workflow for hosting and using experimental data in DBRepo, we had an interactive workshop session with Dr.-Ing. Azrul Azwan on this topic. After this data was stored in DBRepo, we accessed the data via Jupyter Notebook, filtered and processed the data to then visualize the evolution of it over the experiment.

[1] https://inveniosoftware.org/products/rdm/

[2] https://researchdata.tuwien.ac.at/

Vienna (Incoming from UTeM):

The team of UTem visited TU Wien from 30.09.2024-04.10.2024 to follow up on the activities during the workshop in Melaka and extend on the integration of data management at UTeM. We were able to set up a data feed into the DBRepo, directly from the Siemens Tecnomatix Plant Simulation. The data later was analyzed using UTeM’s Jupyterhub service that we deployed during our visit to Melaka. Prof. Rauber conducted a guided tour through the Vienna Scientific Cluster, highlighting the importance of this High-Performance Computing cluster initiative in Austria.