Towards a metrics framework to facilitate the evaluation of temporal data modelling approaches in knowledge graphs

Abstract

In recent years, Knowledge Graphs (KGs) have emerged as a powerful tool for modelling data, utilised by companies like Facebook, Amazon, and IBM in applications such as natural language processing, recommendation systems, and question answering machines. However, modelling temporal data in Resource Description Framework (RDF)-based KGs remains a challenge, as RDF lacks a standardised way to represent time. This temporal dimension is crucial for domains such as healthcare, where time-varying data is essential for effective decision-making. This thesis addresses the challenge of temporal data modelling in KGs by proposing the TEDME-KG metrics framework, a tool designed to evaluate various temporal data modelling approaches in RDF-based KGs. The research is grounded in a real-world healthcare use case focusing on medication management for patients with ANCA vasculitis, a rare disease. The use case leverages the FAIRVASC project, which involves the standardisation of vasculitis patient data across European registries. The TEDME-KG metrics framework was developed using a structured methodology, incorporating expert feedback and literature review. It was evaluated based on its appropriateness, usability, and applicability through both a survey of Semantic Web researchers and its application to real-world temporal data modelling approaches. Key findings include the successful development of the TEDME-KG metrics framework, which comprises three goals, three metric categories, 7 metrics subcategories, and 16 metrics offering a systematic approach to assessing the modelling, uplifting, and querying of temporal data in KGs. The framework was validated through its application in the healthcare domain, where it evaluated different temporal data models, contributing to the enhancement of the FAIRVASC ontology. In addition to the metrics framework, the thesis also presents a quality assessment framework for healthcare data sources and a categorised list of temporal data modelling approaches, offering valuable insights for both researchers and practitioners. The results demonstrate that the TEDME-KG metrics framework provides a practical tool for evaluating temporal data modelling approaches, advancing the field by offering a structured method to address the complexities of temporal data in KGs. The evaluation results indicated that the TEDME-KG metrics framework is highly appropriate for its intended purpose, providing relevant and comprehensive metrics for temporal data modelling. In terms of general usability, Semantic Web researchers found the framework easy to understand and apply, with minimal complexity. Regarding applicability, the framework was demonstrated to be effective in real-world scenarios, particularly in healthcare, where it supported the evaluation of temporal data models in medication data. The TEDME-KG metrics framework, has shown its potential to be applied across a wide range of domains beyond healthcare, offering researchers and practitioners a systematic method to assess and improve temporal data models in knowledge graphs.