← Go back

HOBBIT

Alumni project (December 2015 - November 2018)

Holistic Benchmarking of Big Linked Data

About the project

HOBBIT is a European project that develops a holistic open-source platform and industry-grade benchmarks for benchmarking big linked data.

Summary

Big Data is one of the key assets of the future. However, the cost and efforts required for introducing Big Data technology in a value chain is significant. Mastering the creation of value from Big Data will enhance European competitiveness will result in economic growth and jobs and will deliver societal benefit. To facilitate the use of Big Linked Data, the European Union funds a research and innovation project called "HOBBIT". A European consortium, led by the Institute for Applied Informatics (InfAI) e.V., aims to develop a holistic benchmarking platform for big linked data and corresponding industry-grade benchmarks.

Premises

A key step towards abolishing the barriers to the adoption and deployment of Big Data is to provide European companies with open benchmarking reports that allow them to assess the fitness of existing solutions for their purposes. Achieving this goal demands: The deployment of benchmarks on data that reflects reality within realistic settings. The provision of corresponding industry-relevant key performance indicators. The computation of comparable results on standardized hardware.

Goals

HOBBIT aims to address these tasks by means of a strong team composed of leading research institutes, large industry customers and innovative small and medium-sized enterprises. In particular, the consortium will aim to achieve the following goals: Define benchmarks for domains of industrial relevance in Europe that make use of Big Linked Data. Determine the key performance indicators for processing Big Linked Data by collaborating with stakeholders from industry and research. Create an open benchmarking platform to evaluate the performance of state-of-the-art systems on standardized hardware. Organize yearly evaluation campaigns, using the platform and the industry-defined KPIs.

Funding program
EU H2020 Research and Innovation Program

Publications

HOBBIT: A platform for benchmarking Big Linked Data

By Michael Röder, Denis Kuchelev, Axel-Cyrille Ngonga Ngomo

Data Science, 2020, #article

Benchmarking Question Answering Systems

By Ricardo Usbeck, Michael Röder, Michael Hoffmann, Felix Conrads, Jonathan Huthmann, Axel-Cyrille Ngonga Ngomo, Christian Demmler, Christina Unger

Semantic Web, 2019, #article

Dynamic Planning for Link Discovery

By Kleanthi Georgala, Daniel Obraczka, Axel-Cyrille Ngonga Ngomo

The Semantic Web, ESWC 2018, Lecture Notes in Computer Science, 2018, #inproceedings

Leopard --- a Baseline Approach to Attribute Prediction and Validation for Knowledge Graph Population

By René Speck, Axel-Cyrille Ngonga Ngomo

Web Semantics: Science, Services and Agents on the World Wide Web, 2018, #article

LIdioms: A Multilingual Linked Idioms Data Set

By Diego Moussallem, Mohamed Ahmed Sherif, Diego Esteves, Marcos Zampieri, Axel-Cyrille Ngonga Ngomo

The 11th edition of the Language Resources and Evaluation Conference, 7-12 May 2018, Miyazaki (Japan), 2018, #inproceedings

RDF2PT: Generating Brazilian Portuguese Texts from RDF Data

By Diego Moussallem, Marcos Zampieri Thiago Castro Ferreira, Maria Claudia Cavalcanti, Geraldo Xexéo, Mariana Neves, Axel-Cyrille Ngonga Ngomo

The 11th edition of the Language Resources and Evaluation Conference, 7-12 May 2018, Miyazaki (Japan), 2018, #inproceedings

On Extracting Relations using Distributional Semantics and a Tree Generalization

By René Speck, Axel-Cyrille Ngonga Ngomo

Proceedings of The 21th International Conference on Knowledge Engineering and Knowledge Management (EKAW'2018), 2018, #inproceedings

On the Effect of Geometries Simplification on Geo-spatial Link Discovery

By Abdullah Fathi Ahmed, Mohamed Ahmed Sherif, Axel-Cyrille Ngonga Ngomo

SEMANTiCS 2018 - Research Track, 2018, #inproceedings

Machine Translation using Semantic Web Technologies: A Survey

By Diego Moussallem, Matthias Wauer, Axel-Cyrille Ngonga Ngomo

Journal of Web Semantics, 2018, #article

Extending LargeRDFBench for Multi-Source Data at Scale for SPARQL Endpoint Federation

By Ali Hasnainb, Muhammad Saleem, Axel-Cyrille Ngonga Ngomo, Dietrich Rebholz-Schuhmann

ISWC Satellite, SSWS, 2018, #inproceedings

BENGAL: An Automatic Benchmark Generator for Entity Recognition and Linking

By Axel-Cyrille Ngonga Ngomo, Michael Röder, Diego Moussallem, Ricardo Usbeck, René Speck

Proceedings of the 11th International Conference on Natural Language Generation, Tilburg University, The Netherlands, November 5-8, 2018, 2018, #inproceedings

GERBIL - Benchmarking Named Entity Recognition and Linking consistently

By Michael Röder, Ricardo Usbeck, Axel-Cyrille Ngonga Ngomo

Semantic Web, 2018, #article

Introducing the HOBBIT platform into the Ontology Alignment Evaluation Campaign

By Ernesto Jiménez-Ruiz, Tzanina Saveta, Ondrej Zamazal, Sven Hertling, Michael Röder, Irini Fundulaki, Axel-Cyrille Ngonga Ngomo, Mohamed Ahmed Sherif, Amina Annane, Zohra Bellahsene, Sadok Ben Yahia, Gayo Diallo, Daniel Faria, Marouen Kachroudi, Abderrahmane Khiat, Patrick Lambrix, Huanyu Li, Maximilian Mackeprang, Majid Mohammadi, Maciej Rybinski, Booma Sowkarthiga Balasubramani, Cassia Trojahn

Proceedings of the Ontology Matching Workshop 2018, 2018, #inproceedings

MOCHA2018: The Mighty Storage Challenge at ESWC 2018

By Kleanthi Georgala, Mirko Spasić, Milos Jovanovik, Vassilis Papakonstantinou, Claus Stadler, Michael Röder, Axel-Cyrille Ngonga Ngomo

Semantic Web Evaluation Challenges, 2018, #inproceedings

Open Knowledge Extraction Challenge 2018

By René Speck, Michael Röder, Felix Conrads, Hyndavi Rebba, Catherine Camilla Romiyo, Gurudevi Salakki, Rutuja Suryawanshi, Danish Ahmed, Nikit Srivastava, Mohit Mahajan, Axel-Cyrille Ngonga Ngomo

Semantic Web Evaluation Challenge, 2018, #inproceedings

The Scalable Question Answering Over Linked Data (SQA) Challenge 2018

By Giulio Napolitano, Ricardo Usbeck, Axel-Cyrille Ngonga Ngomo

Semantic Web Challenges - 5th SemWebEval Challenge at ESWC 2018, Heraklion, Greece, June 3-7, 2018, Revised Selected Papers, 2018, #inproceedings

Characterizing Mention Mismatching Problems for Improving Recognition Results

By Jean Carlos Oliveira de Abreu, Renato Fileto, Axel-Cyrille Ngonga Ngomo, Michael Röder, Matthias Wittwer, Horacio Saggion

Proceedings of the 19th International Conference on Information Integration and Web-Based Applications & Services, 2017, #inproceedings

RADON - Rapid Discovery of Topological Relations

By Mohamed Ahmed Sherif, Kevin Dreßler, Panayiotis Smeros, Axel-Cyrille Ngonga Ngomo

Proceedings of The Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), 2017, #inproceedings

MOCHA2017: The Mighty Storage Challenge at ESWC 2017

By Kleanthi Georgala, Mirko Spasić, Milos Jovanovik, Henning Petzka, Michael Röder, Axel-Cyrille Ngonga Ngomo

Semantic Web Evaluation Challenge, 2017, #inproceedings

A High-Performance Approach to String Similarity using Most Frequent K Characters

By Andre Valdestilhas, Tommaso Soru, Axel-Cyrille Ngonga Ngomo

Proceedings of the Ontology Matching Workshop (co-located with ISWC 2017), 2017, #inproceedings

Open Knowledge Extraction Challenge 2017

By René Speck, Michael Röder, Sergio Oramas, Luis Espinosa-Anke, Axel-Cyrille Ngonga Ngomo

Semantic Web Evaluation Challenge, 2017, #inproceedings

Question Answering Over Linked Data: What is Difficult to Answer? What affects the F scores?

By Muhammad Saleem, Samaneh Nazari Dastjerdi, Ricardo Usbeck, Axel-Cyrille Ngonga Ngomo

Natural Language Interfaces workshop at ISWC, 2017, #inproceedings

Holistic and Scalable Ranking of RDF Data

By Axel-Cyrille Ngonga Ngomo, Michael Hoffmann, Ricardo Usbeck, Kunal Jha

2017 IEEE International Conference on Big Data, 2017, #inproceedings

LOG4MEX: A Library to Export Machine Learning Experiments

By Diego Esteves, Diego Moussallem, Tommaso Soru, Ciro Baron Neto, Jens Lehmann, Axel-Cyrille Ngonga Ngomo, Julio Cesar Duarte

WI '17 - IEEE/WIC/ACM International Conference on Web Intelligence, 2017, #inproceedings

An Evaluation of Models for Runtime Approximation in Link Discovery

By Kleanthi Georgala, Michael Hoffmann, Axel-Cyrille Ngonga Ngomo

Proceedings of the International Conference on Web Intelligence, 2017, #inproceedings

All That Glitters is not Gold -- Rule-Based Curation of Reference Datasets for Named Entity Recognition and Entity Linking

By Kunal Jha, Michael Röder, Axel-Cyrille Ngonga Ngomo

The Semantic Web. Latest Advances and New Domains: 14th International Conference, ESWC 2017, Proceedings, 2017, #inproceedings

Benchmarking RDF Storage Solutions with IGUANA

By Felix Conrads, Jens Lehmann, Muhammad Saleem, Axel-Cyrille Ngonga Ngomo

Proceedings of 16th International Semantic Web Conference - Poster & Demos, 2017, #inproceedings

MAG: A Multilingual, Knowledge-base Agnostic and Deterministic Entity Linking Approach

By Diego Moussallem, Ricardo Usbeck, Michael Röder, Axel-Cyrille Ngonga Ngomo

K-CAP 2017: Knowledge Capture Conference, 2017, #inproceedings

Eaglet -- a Named Entity Recognition and Entity Linking Gold Standard Checking Tool

By Kunal Jha, Michael Röder, Axel-Cyrille Ngonga Ngomo

The Semantic Web. Latest Advances and New Domains: 14th International Conference, ESWC 2017, Proceedings, 2017, #inproceedings

Ensemble Learning of Named Entity Recognition Algorithms using Multilayer Perceptron for the Multilingual Web of Data

By René Speck, Axel-Cyrille Ngonga Ngomo

K-CAP 2017: Knowledge Capture Conference, 2017, #inproceedings

GENESIS -- A Generic RDF Data Access Interface

By Timofey Ermilov, Diego Moussallem, Ricardo Usbeck, Axel-Cyrille Ngonga Ngomo

WI '17 - IEEE/WIC/ACM International Conference on Web Intelligence, 2017, #inproceedings

7th Open Challenge on Question Answering over Linked Data (QALD-7)

By Ricardo Usbeck, Axel-Cyrille Ngonga Ngomo, Bastian Haarmann, Anastasia Krithara, Michael Röder, Giulio Napolitano

Semantic Web Evaluation Challenge, 2017, #inproceedings

LargeRDFBench: A Billion Triples Benchmark for SPARQL Endpoint Federation

By Muhammad Saleem, Ali Hasnainb, Axel-Cyrille Ngonga Ngomo

Journal of Web Semantics (JWS), 2017, #inproceedings

Survey on Challenges of Question Answering in the Semantic Web

By Konrad Höffner, Sebastian Walter, Edgard Marx, Ricardo Usbeck, Jens Lehmann, Axel-Cyrille Ngonga Ngomo

Semantic Web Journal, 2017, #article

HOBBIT: Holistic Benchmarking for Big Linked Data

By Axel-Cyrille Ngonga Ngomo, Michael Röder

ESWC, EU networking session, 2016, #inproceedings

Scalable Link Discovery for Modern Data-Driven Applications

By Kleanthi Georgala

Proceedings of the The 15th International Semantic Web Conference (ISWC2016) 2016, Doctoral Consortium Track, Kobe, Japan, 17. October - 21. October 2016, 2016, #inproceedings

DBtrends : Exploring Query Logs for Ranking RDF Data

By Edgard Marx, Amrapali Zaveri, Diego Moussallem, Sandro Rautenberg

Proceedings of the 12th International Conference on Semantic Systems, 2016, #inproceedings

Efficient Source Selection For SPARQL Endpoint Query Federation

By Muhammad Saleem

PhD thesis at AKSW, University of Leipzig Germany, 2016, #inproceedings

Techreport for GERBIL 1.2.2 - V1

By Michael Röder, Ricardo Usbeck, Axel-Cyrille Ngonga Ngomo

, 2016, #techreport

Requirements to Modern Semantic Search Engines

By Ricardo Usbeck, Michael Röder, Peter Haase, Artem Kozlov, Muhammad Saleem, Axel-Cyrille Ngonga Ngomo

KESW, 2016, #inproceedings

MEX Interfaces: Automating Machine Learning Metadata Generation

By Diego Esteves, Pablo N. Mendes, Diego Moussallem, Julio Cesar Duarte, Amrapali Zaveri, Jens Lehmann, Ciro Baron Neto, Igor Costa, Maria Claudia Cavalcanti

12th International Conference on Semantic Systems (SEMANTiCS 2016), 12-15 September 2016, Leipzig, Germany, 2016, #inproceedings

SPARQL Querying Benchmarks

By Muhammad Saleem, Ricardo Usbeck, Michael Röder, Axel-Cyrille Ngonga Ngomo

Tutorial at ISWC, 2016, #inproceedings

Detecting Similar Linked Datasets Using Topic Modelling

By Michael Röder, Axel-Cyrille Ngonga Ngomo, Ivan Ermilov, Andreas Both

The Semantic Web. Latest Advances and New Domains: 13th International Conference, ESWC 2016, Heraklion, Crete, Greece, May 29 -- June 2, 2016, Proceedings, 2016, #inbook

Federated Query Processing: Challenges and Opportunities

By Axel-Cyrille Ngonga Ngomo, Muhammad Saleem

Keynote at PROFILES at Extended Semantic Web Conference (ESWC), 2016, #inproceedings

An Efficient Approach for the Generation of Allen Relations

By Kleanthi Georgala, Mohamed Ahmed Sherif, Axel-Cyrille Ngonga Ngomo

Proceedings of the 22nd European Conference on Artificial Intelligence (ECAI) 2016, The Hague, 29. August - 02. September 2016, 2016, #inproceedings

TAIPAN: Automatic Property Mapping for Tabular Data

By Ivan Ermilov, Axel-Cyrille Ngonga Ngomo

Proceedings of 20th International Conference on Knowledge Engineering and Knowledge Management (EKAW'2016), 2016, #inproceedings

DBtrends : Publishing and Benchmarking RDF Ranking Functions

By Edgard Marx, Amrapali Zaveri, Mofeed Mohammed, Sandro Rautenberg, Jens Lehmann, Axel-Cyrille Ngonga Ngomo, Gong Cheng

2nd International Workshop on Summarizing and Presenting Entities and Ontologies, co-located with the 13th Extended Semantic Web Conference, 2016, #inproceedings

UPSP: Unique Predicate-based Source Selection for SPARQL Endpoint Federation

By Ethem Cem Ozkan, Muhammad Saleem, Erdogan Dogdu, Axel-Cyrille Ngonga Ngomo

PROFILES at Extended Semantic Web Conference (ESWC), 2016, #inproceedings

HOBBIT: A platform for benchmarking Big Linked Data

By Michael Röder, Denis Kuchelev, Axel-Cyrille Ngonga Ngomo

Data Science, 2020, #article

Benchmarking Question Answering Systems

By Ricardo Usbeck, Michael Röder, Michael Hoffmann, Felix Conrads, Jonathan Huthmann, Axel-Cyrille Ngonga Ngomo, Christian Demmler, Christina Unger

Semantic Web, 2019, #article

Dynamic Planning for Link Discovery

By Kleanthi Georgala, Daniel Obraczka, Axel-Cyrille Ngonga Ngomo

The Semantic Web, ESWC 2018, Lecture Notes in Computer Science, 2018, #inproceedings

Leopard --- a Baseline Approach to Attribute Prediction and Validation for Knowledge Graph Population

By René Speck, Axel-Cyrille Ngonga Ngomo

Web Semantics: Science, Services and Agents on the World Wide Web, 2018, #article

LIdioms: A Multilingual Linked Idioms Data Set

By Diego Moussallem, Mohamed Ahmed Sherif, Diego Esteves, Marcos Zampieri, Axel-Cyrille Ngonga Ngomo

The 11th edition of the Language Resources and Evaluation Conference, 7-12 May 2018, Miyazaki (Japan), 2018, #inproceedings