About the demo

We present GERBIL QA, an extension of the general entity annotation system GERBIL. It offers an easy-to-use web-based platform for the agile comparison of QA systems using multiple datasets and uniform measuring approaches. Gerbil QA supports three different setups. The user can either benchmark one of the already integrated QA systems, add another QA system via URI, or upload a JSON file with answers. In order to be used with GERBIL QA, a QA system has to implement the webservice interface as specified here. For easier usage, we provide a template that already implements this interface. Uploaded JSON files also have to be in the QALD-JSON format.

More information about the QA systems, with their papers and webservices, can be found here.