Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DBpedia queries on CoyPu endpoint #10

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Perevalov
Copy link
Collaborator

What was done:

  • Run all the queries on the endpoint https://skynet.coypu.org/dbpedia/
  • Identify the failed ones
  • Fix the failed ones
  • Re-run all the queries to get the updated answers

@RicardoUsbeck
Copy link
Collaborator

Dear Aleksandr, I understand that due to the change from Virtuoso to Apache Fuseki, some queries (actually a lot) needed changes. The answers should not change that drastically since they will change benchmarking results. I am not sure that this is the right way to go. What do you think? It would also change QALD-10 train results, or?

@Perevalov
Copy link
Collaborator Author

Dear Aleksandr, I understand that due to the change from Virtuoso to Apache Fuseki, some queries (actually a lot) needed changes. The answers should not change that drastically since they will change benchmarking results. I am not sure that this is the right way to go. What do you think? It would also change QALD-10 train results, or?

The number of queries that were changed is 14 (train) and 24 (test). For ca. 50% of them, the answers were the same. However, as I also re-run all the queries over the new endpoint, many answers have changed also (some of them maybe just have a different order, but still it is reflected in the files).

As discussed let us create git tags for each version of the dataset

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants