Back to previous page
Conference Publication

Improving NL-to-Query Systems through Re-ranking of Semantic Hypothesis

ZHAW, Switzerland
Ursin Brunner
ZHAW, Switzerland
Kurt Stockinger
Pius von Däniken, Jan Deriu, Eneko Agirre, Mark Cieliebak
accepted in:
5th International Conference on Natural Language and Speech Processing (ICNLSP 2022), December 16 & 17, 2022
Yet to be published
November 17, 2022
Read full article

Natural Language-to-Query systems translate a natural language question into a formal query language such as SQL. Typically the translation results in a set of candidate query statements due to the ambiguity of natural language. Hence, an important aspect of NL-to-Query systems is to rank the query statements so that the most relevant query is ranked on top.

We propose a novel approach to significantly improve the query ranking and thus the accuracy of such systems. First, we use existing methods to translate the natural language question NL_in into k query statements and rank them. Then we translate each of the k query statements back into a natural language question NL_gen and use the semantic similarity between the original question NL_in and each of the k generated questions NL_gen to re-rank the output. Our experiments on two standard datasets, OTTA and Spider, show that this technique improves even strong state-of-the-art NL-to-Query systems by up to 9 percentage points. A detailed error analysis shows that our method correctly down-ranks queries with missing relations and wrong query types.

While this work is focused on NL-to-Query, our method could be applied to any other semantic parsing problems as long as a text generation method is available.

Will be available soon to download

Get in touch

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form, try again please.