Natural Language-to-Query systems translate a natural language question into a formal query language such as SQL. Typically the translation results in a set of candidate query statements due to the ambiguity of natural language. Hence, an important aspect of NL-to-Query systems is to rank the query statements so that the most relevant query is ranked on top.

We propose a novel approach to significantly improve the query ranking and thus the accuracy of such systems. First, we use existing methods to translate the natural language question NL_in into k query statements and rank them. Then we translate each of the k query statements back into a natural language question NL_gen and use the semantic similarity between the original question NL_in and each of the k generated questions NL_gen to re-rank the output. Our experiments on two standard datasets, OTTA and Spider, show that this technique improves even strong state-of-the-art NL-to-Query systems by up to 9 percentage points. A detailed error analysis shows that our method correctly down-ranks queries with missing relations and wrong query types.

While this work is focused on NL-to-Query, our method could be applied to any other semantic parsing problems as long as a text generation method is available.