Back to previous page
Conference Publication

An In-Depth Benchmarking of Text-to-SQL Systems

AUTHORS:
ATHENA, Greece
Georgia Koutrika
ADDITIONAL AUTHORS:
O. Gkini, T. Belmpas, Y. Ioannidis
PUBLISHED IN:   
ACM SIGMOID 2021
DATE:   
February 12, 2021
Read full article

Text-to-SQL systems allow users to explore relational databases by posing free-form queries, alleviating the need for using structured languages, such as SQL. Although numerous systems have been developed so far, existing system evaluations lack in rigour. In this work, we build a text-to- SQL benchmark that covers different classes of queries, and we evaluate the effectiveness of several systems in the field. To evaluate system efficiency, we measure execution time and resource consumption for the different query classes. Our comprehensive evaluation aims at filling in a big gap in understanding the capabilities and boundaries of existing systems and it reveals several open challenges.

Download files:

Get in touch

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form, try again please.