Context-aware SQL generation

The challenge

Every data team knows the queue. How many active customers did we have last quarter, by region? It's a simple question with a slow answer, because the person asking doesn't know the schema and the person who knows the schema is busy.

Large language models look like an obvious shortcut: paste in the question, get SQL back. And for toy databases, that works. On a real warehouse, naive text-to-SQL fails in quiet, dangerous ways. A model that has only seen table names will guess. It joins on the column that looks right. It picks created_at when the business means the booking date. It counts rows when it should count distinct customers. Then it returns a perfectly formatted, completely wrong number.

That's the part that worried me. The failure isn't a syntax error you can catch — it's a plausible answer to a slightly different question. So the real problem was never generating SQL. It was generating SQL that reflects how this particular organization defines its own terms.

What I built

I treated the model as one component in a pipeline rather than an oracle. Instead of asking it to be clever, the system's job is to hand it the right context and then check its work.

• Schema metadata, not just a schema. Table and column names are only the start. Descriptions, relationships, and known quirks give the model the information a new analyst would get in their first week — and only the slice relevant to the question, so it isn't wading through every table at once.
• Business terminology. Words like "active," "revenue," or "churned" have house definitions. A glossary maps that vocabulary to the agreed-upon logic, so the query uses the organization's definition rather than the model's best guess.
• Organizational context. Conventions that live in people's heads — which sources are authoritative, which filters are always applied, how the calendar works — are written down and supplied alongside the question.
• Validation and review. Generated SQL is checked before anyone relies on it, and it's presented with a plain-language explanation of what it does so a person can confirm the intent. When a question is ambiguous, the better move is to ask a clarifying question instead of quietly picking an interpretation.

Why it mattered

The point of a tool like this is trust. A system that's right most of the time and confidently wrong the rest is arguably worse than no system, because people stop checking.

Context changes the model's job from guessing to applying known definitions. Review turns whatever uncertainty is left into a visible step instead of a hidden one. Together, they make the output something an analyst can sign off on rather than something they have to reverse-engineer.

It also changes where people spend their time. Less effort goes into translating a question into joins and filters, and more goes into the parts that need human judgment: is this the right question, and what does the answer mean?

Results and lessons

I'm deliberately not putting numbers on this page. Accuracy and time savings depend heavily on the data, the questions, and how you measure, and I'd rather describe what I learned than quote a figure without its context.

What I'm confident saying:

1. Context beats cleverness. Better metadata and clearer definitions moved the needle more reliably than prompt tricks.
2. A glossary is a product, not a spreadsheet. The hardest part wasn't technical — it was getting agreement on what words mean. Building the glossary surfaced inconsistencies people didn't know they had.
3. Show your work. Pairing a query with a plain-English explanation makes review fast enough that people actually do it.
4. Design for "I'm not sure." A clarifying question is a feature. A confident wrong answer is a bug, no matter how nicely it's formatted.

If I had to fit the whole project on a sticky note: text-to-SQL is a context problem wearing a language-model costume.