How I made AI Search “Whatever data you want to see…” for Find Me in Chicago.
If a user asks to see the “worst crashes,” they are using a word that means different things to different people. Does “worst” mean the crashes with the most fatalities, the most injuries, or the ones that happened at the highest speeds? A human analyst would ask clarifying questions, but what does an LLM with a single shot to guess do ?
Unfortunately, there is no easy way for the system to verify semantic correctness at runtime. This is why Text-to-SQL can be a trap for large enterprise database search. If a user asks “What was our revenue last quarter?”, raw Text-to-SQL generation will guess which table and column to join (e.g., gross_bookings vs net_revenue vs recognized_arr vs whatever). The query runs without error, but the number is completely wrong.
So it is also important to define common terms into consistent queries with Semantic Grounding. Add a Semantic grounding to force the LLM to target clear definitions for intermediate vocabulary. For example, worst is defined for Chicago Traffic Crashes as:
- worst/dangerous -> order by injuries_fatal DESC, injuries_incapacitating DESC, injuries_total DESC, crash_date DESC
It is open for debate if this is the best or only definition of “worst” for crash data, but it is grounded and will give more consistent results.
Similar to this is a more general concern defining what field to use when there are any multiple candidates and the user doesn’t specify or even know what to use. Global Placeholder rules can help with this, by keeping global SoQL Grounding date syntax rules separate from Schema Grounding rules. So keep all the date handling instructions in the global SoQL Grounding and just add this to the your SEMANTIC MAPPING RULES:
- [PRIMARY_DATE_FIELD] column for all date queries is “…”
- [PRIMARY_TYPE_FIELD] column for all type queries is “…”
- [PRIMARY_REVENUE_FIELD] column for all revenue queries is “…”
Another challenge is large, well defined lookup tables. These can take up a lot of space in your prompt, and if they relate to other data, the token consumption can get high and you risk increased non-determinism as the LLM starts losing focus. For Find Me in Chicago, this was a problem in defining Chicago Neighborhoods solved by a simple Macro Expansion routine. In my base SoQL prompt, I added this to LOCATION & ADDRESS RULES:
- Neighborhoods: Use __neighborhood__ = ‘Neighborhood Name’ (e.g. ‘Downtown’, ‘Loop’, ‘Englewood’).
This instructed the LLM to not worry about a very long list of Neighborhood boundaries. Before the query is actually executed, a simple RegEx expression for “__neighborhood__ = ‘Neighborhood Name’ “ does a simple lookup on Neighborhood Name, and adds the correct location boundaries to the query.