How I made AI Search “Whatever data you want to see…” for Find Me in Chicago.
Human language is built on ambiguity. Ambiguity is not a bug to be fixed in natural language; it is an essential feature of human communication. As Ludwig Wittgenstein observed in his Philosophical Investigations (S43), “the meaning of a word is its use in the language.” Under this view, language is an open-ended “game” where meaning shifts dynamically with context. Hardcoded systems that attempt to account for every possible combination of human phrasing are an exercise in futility. The rigid boundaries required by text parsers cannot be drawn over natural language because, as Wittgenstein remarked (S68), “none have been drawn.”
Even simple user requests require an impressive series of semantic reasoning that LLMs have shown to excel. A user might ask cleanly: “Show me a line chart trending hit-and-run crashes on Friday in Wicker Park over the last 2 years by month.” Or they might throw bizarre grammar, typos, and conversational fluff at the prompt: “Showy me a of hit run crashes on fri in Wicker Pk for two years brt group it by month pretty please. And maybe a line chart What tacos for dinner?”. You will never extract meaning from that using generic RegEx.
Even with erratic phrasing, LLMs strip away the noise and map the intent:
- Recognize implicitly this requires $group and count(*)
- Map “hit-and-run” or “hit run” to hit_and_run_i = ‘Y’
- Map “Friday” or “fri” to date_extract_dow(crash_date) = 5 (0 = Sunday)
- Map “Wicker Park” or “Wicker Pk” to __neighborhood__ macro expansion
- Map “last 2 years” or “for two years” to date_diff_d(…) <= 730
- Map “by month” or “grouped by month” to date_trunc_ym(crash_date)
- Enforce chart rules to constrain $select to 2 fields and validate $order against group keys or aggregates
- Ignore conversational noise and typos things like “Show me”, “Showy me a of”, “brt”, “pretty please, “What tacos for dinner?”
The semantic foundation is similar to the Distributional Hypothesis, proposed by the linguist J.R. Firth. Influenced by Wittgenstein’s philosophy of language he remarked in 1957 “You shall know a word by the company it keeps.” rejecting the idea that words possess intrinsic, ideal definitions independent of their use in context.
By representing tokens as vectors within high-dimensional space, models capture operational meaning mathematically. Combine this with transformer attention, and ambiguity can be transformed into reasonable database queries using Firth’s principle. The ideas may be old, but the ability for LLM’s to execute on these principles is the revolution.