Blueprints BP-06
Natural-language analytics interface
Every new question waits on an analyst and a dashboard build.
01 / Fit
Use it when
- New questions wait in an analyst queue for a query or a dashboard.
- The core metrics have agreed definitions, or can be given them.
- Users can judge whether an answer looks right when the plan is shown in words.
- The warehouse enforces permissions per user.
Avoid it when
- Metric definitions are disputed. The tool will answer with whichever one it finds.
- The same few questions are asked every week. A dashboard answers them faster.
- The figures feed regulatory or financial reporting without an analyst checking them.
02 / The diagram
Run it. Then run the failure path.
Select a node to see what it does, how it fails and what we measure.
- Question (Plain language). A plain-language question from someone who does not write SQL. Fails when: It uses a term the business defines three different ways. Safeguard: An ambiguous term triggers a clarifying question before any query runs.
- Semantic layer (Metrics, joins). Supplies the tables, joins and metric definitions the model is allowed to use. Fails when: A metric definition is missing or out of date. Safeguard: Definitions are owned and versioned. The model cannot invent a metric.
- Write SQL (Plan and query). Plans the analysis and writes the query, using verified examples as a guide. Fails when: The query runs and answers a slightly different question. Safeguard: The plan is shown in words beside the result, so a reader can catch it.
- Validate (Static checks). Checks the SQL before it runs: read-only, allowed tables, row and cost limits. Fails when: The query is valid and far too expensive. Safeguard: A cost estimate before execution. Anything over budget is rejected.
- Execute (Warehouse). Runs the query in the warehouse under the asker’s own permissions. Fails when: The warehouse is slow or times out. Safeguard: Timeouts, result caching and a row cap.
- Compose view (Table, chart). Chooses the components that fit the result and assembles the view. Fails when: A chart type misrepresents the data. Safeguard: Components come from a fixed, tested library. The model picks; it does not draw.
- Verified queries (Checked pairs). A library of question-and-SQL pairs that analysts have checked. Fails when: It covers last year’s questions. Safeguard: Answers marked correct are proposed for review and added.
- Clarify (Ask the user). Asks the user which meaning they intend, or offers the closest defined metric. Fails when: Too many questions make the tool slower than asking an analyst. Safeguard: One clarifying question at most, then a best effort with the assumption stated.
Read the diagram as text
- Question (Plain language). A plain-language question from someone who does not write SQL. Fails when: It uses a term the business defines three different ways. Safeguard: An ambiguous term triggers a clarifying question before any query runs.
- Semantic layer (Metrics, joins). Supplies the tables, joins and metric definitions the model is allowed to use. Fails when: A metric definition is missing or out of date. Safeguard: Definitions are owned and versioned. The model cannot invent a metric.
- Write sql (Plan and query). Plans the analysis and writes the query, using verified examples as a guide. Fails when: The query runs and answers a slightly different question. Safeguard: The plan is shown in words beside the result, so a reader can catch it.
- Validate (Static checks). Checks the SQL before it runs: read-only, allowed tables, row and cost limits. Fails when: The query is valid and far too expensive. Safeguard: A cost estimate before execution. Anything over budget is rejected.
- Execute (Warehouse). Runs the query in the warehouse under the asker’s own permissions. Fails when: The warehouse is slow or times out. Safeguard: Timeouts, result caching and a row cap.
- Compose view (Table, chart). Chooses the components that fit the result and assembles the view. Fails when: A chart type misrepresents the data. Safeguard: Components come from a fixed, tested library. The model picks; it does not draw.
- Verified queries (Checked pairs). A library of question-and-SQL pairs that analysts have checked. Fails when: It covers last year’s questions. Safeguard: Answers marked correct are proposed for review and added.
- Clarify (Ask the user). Asks the user which meaning they intend, or offers the closest defined metric. Fails when: Too many questions make the tool slower than asking an analyst. Safeguard: One clarifying question at most, then a best effort with the assumption stated.
03 / Components
Every component, and how it fails.
| Component | Job | Typical choices | Fails when | Safeguard |
|---|---|---|---|---|
| Question | A plain-language question from someone who does not write SQL. | Chat or search-style input with user identity attached | It uses a term the business defines three different ways. | An ambiguous term triggers a clarifying question before any query runs. |
| Semantic layer | Supplies the tables, joins and metric definitions the model is allowed to use. | Semantic layer with owned, versioned metric and join definitions | A metric definition is missing or out of date. | Definitions are owned and versioned. The model cannot invent a metric. |
| Write sql | Plans the analysis and writes the query, using verified examples as a guide. | Language model prompted with the schema, the definitions and verified examples | The query runs and answers a slightly different question. | The plan is shown in words beside the result, so a reader can catch it. |
| Validate | Checks the SQL before it runs: read-only, allowed tables, row and cost limits. | SQL parser with an allow-list, read-only enforcement and a cost estimate | The query is valid and far too expensive. | A cost estimate before execution. Anything over budget is rejected. |
| Execute | Runs the query in the warehouse under the asker’s own permissions. | Data warehouse with per-user permissions, timeouts and result caching | The warehouse is slow or times out. | Timeouts, result caching and a row cap. |
| Compose view | Chooses the components that fit the result and assembles the view. | Fixed component library for tables and charts, selected by the model | A chart type misrepresents the data. | Components come from a fixed, tested library. The model picks; it does not draw. |
| Verified queries | A library of question-and-SQL pairs that analysts have checked. | Reviewed library of question and SQL pairs | It covers last year’s questions. | Answers marked correct are proposed for review and added. |
| Clarify | Asks the user which meaning they intend, or offers the closest defined metric. | Single-turn clarification prompt offering the closest defined metrics | Too many questions make the tool slower than asking an analyst. | One clarifying question at most, then a best effort with the assumption stated. |
04 / Budgets and evals
What the design is tuned for.
Query correctness, time to answer. These are design targets, not results.
| Stage | Design target |
|---|---|
| Write SQL | Correctness over speed. Verified examples are retrieved for every question. |
| Validate | Static checks, so a bad query is rejected before it costs anything. |
| Execute | Time to answer. Cached results and row caps keep the common questions quick. |
| Clarify | One question at most, so the tool stays faster than asking an analyst. |
Eval plan 01
Golden set
Question and SQL pairs checked by analysts, with expected results on a fixed snapshot. It includes ambiguous questions that should trigger a clarification.
Eval plan 02
Release gates
Result correctness against the verified set, share of queries rejected by validation, and correct handling of the ambiguous questions.
Eval plan 03
Catching regressions
The verified set is replayed on every prompt, model or semantic-layer change. Answers that users mark wrong are reviewed and added.
05 / Design notes
Where the whiteboard and production differ.
- Most wrong answers come from metric definitions, not from SQL syntax.
- A query that runs and answers a slightly different question is the common failure, and it is the hardest one for a user to notice.
- Coverage of the semantic layer limits what can be answered. Extending it is ongoing work with an owner.
- People ask follow-up questions that depend on the previous answer, so conversation state needs the same validation as a first question.
- Showing the plan in words beside the result does more for trust than a higher accuracy figure.
06 / Deploy
Deploy this with us.
A pod can take BP-06 from this page to your stack. Tell us the workflow and we will tell you what changes for your systems.
You will talk to an engineer