You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2026-07-27-ai-ready-data-semantic-layer-knowledge-graph-en.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -402,7 +402,7 @@ That property comes with a condition, though. "Tells you it can't answer" holds
402
402
403
403
Back to the table: the left column has a trap too, and it applies to the whole table rather than any single row. To make text-to-SQL work at all, they loaded the entire schema as context, and the authors noted that "isn't practical for larger datasets." In practice you can't do that, so text-to-SQL's real score sits below what's tabulated here, and the semantic layer's lead is wider than the table shows. What to take from this table, then, is the character of the failures rather than the absolute values: the side with modeling done in advance wins, and when it can't, it stays silent.
404
404
405
-
So what do you actually get under real conditions? There's an independent benchmark that measured the same problem on internal warehouses. [**BEAVER**](https://arxiv.org/abs/2409.02038), built by an MIT-centered team (Michael Stonebraker among the authors), is "the first text-to-SQL benchmark derived from private data warehouses," comprising 9,128 question-SQL pairs pulled from real query logs across 812 tables in 19 domains. The result: state-of-the-art agentic frameworks using the latest model (GPT-5.2) score **10.8%**. Same task as the 90% and 100% in the table above, and it splits 10.8% versus 90–100%. The reason is that public benchmarks are built from "well-organized schemas and simple question-SQL pairs" while internal warehouses hold "complex schemas, domain knowledge, and analytical queries requiring sophisticated structures and functions."
405
+
So what do you actually get under real conditions? There's an independent benchmark that measured the same problem on internal warehouses. [**BEAVER**](https://arxiv.org/abs/2409.02038), built by an MIT-centered team (Michael Stonebraker among the authors), is "the first text-to-SQL benchmark derived from private data warehouses," comprising 9,128 question-SQL pairs pulled from real query logs across 812 tables in 19 domains. The result: state-of-the-art agentic frameworks using the latest model (GPT-5.2) score **10.8%**. Same task as the 90% and 100% seen earlier, and it splits 10.8% versus 90–100%. The reason is that public benchmarks are built from "well-organized schemas and simple question-SQL pairs" while internal warehouses hold "complex schemas, domain knowledge, and analytical queries requiring sophisticated structures and functions."
406
406
407
407
The more important number in that paper isn't 10.8% but what comes next. Feed the model **all** the human-annotated answers to five subtasks as ground truth and it reaches **30.1%** (the paper's term is oracle hints — telling the model the intermediate steps it should have worked out itself). Fill in context by hand and you get 2.8x (the same direction as §1's 16.7%→54.2%), and it stops there. That's a different research team independently measuring, on the structured side, the two claims §1 established: codifying context improves answers, and it doesn't finish the job.
408
408
@@ -430,7 +430,7 @@ The stack diagram Databricks published alongside shows the arrangement at a glan
430
430
431
431
### 6.1 Joining Structured and Unstructured Into One Context Layer
432
432
433
-
Start with the most practical problem. Enterprise data splits into structured (the customer table in a warehouse) and unstructured (support logs, contracts), and real questions usually cross both: "has this customer complained recently, and is their contract renewal imminent?" The first half (renewal imminent) is a SQL query against a date column in the CRM table; the second half (complained) requires vector search over unstructured documents — support logs and email. The problem is **how to join the two into one answer**. Hand an agent a SQL tool and a vector search tool and tell it to figure out the combination, and the agent can't even be sure the two results refer to the same customer — if the CRM customer is `cust_id=12345` while the support log says only "Mr. Hong," the link establishing they're the same person exists nowhere in the data.
433
+
Start with the most practical problem. Enterprise data splits into structured (the customer table in a warehouse) and unstructured (support logs, contracts), and real questions usually cross both: "has this customer complained recently, and is their contract renewal imminent?" The first half (complained) requires vector search over unstructured documents — support logs and email; the second half (renewal imminent) is a SQL query against a date column in the CRM table. The problem is **how to join the two into one answer**. Hand an agent a SQL tool and a vector search tool and tell it to figure out the combination, and the agent can't even be sure the two results refer to the same customer — if the CRM customer is `cust_id=12345` while the support log says only "Mr. Hong," the link establishing they're the same person exists nowhere in the data.
434
434
435
435
One orthodox solution is **OBDA (Ontology-Based Data Access)**, now usually called a **Virtual Knowledge Graph (VKG)**, with [Ontop](https://ontop-vkg.org/guide/) — mentioned by name in §5.3 — as the reference implementation. The name is off-putting, the idea simple: **make data look like a graph without moving it.**
436
436
@@ -550,7 +550,7 @@ The first case is a multinational bank with 80,000 employees (a 2024 case study
550
550
551
551
The problem is harder than it looks, and here's why. One system writes "credit risk," another "lending risk," a third "borrower default probability," and whether those three point at the same thing is knowable only by a human reading context. Producing a risk total means comparing tens of thousands of these free-text entries one by one and grouping them as the same category, and with twenty systems the combinations explode. This is why data accumulated as free text is expensive — people read it and know; machines can't group it.
552
552
553
-
The solution was an ontology and a semantic layer. The core was **normalizing more than 20,000 free-text risk descriptions into 1,100 standardized taxonomies** — mapping scattered expressions onto standard concepts so that "credit risk" and "lending risk" both point at the same taxonomy node. That's exactly the middle rung of §3.1's ladder, the taxonomy, and the physical realization of §5's descent from CQ to certified definition. The results: integration timelines dropped **from one year to two months** across seven connected programs, and connecting 13 data providers and more than 40 systems while decommissioning 6 applications cut millions of dollars in operating and licensing costs. Eight core taxonomies are still shared across multiple enterprise applications.
553
+
The solution was an ontology and a semantic layer. The core was **normalizing more than 20,000 free-text risk descriptions into 1,100 standardized taxonomies** — mapping scattered expressions onto standard concepts so that "credit risk" and "lending risk" both point at the same taxonomy node. That's exactly the bottom rung of §3.1's ladder, the taxonomy, and the physical realization of §5's descent from CQ to certified definition. The results: integration timelines dropped **from one year to two months** across seven connected programs, and connecting 13 data providers and more than 40 systems while decommissioning 6 applications cut millions of dollars in operating and licensing costs. Eight core taxonomies are still shared across multiple enterprise applications.
554
554
555
555
These figures are self-reported by the consultancy that did the work. And whether "one year to two months" is purely the semantic layer's doing or partly the organizational and process improvements running alongside can't be separated from the case study alone — vendor success stories always carry this confounder.
556
556
@@ -711,9 +711,9 @@ One practical note on adoption order: most organizations are safer evolving **ri
711
711
712
712
Looking back, one argument ran through this whole post: **the essence of AI-ready data is context written into a machine-executable form, not cleanliness**, the approach splits three ways by data shape (unstructured → knowledge graphs, structured → semantic layers, metadata → catalogs), and all three converge on agents through one interface, MCP.
713
713
714
-
The starting point was §1's 16.7%→54.2%. Making context explicit with an ontology and mappings tripled accuracy, with the caveat attached that the human labor behind that explicitness was half the value. After that, this post did two things: dissected what gets written in and how, per path (§3–§6), and verified with numbers whether it pays off and where it breaks (§7).
714
+
The starting point was §1's 16.7%→54.2%. Making context explicit with an ontology and mappings tripled accuracy, with the caveat attached that the gain came from human labor spent making it explicit. After that, this post did two things: dissected what gets written in and how, per path (§3–§6), and verified with numbers whether it pays off and where it breaks (§7).
715
715
716
-
This post parts ways with vendor marketing on two claims. First, **context engineering is still expensive and still not universal** — §7's measured failure modes are the evidence, and Cerebras won by *not*writing context in, because it didn't need to. Second, **the break-even has nonetheless moved** — in a world where KET-RAG splits indexing into two tiers and LazyGraphRAG defers LLM use to query time, the 2022 judgment that graphs cost too much has to be recalculated. The latter carries the caveat of having no public implementation, and the direction still points one way. What IBM Watson proved by spending $4B was that context matters, not that acquiring context is impossible.
716
+
This post parts ways with vendor marketing on two claims. First, **context engineering is still expensive and still not universal** — §7's measured failure modes are the evidence, and Cerebras won by writing context only into its retrieval pipeline, because it had no need for a heavyweight context layer. Second, **the break-even has nonetheless moved** — in a world where KET-RAG splits indexing into two tiers and LazyGraphRAG defers LLM use to query time, the 2022 judgment that graphs cost too much has to be recalculated. The latter carries the caveat of having no public implementation, and the direction still points one way. What IBM Watson proved by spending $4B was that context matters, not that acquiring context is impossible.
717
717
718
718
So the answer to whether you should build a knowledge graph remains the shape of your data and your problem. But a 2026 caveat attaches: **the cost figures going into that judgment differ from last year's.** What produced that 3x was better context, not a bigger model, and the price of writing that context in is falling fast. The architect's job is redoing that arithmetic every year.
0 commit comments