Skip to content

Commit ab8ea05

Browse files
Merge pull request #55 from bits-bytes-nn/post/ai-ready-data-korean-prose
content: rewrite prose that read as translated Korean
2 parents 59ffad8 + 27f79a5 commit ab8ea05

2 files changed

Lines changed: 131 additions & 129 deletions

File tree

_posts/2026-07-27-ai-ready-data-semantic-layer-knowledge-graph-en.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -402,7 +402,7 @@ That property comes with a condition, though. "Tells you it can't answer" holds
402402

403403
Back to the table: the left column has a trap too, and it applies to the whole table rather than any single row. To make text-to-SQL work at all, they loaded the entire schema as context, and the authors noted that "isn't practical for larger datasets." In practice you can't do that, so text-to-SQL's real score sits below what's tabulated here, and the semantic layer's lead is wider than the table shows. What to take from this table, then, is the character of the failures rather than the absolute values: the side with modeling done in advance wins, and when it can't, it stays silent.
404404

405-
So what do you actually get under real conditions? There's an independent benchmark that measured the same problem on internal warehouses. [**BEAVER**](https://arxiv.org/abs/2409.02038), built by an MIT-centered team (Michael Stonebraker among the authors), is "the first text-to-SQL benchmark derived from private data warehouses," comprising 9,128 question-SQL pairs pulled from real query logs across 812 tables in 19 domains. The result: state-of-the-art agentic frameworks using the latest model (GPT-5.2) score **10.8%**. Same task as the 90% and 100% in the table above, and it splits 10.8% versus 90–100%. The reason is that public benchmarks are built from "well-organized schemas and simple question-SQL pairs" while internal warehouses hold "complex schemas, domain knowledge, and analytical queries requiring sophisticated structures and functions."
405+
So what do you actually get under real conditions? There's an independent benchmark that measured the same problem on internal warehouses. [**BEAVER**](https://arxiv.org/abs/2409.02038), built by an MIT-centered team (Michael Stonebraker among the authors), is "the first text-to-SQL benchmark derived from private data warehouses," comprising 9,128 question-SQL pairs pulled from real query logs across 812 tables in 19 domains. The result: state-of-the-art agentic frameworks using the latest model (GPT-5.2) score **10.8%**. Same task as the 90% and 100% seen earlier, and it splits 10.8% versus 90–100%. The reason is that public benchmarks are built from "well-organized schemas and simple question-SQL pairs" while internal warehouses hold "complex schemas, domain knowledge, and analytical queries requiring sophisticated structures and functions."
406406

407407
The more important number in that paper isn't 10.8% but what comes next. Feed the model **all** the human-annotated answers to five subtasks as ground truth and it reaches **30.1%** (the paper's term is oracle hints — telling the model the intermediate steps it should have worked out itself). Fill in context by hand and you get 2.8x (the same direction as §1's 16.7%→54.2%), and it stops there. That's a different research team independently measuring, on the structured side, the two claims §1 established: codifying context improves answers, and it doesn't finish the job.
408408

@@ -430,7 +430,7 @@ The stack diagram Databricks published alongside shows the arrangement at a glan
430430

431431
### 6.1 Joining Structured and Unstructured Into One Context Layer
432432

433-
Start with the most practical problem. Enterprise data splits into structured (the customer table in a warehouse) and unstructured (support logs, contracts), and real questions usually cross both: "has this customer complained recently, and is their contract renewal imminent?" The first half (renewal imminent) is a SQL query against a date column in the CRM table; the second half (complained) requires vector search over unstructured documents — support logs and email. The problem is **how to join the two into one answer**. Hand an agent a SQL tool and a vector search tool and tell it to figure out the combination, and the agent can't even be sure the two results refer to the same customer — if the CRM customer is `cust_id=12345` while the support log says only "Mr. Hong," the link establishing they're the same person exists nowhere in the data.
433+
Start with the most practical problem. Enterprise data splits into structured (the customer table in a warehouse) and unstructured (support logs, contracts), and real questions usually cross both: "has this customer complained recently, and is their contract renewal imminent?" The first half (complained) requires vector search over unstructured documents — support logs and email; the second half (renewal imminent) is a SQL query against a date column in the CRM table. The problem is **how to join the two into one answer**. Hand an agent a SQL tool and a vector search tool and tell it to figure out the combination, and the agent can't even be sure the two results refer to the same customer — if the CRM customer is `cust_id=12345` while the support log says only "Mr. Hong," the link establishing they're the same person exists nowhere in the data.
434434

435435
One orthodox solution is **OBDA (Ontology-Based Data Access)**, now usually called a **Virtual Knowledge Graph (VKG)**, with [Ontop](https://ontop-vkg.org/guide/) — mentioned by name in §5.3 — as the reference implementation. The name is off-putting, the idea simple: **make data look like a graph without moving it.**
436436

@@ -550,7 +550,7 @@ The first case is a multinational bank with 80,000 employees (a 2024 case study
550550

551551
The problem is harder than it looks, and here's why. One system writes "credit risk," another "lending risk," a third "borrower default probability," and whether those three point at the same thing is knowable only by a human reading context. Producing a risk total means comparing tens of thousands of these free-text entries one by one and grouping them as the same category, and with twenty systems the combinations explode. This is why data accumulated as free text is expensive — people read it and know; machines can't group it.
552552

553-
The solution was an ontology and a semantic layer. The core was **normalizing more than 20,000 free-text risk descriptions into 1,100 standardized taxonomies** — mapping scattered expressions onto standard concepts so that "credit risk" and "lending risk" both point at the same taxonomy node. That's exactly the middle rung of §3.1's ladder, the taxonomy, and the physical realization of §5's descent from CQ to certified definition. The results: integration timelines dropped **from one year to two months** across seven connected programs, and connecting 13 data providers and more than 40 systems while decommissioning 6 applications cut millions of dollars in operating and licensing costs. Eight core taxonomies are still shared across multiple enterprise applications.
553+
The solution was an ontology and a semantic layer. The core was **normalizing more than 20,000 free-text risk descriptions into 1,100 standardized taxonomies** — mapping scattered expressions onto standard concepts so that "credit risk" and "lending risk" both point at the same taxonomy node. That's exactly the bottom rung of §3.1's ladder, the taxonomy, and the physical realization of §5's descent from CQ to certified definition. The results: integration timelines dropped **from one year to two months** across seven connected programs, and connecting 13 data providers and more than 40 systems while decommissioning 6 applications cut millions of dollars in operating and licensing costs. Eight core taxonomies are still shared across multiple enterprise applications.
554554

555555
These figures are self-reported by the consultancy that did the work. And whether "one year to two months" is purely the semantic layer's doing or partly the organizational and process improvements running alongside can't be separated from the case study alone — vendor success stories always carry this confounder.
556556

@@ -711,9 +711,9 @@ One practical note on adoption order: most organizations are safer evolving **ri
711711

712712
Looking back, one argument ran through this whole post: **the essence of AI-ready data is context written into a machine-executable form, not cleanliness**, the approach splits three ways by data shape (unstructured → knowledge graphs, structured → semantic layers, metadata → catalogs), and all three converge on agents through one interface, MCP.
713713

714-
The starting point was §1's 16.7%→54.2%. Making context explicit with an ontology and mappings tripled accuracy, with the caveat attached that the human labor behind that explicitness was half the value. After that, this post did two things: dissected what gets written in and how, per path (§3–§6), and verified with numbers whether it pays off and where it breaks (§7).
714+
The starting point was §1's 16.7%→54.2%. Making context explicit with an ontology and mappings tripled accuracy, with the caveat attached that the gain came from human labor spent making it explicit. After that, this post did two things: dissected what gets written in and how, per path (§3–§6), and verified with numbers whether it pays off and where it breaks (§7).
715715

716-
This post parts ways with vendor marketing on two claims. First, **context engineering is still expensive and still not universal** — §7's measured failure modes are the evidence, and Cerebras won by *not* writing context in, because it didn't need to. Second, **the break-even has nonetheless moved** — in a world where KET-RAG splits indexing into two tiers and LazyGraphRAG defers LLM use to query time, the 2022 judgment that graphs cost too much has to be recalculated. The latter carries the caveat of having no public implementation, and the direction still points one way. What IBM Watson proved by spending $4B was that context matters, not that acquiring context is impossible.
716+
This post parts ways with vendor marketing on two claims. First, **context engineering is still expensive and still not universal** — §7's measured failure modes are the evidence, and Cerebras won by writing context only into its retrieval pipeline, because it had no need for a heavyweight context layer. Second, **the break-even has nonetheless moved** — in a world where KET-RAG splits indexing into two tiers and LazyGraphRAG defers LLM use to query time, the 2022 judgment that graphs cost too much has to be recalculated. The latter carries the caveat of having no public implementation, and the direction still points one way. What IBM Watson proved by spending $4B was that context matters, not that acquiring context is impossible.
717717

718718
So the answer to whether you should build a knowledge graph remains the shape of your data and your problem. But a 2026 caveat attaches: **the cost figures going into that judgment differ from last year's.** What produced that 3x was better context, not a bigger model, and the price of writing that context in is falling fast. The architect's job is redoing that arithmetic every year.
719719

0 commit comments

Comments
 (0)