A Reliable Knowledge Graph Is What We Need

I've been spending time with knowledge-graph-based reasoning, and recently came across a paper that argues for a bottom-up approach to domain-specific models, grounding them in structured knowledge graphs rather than vast, unstructured text corpora. The more I sit with it, the more it feels like the missing layer beneath the LLM stack.

We've spent two years asking how do we make LLMs smarter? The answer the field has converged on is mostly "more data, more parameters, more compute." But there's a quieter answer that keeps surfacing in adjacent fields (biomedical informatics, finance, regulatory tech): structure the world first, then teach the model to reason over it. That second answer is what this piece is about.

Two Paradigms, One Goal

Most of the discourse treats LLMs as a single thing, but there are really two competing philosophies of how you teach a machine to know something.

The provocation of the recent literature is that bottom-up doesn't lose to scale. it complements it. You don't need a trillion-parameter model to outperform GPT-4 on medical reasoning. You need 32B parameters and a clean graph.

Where Top-Down LLMs Fall Short

LLMs trained top-down on general web corpora are remarkable, but their failure modes are predictable:

• Hallucination. Without explicit grounding, the model fills gaps with plausible-sounding fiction. In high-stakes domains (medicine, law, finance) plausibility is not enough.
• Shallow multi-step reasoning. Chain-of-thought helps, but the model is still navigating implicit relationships hidden inside its weights, not an explicit map of how concepts connect.
• Vector similarity is lossy. Most retrieval pipelines return only top-k chunks. The k+1th chunk (slightly less similar but factually critical) gets dropped on the floor. Unstructured text simply doesn't expose the deep abstractions a domain expert relies on.

A graph reasons differently. It traverses explicit, named relationships between nodes, so the path from symptom → disease → treatment is something the system can walk, not approximate.

What a Knowledge Graph Actually Looks Like

A knowledge graph is a network of entities (people, drugs, diseases, papers, places) connected by typed relationships. Below is a class-level view of the biomedical subset of Wikidata: every node is a kind of thing, every edge a verifiable claim about how those things relate.

Source: Wikimedia Commons

Anatomy of a Graph: Triplets, Types, Ontologies

A knowledge graph, stripped to its bones, is a collection of triplets:(subject, predicate, object), facts in atomic form. (aspirin, treats, headache) is a triplet. So is (aspirin, hasContraindication, hemophilia).

On top of triplets sit two layers that turn a flat soup of facts into something you can reason over:

• An ontology. A schema that names the kinds of things in your world (Drug, Disease, Symptom, Patient) and the kinds of relationships allowed between them (treats, contraindicates, dosage). Without an ontology, "treats" and "is used for" become different predicates and your graph fragments.
• An inference layer. Rules and constraints that derive new edges from existing ones. If A is a kind of B, and B treats C, then A treats C, except when overridden. The graph holds both the facts and the entailment rules, so you don't need to enumerate every consequence.

This is the part that's easy to under-appreciate. A graph isn't just data. it's data plus a typed grammar that tells you which questions are even askable. That grammar is what lets a small model punch above its weight: it doesn't need to learn that drugs treat diseases from a million sentences, because the schema already says so.

Case Study: QwQ-Med-3

The paper makes the case concretely in medicine. By fine-tuning QwQ-32B on 24,000 tasks generated from a medical knowledge graph, the authors produced QwQ-Med-3, a model that outperforms state-of-the-art systems on the ICD-Bench evaluation suite.

The crucial detail: the training data wasn't scraped text. It was synthesized from the graph itself: paths, multi-hop questions, entailment chains. The model learned to reason like a graph, not just to retrieve passages that mention the right keywords.

What this means in practice: the model is being asked questions whose answers it has never seen written down anywhere, but which are derivable by walking edges. That's a fundamentally different training signal than next-token prediction over Reddit threads. You're teaching the model the operation of reasoning, with the graph as the curriculum.

How Graph-Trained Models Reason Differently

When a vanilla LLM answers a medical question, it's doing high-dimensional pattern matching over the phrasing of its training data. When a graph-trained model answers the same question, it's effectively executing a traversal it learned to imitate, and the traversal is auditable.

Consider the question: "A 64-year-old patient on warfarin presents with a new prescription for ciprofloxacin. Any concerns?"

• Vanilla LLM path: retrieve passages discussing warfarin and ciprofloxacin together. Hope a relevant interaction is mentioned. Compose a fluent answer. The failure mode is plausible-but-wrong.
• Graph-trained path: warfarin → metabolizedBy → CYP2C9; ciprofloxacin → inhibits → CYP2C9; therefore ciprofloxacin → potentiates → warfarin; flag bleeding risk. The failure mode is silence. if a needed edge isn't in the graph, the model can say so rather than confabulate.

That second mode is what high-stakes domains actually want. Medicine, law, finance, and compliance need systems that can refuse to guess. A confident wrong answer is worse than a flagged uncertainty, and graph-grounded reasoning gives you a principled way to produce the latter.

This Isn't New: Google, 2012

If this feels familiar, it should. In 2012 Google announced its Knowledge Graph with a now-famous tagline:

things, not strings.

That was the moment search stopped being keyword matching and started being entity-centric. The graph powered structured fact retrieval, and was later extended with Knowledge Vault, neural matching, and BERT. The present LLM era didn't replace that infrastructure. it was layered on top of it.

A biodiversity knowledge graph, Rod Page. Source

The Wave Forming Right Now

A similar shift is happening again, this time at the application layer. A few moves I've been watching:

• Narrativa builds domain-specific KGs for finance, sports, entertainment, and weather, and uses them to power automated long-form journalism, including for the Wall Street Journal.
• Mem0 and Supermemory are building hybrid memory layers (graph plus vector) because pure embeddings forget the structure of what was said, and graphs alone forget the texture.
• Microsoft's GraphRAG. The research team at Microsoft published a system that builds community-detected knowledge graphs from arbitrary text and uses them as a retrieval substrate. On multi-hop questions over private corpora, it consistently outperforms vector-only RAG.
• LinkedIn's Economic Graph. Years before LLMs, LinkedIn was modeling members, companies, skills, schools, and titles as a single connected graph. Today every "people you may know", every job match, every skill recommendation runs over that graph, not raw text.
• Neo4j and the graph database renaissance. Tools that were enterprise back-office ten years ago are suddenly the substrate for AI applications, because they're the only thing that cleanly answers how is X connected to Y, and through whom?

The Hard Parts: Why Graphs Don't Just Win

If graphs are so powerful, why hasn't the world converged on them? Because building a good one is genuinely difficult, and the difficulty is concentrated in places that don't respect a startup's timeline.

• Curation is expensive. A high-quality biomedical KG took the community years and thousands of person-hours of expert review. You can't scrape it; you have to build it. The flip side is that once it exists, every model trained against it inherits that work.
• Schema evolution is brutal. The world changes: new drug classes, new regulations, new org structures. An ontology that was right two years ago can quietly become wrong, and migrating live data is the kind of work nobody volunteers for.
• Identity resolution. Is "J. Smith" in this paper the same person as "Jane Smith, MD" in that one? Cross-system entity resolution is the unglamorous core of every real-world graph project.
• Querying isn't free. Cypher and SPARQL are powerful, but you need someone fluent in them, or a layer that translates natural-language questions into traversals reliably. Both have a learning curve.

None of these are dealbreakers. They're the reasons graph adoption looks slow if you're measuring quarterly, and inevitable if you're measuring decade-over-decade.

The Practical Stack: KG + Vector + LLM

The most useful framing I've found is this: the three layers serve different purposes, and the best systems use all three.

• Knowledge graph: for verifiable facts, traversal, and constraints.
• Vector store: for fuzzy semantic recall over text the graph doesn't structure.
• LLM: for natural-language understanding, query translation, and synthesis of the retrieved structure into something a human can read.

The LLM stops being the source of truth and becomes a really good interpreter sitting on top of structured memory. That's a much more defensible architecture than "ask the model and hope."

A Different Shape of Superintelligence

The dominant narrative says scale is everything: more parameters, more tokens, more compute. The graph camp says something quieter: structure is everything. A model is only as smart as the world model it can reason over, and a graph is the cleanest expression of a world model we've built so far.

The most likely future, to me, isn't one giant generalist. It's a network of small, sharp, domain-expert models (each grounded in a high-quality knowledge graph) coordinated by a generalist that knows when to defer. That feels not just possible, but inevitable.

If you're building in this space, or you think I've got it wrong, I'd love to hear it.