Answer from the graph, or don’t answer at all

Four terms, defined up front

Grounded question answering is easy to follow once four pieces of vocabulary are clear.

Knowledge graph

A database organized as nodes (the things: a text, a verse, a person) connected by edges (the relationships between them: this teaching cites that verse, this scholar taught that student).

Typed graph

A knowledge graph in which every node and every edge carries a declared type. A node is not just “a thing” — it is specifically a Verse, a Teaching, or a Sage — so a query can ask precise questions like “which Teachings cite this Verse?”

Embedding

A list of numbers that represents the meaning of a piece of text, computed so that texts about similar things end up with similar numbers. Embeddings let a computer find passages related to a question even when the wording differs.

Large language model (LLM)

An AI system trained to produce fluent text. On its own, an LLM writes what is statistically plausible — which is not the same as what is true.

The pipeline, step by step

A graph-grounded system answers a question in four stages. The order matters: the model composes text last, after the evidence is already on the table.

1. The question is matched against the typed graph.

   The system computes an embedding of your question and compares it against the embeddings of every passage stored in the graph. Because the graph is typed, the search can also be narrowed structurally — only Teachings, only passages connected to a particular Verse or Sage.

2. Candidate passages come back with their sources attached.

   Retrieval returns actual stored text, and because citation is an edge in the graph, each passage arrives already linked to the work, author, and location it came from. Provenance is not looked up afterward; it travels with the passage.

3. The model composes an answer only from what returned.

   The LLM is instructed to write its answer using the retrieved passages and nothing else, and to cite each passage it uses. Its job shifts from “know the answer” to “summarize this evidence faithfully.”

4. Or it declines.

   If retrieval returns nothing relevant, the correct output is a plain statement that the graph holds no source for this question. A grounded system treats “I don’t have a source for that” as a valid, first-class answer.

Why cite-or-decline beats fluent guessing

An unconstrained LLM produces confident prose whether or not it actually has the facts. When it fills a gap with plausible invention, that failure is called a hallucination — and it is dangerous precisely because it reads exactly like a correct answer. The cite-or-decline rule changes the economics of trust in three ways:

• Verification becomes cheap. Every sentence in a grounded answer points at a passage you can open and read. Checking the answer takes seconds, not a research session.
• Failure becomes visible. A guessing system fails silently, in fluent prose. A grounded system fails out loud, by declining — so you always know which kind of answer you are holding.
• Coverage becomes honest. A declined question is a signal about what the graph does not yet contain, which tells the builders exactly what to add next. A hallucinated answer hides that gap.

The trade is straightforward: a grounded system answers fewer questions, and the answers it does give are ones you can check.

A working example you can try

This is not a hypothetical design. OpenRabbi is a question-answering assistant for classical Jewish texts built on exactly this pipeline. It runs on a typed knowledge graph of 157,143 nodes and 215,426 relationships, including 17,147 cited teachings — every one of them stored with its Hebrew text, its citation, and its embedding — alongside 33,323 verses, 15,910 teaching-to-verse citation edges, and 2,504 rabbi-to-rabbi edges joining 2,185 sages.

Ask it a question and it either answers from those cited teachings, with the sources shown, or it tells you the graph has no source. The graph's structure and the measurements behind these numbers are documented on the AI Sages page at jew.tech.

Part of a larger project

This page is one node in a network of sites exploring where Judaism and computer science meet — each one a plain-language explanation of a real, running system.