The Plumb Line

What it is

A discernment tool for an ordinary believer who has just heard something that sounded off and has no theologian on speed-dial. You give it a teaching in English, Ukrainian, or Russian; it decomposes the claims, retrieves the Scripture, creeds, and confessions that bear on them, and returns a grounded reading: universally rejected, contested (a family difference, not heresy), sound, or — when it cannot ground a verdict — an honest refusal that sends you to your church and elders.

It tests the teaching, never the person. It never names a false teacher and never rules on anyone's salvation. A difference between historic traditions — baptism, the Lord's Supper, election — is labeled contested and shown with both sides, because the whole failure mode of a tool like this is weaponizing denominational difference into a heresy charge.

The Scripture layer is the real thing: the Ukrainian is Іван Огієнко's translation and the Russian is the Synodal 1876, pulled verbatim from a machine-readable source — not the English verses run through a translator. When it says «нехай буде проклятий» for Galatians 1:8, that is the text a Ukrainian believer actually reads.

This is the enhanced successor to an earlier hard-coded mock. That first artifact simulated the idea with canned questions; this one runs the actual pipeline — retrieval, a cited model stance, and a deterministic verdict — over whatever you paste.

The design world — a mason's plumb line, hung against the true vertical

A plumb line is the oldest instrument of judgment there is: a weight on a cord that finds true vertical by gravity alone, indifferent to the wall it hangs beside. The whole page is built around that one honest object — a brass bob on a cord that settles on load, a fixed reference the eye trusts before it reads a word. The name carries in three registers: Plumbline, виска, отвес.

Everything reverent and nothing flashy. The measure is set against a dark ground; the readings land on warm paper. Verdicts wear sober, earned colors — oxblood for what the whole church rejects, ochre for a contested family difference, olive for the sound, slate for an honest refusal — never a traffic light, because this is not a pass/fail buzzer.

Unlike a chatbot that answers in its own voice, Plumbline's page is a reading instrument: the model's words are quoted and cited, the verdict is a stamped label the model was never allowed to write, and the honest thing the tool can say most often is 'I can't ground this — ask your church.'

How it was evaluated

A golden set of 15 cases across three languages and all five verdicts was run through the pipeline and scored by a deterministic harness — tier-correct, a no-mislabel gate (a difference must never be flagged heresy), no person-judgments, and refusal calibration. Result: 15/15, with a mislabel rate of 0 and both out-of-corpus questions correctly refused.

Because the funded API key was spent mid-build, the model stage was run on a Claude subscription instead — fresh subagents fed the byte-identical production prompts, blind to the answer key, with the deterministic aggregator and gates scoring their output. Same model family, same prompts; an honest stand-in for the raw API path, clearly labeled as such. The deployed site's live 'analyze your own text' path is that same engine and goes live the moment a funded key is added; until then every on-page example is a genuine engine output, not a mock.

The sharpest result is the negative one: infant baptism, believer's baptism, a memorialist Lord's Supper, and unconditional election all came back contested — never heresy — with both sides named. A tool that only caught heresies would be easy; one that refuses to weaponize a difference is the actual product.

golden cases passed (English + Ukrainian + Russian), mislabel rate 0

15/15

verdicts emitted by the language model — a deterministic aggregator assigns every one

0

languages, with real Огієнко + Synodal Scripture, never translated from English

3

cited source units (Scripture, creeds, confessions) and human-curated registry entries

24 / 10

The whole guarantee in one function: the model can only propose a stance; 'universally rejected' requires a cited Scripture (R0) or ecumenical-council (R1) warrant of a matching human-registry entry, and nothing else can escalate.

def aggregate(stance, best_unit_score, cand_ids):
    mid = stance.matched_registry_id
    # a difference is grounded by the registry itself — and is NEVER heresy
    if stance.match_kind == "contested_difference" and mid in CONTESTED_BY_ID:
        return {"tier": "CONTESTED", "entry": CONTESTED_BY_ID[mid]}
    # 'universally rejected' needs a CITED R0/R1 warrant of that anathema entry
    if stance.match_kind == "affirms_anathema" and mid in ANATH_BY_ID:
        entry = ANATH_BY_ID[mid]
        warrant = set(entry["warrant_source_ids"])
        cited = [s for s in stance.supporting_source_ids if s in warrant]
        rungs = {UNITS_BY_ID[s]["rung"] for s in cited}
        if cited and (rungs & {"R0", "R1"}):
            return {"tier": "UNIVERSALLY_REJECTED", "entry": entry}
        return {"tier": "INSUFFICIENT_GROUNDING"}  # matched a label but nothing grounded it
    # no registry match: refuse rather than guess
    return {"tier": "NO_MATCH"}

What the AI learned

• The trust comes from what the model is forbidden to do, not from prompting it to be careful. Moving the verdict out of the model and into a deterministic rule over a human-curated registry is the entire product; the LLM is demoted to a cited evidence-reporter.
• Retrieval in a morphologically rich language breaks naive keyword search — Ukrainian «багатство», «багатіти», «багатий» share no exact token. A crude 5-character Slavic stem restored recall enough to prove the pipeline, and named the real next step (dense multilingual embeddings).
• The sacred layer cannot be approximated. Reconstructing a verse from memory or translating it from English would quietly falsify the one thing a believer checks first; the Огієнко and Synodal texts had to be the actual published words, pulled from source and flagged for a pastor's confirmation.
• The hardest correctness bar is the false positive, not the true one. Any tool can shout heresy; the discipline is labeling a genuine denominational difference as contested and refusing when the ground is thin — the no-mislabel gate, not the catch rate, is the eval that matters.
• An eval can be run honestly on a subscription with no API key: split each model call into a deterministic 'build the prompt' half and a subagent 'answer it' half, keep the subagent blind to the answer key, and let plain code do the scoring.