The graphitron-model house cleaning party: relation descriptions are argument transcripts, so nobody reads them and the same fact gets a second relation

A grain is what one row of a relation is about. graphql_field is at the grain of one field of one type in one graph; sql_column is at the grain of one column of one table. The store already has this concept in two places without naming it as data: a relation’s primary key is its grain expressed as columns, and the first sentence of its comment is its grain expressed as prose, which GrainSentence already extracts on that convention.

The grain roster is not guesswork, because the keys already state most of it. Across the 161 tables in the schema, 141 declare a primary key and those keys fall into a short head:

Twelve key shapes cover 74 of the 141 keyed tables. The tail is long, 75 distinct shapes in all, and part of that tail is the problem rather than the domain: a shape used once may be a genuine grain or may be a relation that never decided what it was about.

The twenty tables in this schema with no declared primary key are exactly the twenty registered materialization targets. Not mostly. Exactly, both directions.

That is not a coincidence and the register gate already half explains it: most of those grains include a meaningfully nullable column, and H2 refuses a primary key over one, which is why each target has to declare an index instead. Read the other way round it says something sharper. A relation that cannot state its grain as a key is a relation whose grain is conditional, one row per this except when that, in which case per something else. Conditional grain is what makes a relation impossible to key, hard to index, and expensive to read, and every one of them ended up with a registration in front of it.

So declaring grain as data is not only a documentation change. It is the check that would have made those twenty visible as modelling defects before each became a materialization decision.

One row per fact gatherer, keyed on its name, and each row names the Java class that is the gatherer. Naming the class is what stops this from becoming a second vocabulary: a gate loads each named class, so a roster that points at SdlFactCapture cannot quietly outlive it. Columns: the name and the class, nothing else. Named meta_gatherer rather than meta_owner because the store’s naming discipline names a relation for what its rows are, never for their role, and a gatherer is a gatherer whether or not anything points at it; "owner" survives as the role-named foreign-key column on meta_relation, the same way meta_family_bridge names its prefix columns for their roles. Not meta_crawler either: the tree uses "crawler" consistently for the corpus-reading transcription passes and "gatherer" for the ownership concept, and the derivation gatherer crawls nothing.

What a gatherer reads lives in two junction relations rather than in columns, because both facts are set-valued and absence is a fact worth stating as no row rather than as NULL. meta_gatherer_corpus says which corpora a gatherer reads: CatalogFactCapture reads two (the jOOQ catalog and the declared classpath), the derivation gatherer reads none, and a scalar column could state neither. A crawler is precisely a gatherer with at least one corpus row, which makes this junction the store’s definition of a word the fact model page already leans on. meta_gatherer_dependency says whose rows a gatherer may read, declared rather than derived: the corpus gatherers carry no edges, which states their mutual independence, and the derivation gatherer depends on every one of them, which is what "runs last" means once it stops being a position. There is no run-order column: the order gatherers run in is FactCapture.capture’s statement order, a fact of the code this model would only transcribe, and an ordinal would flatten "these are independent" into a total order the model does not have. The dependency roster drives no execution; it is a declaration a gate holds the views to. `meta_corpus itself is the tiny roster both junctions and meta_grain key into: the corpus name and a one-sentence definition.

The last owner is a real gatherer with no corpus rows. The derivation gatherer runs after every corpus gatherer has finished, which is the earliest point at which a rule crossing families has all its inputs. It owns the intent_ relations, and it owns materializing the grain tables the views and the queries above it stand on. That is what meta_materialize becomes: not a mechanism of its own standing outside the ownership rule, but one owner’s refresh plan, the same kind of thing any other gatherer would hold for its own family.

One row per grain, keyed on its name. Columns: the name; a one-sentence statement of what one instance is, length-checked; the canonical key shape as a column list; and the corpus the grain lives in, a foreign key into meta_corpus.

One row per relation the migration has reached, keyed on the relation name, covering views as well as tables. Every column NOT NULL.

A relation nobody has reached yet has no row, and that absence is the pending state. This supersedes the nullable-with-state design round 1 settled, with a third option neither review round weighed: no pending representation at all. The store’s census (meta_relation_family over INFORMATION_SCHEMA) already says which relations exist, so which ones lack a meta_relation row is a query, and a row that exists answers all four questions or fails a NOT NULL. What the pending state bought, "a relation joining the store is a decision recorded rather than a row appearing", the absence form buys identically through the roster gate below, without a state column, without five two-way CHECKs, and without 276 content-free INSERT rows in a file this item exists to slim down. It also dissolves the mechanical half-migration: there is no verbatim-tail-into-rationale step, a row exists exactly when its four questions have been answered, which is what the migration section below already said the real work is.

The ratchet pins the roster, not a count. The gate test carries the list of relations that have no meta_relation row yet, frozen at migration start and only ever shrinking. A count admits a swap (declare one relation, sneak in a new undeclared one, number unchanged); a shrink-only list does not, and it is the exemption polarity the schema gates already use throughout. A new relation is on no frozen list, so it fails the build until someone declares it, mid-migration and after alike. The item is not done while the list is non-empty; once it empties, the gate is the both-ways equality the Gates section states.

The owner column is where this meets the ownership rule on the fact model page. That page states that a view reading one family belongs to that family and is owned by that family’s gatherer, and that a rule whose facts cross families is owned by the gatherer that runs last. It also states, in its own enforcement line, that nothing checks this. The owner_name column plus the declared dependency edges are what make it checkable, and ViewReferences already parses stored view definitions, so the gate walks each declared view’s reads: every relation a view reads must be owned by the view’s own owner or by a gatherer in that owner’s declared dependency set. The gate covers views; the hand-written producers read through jOOQ code no stored definition exposes, and CaptureCorpusIsolationTest keeps covering that side dynamically.

Owner and grain must agree about the corpus, which is a second free check. A relation whose grain is a catalog grain and whose owner is the SDL gatherer is a capture-time cross-corpus read. CaptureCorpusIsolationTest already catches that dynamically by capturing twice; this states it declaratively, and the two disagreeing is worth knowing about either way.

The comment stops being the documentation and becomes a label. One or two sentences: name the row, give an example. Nothing else. That is grain_text and example and no third thing.

This matters more than it sounds, because it retires the fix this item started with. An earlier section proposed a word cap on comments, ratcheted down from today’s 1431-word maximum. That was treating the symptom. If the comment’s job is to name the row and show one, it is two sentences by construction and no cap is needed to make it so; a cap would only be measuring how far the file still is from a job nobody had stated. Consider the word-cap sketch superseded by this section. What made comments sprawl is that they were the only home for everything, and the fix is to give the other things homes rather than to squeeze the one home smaller.

It also settles the SQL-client question the migration section raises. A relation whose inline description reads "one field of one type in one graph; for example Film.title`" is more useful at a `\d+ prompt than 1431 words was, so nothing is lost by shortening it and the discoverability argument for keeping prose in the comment survives intact.

One authored source, one echo, one gate. The prose is authored in meta_relation, where a CHECK can constrain it. The comment repeats those two columns, because the DDL is static text and cannot interpolate them. A gate requires the relation’s comment to equal its row’s grain_text and example joined, so the echo cannot drift from the source and neither can be edited alone. That is a small duplication bought for an engine-enforced constraint, and it is the reason grain_text is a column at all rather than being left in the comment.

Split every relation comment at its first sentence. The first sentence stays as the COMMENT ON, so a SQL client still describes the relation inline, and becomes grain_text. Everything after it moves verbatim into rationale. That step is mechanical, reversible and makes no judgment about any relation.

example cannot be migrated because there is almost nothing to migrate: four comments in the file contain the words "for example". Filling it is the real work and it is per relation.

rationale is not where the old comment tails go, and this is the decision that gives the slog its point. It holds why the relation exists, which is a paragraph, and it is required. A relation whose reason for existing cannot be written is a relation that probably should not exist, so the field is the question that gets asked 276 times rather than a box that gets filled. The material currently sitting from the second sentence onward is mostly not that: it is measurements, shapes that were tried, and arguments with earlier authors. Those go to docs/architecture/ where rationale lives, or they go, and the relation’s own row carries the reason and not the transcript.

That makes the migration a decision per relation rather than a copy, which is slower and is the whole value. 276 relations, each asked what it is about, who owns it, what one row looks like, and why it exists at all.

All 107 intent_ views declare a grain and an owner, the same as any table. None of them has a key to derive one from, so each is a modelling decision rather than a transcription, and that is the bulk of the work in this item. It is also the point of it: a rule that cannot say what one of its rows is about is the defect being looked for, and the only way to find out which ones cannot is to make all of them try.

The twenty relations with no primary key do not get to state a conditional grain in prose and stop there. A grain that cannot be keyed is bad modelling rather than a hard case, and each of the twenty owes a real key before it owes a registration.

The tree already contains the proof that this is achievable rather than a slogan. The declared type reference collapsed three census relations into one with an owner_kind discriminator, and hit exactly this wall: a collapsed table cannot key on the arm-determined parts because a key column cannot be nullable. The answer that shipped was to spell not-applicable as a value, the empty descriptor and the negative position, each bound to owner_kind by a check constraint in both directions. That relation is keyed today and states more about itself than the three separate ones could. Whatever the remaining twenty need will look like that: a discriminator, a spelled not-applicable, or a split into the two relations the conditional grain was hiding.

276 relations. Each is asked the same four questions and each is one decision, so the item is long rather than hard, and it is done in family order rather than in one pass. The model itself lands first, before any family: the six relations, the gatherer and corpus rosters populated (those are global and small), meta_grain and meta_relation empty, and every gate live with the frozen roster carrying all 276 relations. The new meta_ relations themselves start on that roster too: declaring them needs the store_/meta_ owner question answered, which is deliberately that slice’s decision, not the first one’s.

What stops the second half from never happening, since a family-by-family plan invites exactly that. The gate test carries the roster of relations with no meta_relation row, frozen when the model lands and only ever shrinking, which is the ratchet this repo already uses for read cost: the roster is in a test somebody has to edit, so a family that stalls is visible in a diff rather than in nobody’s memory, and a new relation is on no frozen roster so it cannot arrive undeclared. The item is not done while the roster is non-empty.

Verdict: withhold. Two findings, both on question one, both cheap to resolve. Everything else checks out against the tree: the headline finding (the twenty unkeyed tables are exactly the twenty meta_materialize targets, both directions), the key-shape head, the family counts, every named class and gate method, the fact-model page’s ownership rule and its "not mechanically enforced" line, the six table_ref relations plus graphitron_routine_entry’s `routine_ref, the ten union pairs, and the owner_kind precedent. The goal is well communicated, the plan is reachable, and the design extends the existing meta_ pattern rather than standing a mechanism beside it. Stale counts in the measurement sections were corrected in this commit rather than raised as findings; the corrected grain-sentence figures (20-word median, eleven over 40 words, measured with `GrainSentence’s own terminator rule) weaken the "order of magnitude" phrasing but not the control-case argument, which survives at a factor of five.

Finding 1 (question one). The pending state and the meta_relation column contract contradict each other as written. The ### meta_relation section declares example and rationale NOT NULL and length-checked. "What stops the second half from never happening" makes a pending row legal, with the CHECK constraints conditional on a state the column list does not carry. A NOT NULL column admits no pending row unless not-applicable is spelled as a value, which is the owner_kind precedent this spec itself cites, but the spec never says which design it intends: nullable columns whose CHECK ties non-null to the declared state, or spelled pending values under NOT NULL. The implementer builds a different DDL depending on the answer. Say which, and add the state column to the column list. The same reconciliation owes a sentence on the gates: the echo gate (comment equals grain_text and example joined) cannot hold while a pending relation still carries its old multi-sentence comment, and the Gates bullet’s weaker "grain_text equals the comment’s first sentence" is the form that holds during migration. State which gate binds in which state, or slice one fails on every relation slice one has not reached.

Finding 2 (question one). The owner-grain corpus agreement gate, as specified, flags legitimate rows. "Owner and grain must agree about the corpus" reads as an equality join, but the relations owned by the last gatherer sit at grains that live in captured corpora: intent_field_scope_table is at the field grain, an SDL-corpus grain, and its owner is the derivation gatherer. Blanket equality flags every such row. The rule that matches the section’s own example (a catalog grain owned by the SDL gatherer is a cross-corpus read) is directional: a corpus gatherer may own only relations whose grain lives in its corpus, and the last gatherer is exempt because crossing is its job. State the rule in that form. Two smaller danglers in the same area: what meta_owner.corpus holds for the derivation gatherer, which reads no corpus, and whether the store_/meta_ owner settled in the third slice also gets an exemption or a corpus of its own. The first needs an answer before `meta_owner’s DDL is writable; the second can stay a question for the slice, but say so.

Both findings accepted and resolved in the spec above; neither needed a design argued from scratch, which is what makes them worth recording as findings rather than as edits.

Finding 1. The column contract now says nullable with the CHECK constraints tying all four content columns to the state, both ways, and state is in the column list. Nullable rather than a spelled pending value, and the spec says why: a spelled value would need a meta_grain row and a meta_owner row named pending, which puts a work queue permanently into two rosters that exist to say what the store is about. The owner_kind precedent is for an absence that is part of the model, and this absence is not. On the gates, the reconciliation is simpler than the finding assumed: a pending row carries no grain_text, so no prose gate binds on it and there is no weaker intermediate form to specify. Only declared rows are held to the echo.

Finding 2. The corpus rule is now stated directionally, and the exemption falls out of the data rather than sitting as a case in the gate: meta_owner.corpus is nullable, null exactly for a gatherer that reads no corpus, so the gate reads "a row whose owner names a corpus must have a grain in that corpus" and is silent about the rest. That answers the first dangler, the derivation gatherer’s corpus is null, and it is what makes it the owner that may cross. The second dangler is now explicit in the family table rather than implied: whether the store_/meta_ owner takes a corpus of its own or a null is left to that slice, because the answer depends on what those relations turn out to be about.

One correction accepted with thanks rather than argued. The grain-sentence figures in this item were measured with a hand-rolled sentence split over the escaped comment text rather than with `GrainSentence’s own terminator rule over the unescaped text. The reviewer’s remeasurement is the right one: a 20-word median and eleven of 276 over forty words, which makes the control-case gap a factor of five rather than an order of magnitude. The argument stands and was overstated.

Verdict: sign off. Both round-1 findings are resolved in the plan body, and each resolution is stronger than what the finding asked for. On finding 1, nullable-with-two-way-CHECK beats the spelled-pending alternative for exactly the reason the response gives: a spelled value would plant a work-queue row in two rosters whose job is to describe the model, where the owner_kind precedent covers absences that are part of the model. The gate reconciliation dissolves the intermediate form the finding assumed was needed, since a pending row carries nothing for a prose gate to bind on, and the state column earning a permanent job (the ratchet pinning zero afterwards) closes the "arrives pending forever" hole without a new mechanism. On finding 2, deriving the exemption from a nullable meta_owner.corpus makes the gate one directional join with no named exception, which is the better shape, and the store_/meta_ owner question is now explicitly parked with its slice in the family table. Two counts corrected in passing in this commit: the content-column paragraph said four where the list above it carries five (grain_name, owner, grain_text, example, rationale; the two-way CHECK’s intent was unambiguous either way), and one leftover 2563 from the measurement correction in round 1.

Three design changes agreed with the owner before slice 1, each amended into the plan body above so the body reads as one design; recorded here because two of them revise what round 2 signed off.

Pending state replaced by absence. Round 1 weighed nullable-with-state against spelled pending values and round 2 signed off nullable; the owner put a third option on the table that neither round weighed, and it wins: no pending representation at all. The census over INFORMATION_SCHEMA already says which relations lack a row, so meta_relation holds only answered rows, every column NOT NULL, and the ratchet pins a shrink-only roster of unmigrated relations in the gate test rather than a count (a count admits a swap; a frozen list does not). This also deletes the mechanical tail-into-rationale migration step, which the spec’s own doctrine said was not the real work anyway.

Run order replaced by declared dependencies. The spec gave the owner roster a run-order column; the order gatherers run in is FactCapture.capture’s statement order, a fact of the code the column would only transcribe, and an ordinal asserts a total order the model does not have (the corpus gatherers are mutually independent by design). `meta_gatherer_dependency declares whose rows a gatherer may read, and a gate denies declared views that read outside their owner’s dependency set, which is the fact model page’s ownership rule made mechanical. The roster drives no execution.

meta_owner renamed meta_gatherer, corpus moved to a junction. A relation is named for what its rows are, never for their role ("owner" survives as the role-named FK column on meta_relation); "crawler" was considered and rejected because the tree uses that word for the corpus-reading passes specifically and the derivation gatherer crawls nothing. The scalar nullable corpus column is replaced by meta_gatherer_corpus plus a meta_corpus roster: CatalogFactCapture reads two corpora (catalog and classpath), which a scalar could not state, and the derivation gatherer’s exemption becomes absence of rows rather than a NULL.

Slice 1 decision: the derivation gatherer’s row names Materializations. The roster names each gatherer’s class, and the derivation gatherer has no orchestrating class of its own today: its producers are several classes run in sequence by FactCapture.capture. The row names Materializations, the executor of the refresh plan meta_materialize declares, which is the one class that acts for the gatherer as a whole. If a later slice extracts the derive block into a class of its own, the row repoints and the class gate holds the rename honest.

All 14 relations declared, at 11 grains, every one owned by the catalog gatherer, and struck from the frozen roster, which drops from 281 to 267. Each relation’s COMMENT ON TABLE is now its grain_text and example and nothing else; the column comments are untouched, which is where most of the family’s prose lives and where none of the sprawl this item measured was. Four decisions the slice made, since the family table says this one is meant to settle the model end to end at the smallest scale.

Three relations are at a grain another relation already declares, and that is the model working rather than a collision. sql_primary_key and sql_node_metadata are both at database-table alongside sql_table: each says one thing about one table, which is one grain and three facts, and their keys agree with the grain’s shape because they are the same key. A grain is a roster entry several relations point at, so nothing here needed a per-relation grain to be minted. sql_referential_constraint sits at table-constraint beside sql_constraint on the same reading.

The ordered-child grains are keyed by position, and the name says so. constraint-column, index-column, routine-parameter and node-key-column are each one position in a list rather than one column of a thing, because that is what the key states and what the order is for. The instance sentence is the place that distinction becomes legible, and writing it is what turned four near-identical relations into four grains that state the same shape deliberately.

bound-enum-class lives in the catalog corpus, not the classpath one. The class it names is on the classpath, but the classpath census excludes the generated jOOQ package by design and the row is reachable only through the catalog’s column walk, so the catalog is the corpus the grain lives in. The gate would have passed either way, the catalog gatherer reading both; the choice is stated here because the wrong answer is available and cheap.

What rationale holds, settled by getting it wrong first. The slice’s first pass moved each comment’s tail into rationale more or less intact, which is the one thing the plan above says the column is not for, and the family’s fourteen rationales came to 8548 characters. A second pass cut them to 5932 by asking each one only why the relation exists and deleting the rest. The rule the family order should carry forward: a rationale is two or three sentences, and a sentence that restates the grain, enumerates a domain the DDL already constrains, or recounts what an earlier author argued is not one of them. Two facts that fell out were not lost but rehoused on the columns they are actually about, an overload set colliding in sql_routine’s key and an array-typed column contributing no `sql_enum_binding row, which is the shape of the answer whenever a cut sentence is a real fact: it belongs to a column, or to docs/architecture/, and only then to nowhere.

The echo can strand a column comment that pointed at the old tail. sql_routine_parameter’s `binding_type comment ended "the relation''s own comment says why", and after the rewrite the relation’s comment says no such thing. Shortening a relation comment means grepping its own family’s column comments for references to it; this one is now self-contained.

The seeded detection case now clears the shipped declarations first. MetaDeclarationGateTest.theGatesDetectWhatTheyClaimTo proves each gate detects its own violation by seeding declarations, and it seeded sql_table at a database-table grain of its own, which this slice declared underneath it. It deletes the store’s own meta_relation and meta_grain rows before seeding, so a subject the case wants to state something false about stays available however far the migration gets. The alternative, moving the case onto whichever relation is still undeclared, buys one slice of quiet and breaks again at the next.

Seven relations at seven grains, all owned by the catalog gatherer, roster 267 to 260. Rationales total 3273 characters for the family, which is the standard the sql_ correction set rather than a coincidence: the pass that writes them now asks only why the relation exists.

The expected second owner is not there, and the corpus junction is why. The family table above put jvm_ second because it looked like the second gatherer. It is not: CatalogFactCapture writes the classpath census as well as the catalog one, which is exactly the two-row case meta_gatherer_corpus was introduced for, against a scalar corpus column that could not have stated it. So the slice proves the junction rather than a second owner, and the first genuinely new owner arrives with graphql_. The family-table cell is corrected above rather than left to mislead the next reader.

Both transcripts in this family were already relocated, which is a check rather than a move. The plan says measurements and rejected shapes go to docs/architecture/, and for these two they were there already. jvm_class_supertype’s comment argued the retired assignability closure, its seventeen seconds and its two measured constraints; the fact model page states all of it under derived reads, as the general rule rather than as this relation’s anecdote. `jvm_declared_type_ref’s comment argued the not-applicable-as-value key trade at length, and its own `owner_descriptor and owner_position columns already carry that argument where a reader of those columns will meet it. Cutting a comment tail is therefore two different acts: relocating what has no home, and deleting what turns out to be the third copy. This slice was the second, both times, and the check that distinguishes them is grepping for the fact before cutting it, not judging the prose.

Two facts had no other home and moved onto the column they are about. java.lang.Object being deliberately absent from the supertype edges, and a chain ending at a name outside the census reading as not-known-to-be-assignable rather than as not-assignable, are both about what supertype_name holds. They are on that column now, which also repairs the pointer this slice would otherwise have stranded: it read "see this relation''s comment", and the relation’s comment no longer argues anything. That is the second time the echo rewrite has stranded such a pointer, so the check is now part of the per-family loop rather than a lesson.

Five relations at five grains, roster 260 to 255. java_ is owned by the java-source gatherer over the java-source corpus and javac_ by the compile gatherer over the javac corpus, both already rostered, so this half needed no roster change and is what the family order calls settling the model one corpus over.

The other three families in this group have no owner on the roster, and the answer is to wait rather than to invent one. rejection_, lint_ and build_warning_ are written by RejectionFacts and BuildWarningFacts, neither of which is a rostered gatherer and none of which any rostered gatherer runs. The tempting reading is the one both writers' javadocs invite, that their cadence is CompileFacts’s and so they are its siblings: two gatherers reading a corpus the run produced, on the precedent the `javac corpus sets. Tracing what they actually depend on refutes it twice, and both refutations are worth keeping because the tempting reading is the one a reader arrives at from the writers' own account of themselves.

A build report is a product, not a corpus. GraphQLRewriteGenerator.withLintFindings fuses four producers into the warning list: the classifier’s own advisories from the walk, the lint engine over the parsed registry, the session-state advisories from configuration, and the dependency-currency nudge. A corpus is an input that exists outside the store and independently of the other inputs, and the fused report is neither: three of those four read facts the store already captures, so naming their output a corpus would have made those gatherers crawlers over their own product, and the corpus-agreement gate would have passed while stating something false. The fourth producer is the exception that proves the point rather than a counter-example, and it is filed as R908: the dependency-currency advisory reads resolved dependency versions no relation holds, where its three siblings all read captured facts.

All four families draw on the walk, which is being dissolved as this is written. rejection_ is GraphitronSchemaValidator over the walk’s own model. build_warning_no_rule’s only two producers are `GraphitronSchemaBuilder and EntityResolutionBuilder. lint_finding takes rows from TypeBuilder and FieldBuilder as well as from the lint engine. So the ownership question is not merely open, it is not answerable from the current tree: the producers are moving. R870 knows it, its delivery criteria naming the rejection_ and build_warning_ charters directly. The four stay on the frozen roster, which costs nothing, the ratchet forbidding only that a relation arrive undeclared. Whoever picks this up after R870 should re-trace before declaring: if the advisory minting leaves the walk for store-native detections, these producers read captured rows, which makes them corpus-less gatherers on the pattern the graphitron and derivation gatherers already set, with no new corpus at all.