Test-tier guide

Read top-down; stop at the first match.

When two tiers could apply, prefer the one that captures the behaviour most directly. Pipeline beats unit: per-variant structural tests are bookkeeping; the primary signal is that a realistic SDL produces a realistic TypeSpec end-to-end. Pipeline also beats compilation and execution where the behaviour can be asserted on the classified model or TypeSpec shape, since pipeline runs without jOOQ codegen or Postgres. Execution beats compilation only when SQL behaviour or row content is the contract.

Tier is determined by what’s asserted, not by what module the file lives in. graphitron-sakila-example hosts tests at every tier; its module dependency on post-generator artifacts is the reason those tests live there, not a tier signal.

Each tier has a JUnit 5 meta-annotation in graphitron’s test source root (`no.sikt.graphitron.rewrite.test.tier), republished as a tests test-jar so other modules can consume them. Reachable from every test class in graphitron and graphitron-sakila-example:

Place exactly one annotation at the class level. Tests that don’t fit any of the four tiers (GeneratorDeterminismTest is the only current example) carry @Tag("cross-cutting") directly.

With class-level tags in place, mvn test -Dgroups=pipeline runs only pipeline-tier classes; -DexcludedGroups=execution skips Postgres for fast inner loops. Both Surefire and Failsafe honour these flags without further config.

An enforcement test in each in-scope module (graphitron, graphitron-sakila-example) walks that module’s own test classpath and fails the build if any @Test-bearing class lacks a tier identity, or carries more than one.

Structural invariants on individual classifiers, builders, emitters, and runtime helpers.

Where: graphitron/src/test/java/…​ next to the production class.

Three sub-families:

Generator unit tests (TypeFetcherGeneratorTest, GeneratorCoverageTest; and the generators/schema/ subdirectory: EnumTypeGeneratorTest, GraphitronFacadeGeneratorTest, InputTypeGeneratorTest, etc.). Take pre-built model fixtures via TestFixtures; assert TypeSpec shape (method names, return types, parameter signatures). Banned: code-string body matching on the generated MethodSpec body; that is what compilation and execution cover.

Validator unit tests (*ValidationTest family, e.g. ColumnFieldValidationTest, QueryTableFieldValidationTest). Build a GraphitronSchema with one parent type and one field at a known coordinate; assert validate() outcomes by RejectionKind and message substring.

Builder / catalog / writer unit tests (JooqCatalogFindColumnTest, IdempotentWriterTest, ArgBindingMapTest, ServiceCatalogTest, etc.). Targeted constructor or single-method assertions; no full-pipeline plumbing.

Renderer arm tests (ProjectionUnitRendererTest; the command-driven families). A renderer is a total function over a command’s sealed arms, so its inputs are record literals constructed at the point of assertion; no TestSchemaHelper, no fixture, no catalog. Every arm of the sealed set is reachable directly, which per-arm coverage of the emit never had while reaching an emitter branch meant driving the whole pipeline into one leaf configuration. The "pipeline beats unit" doctrine above was written against fixture-plumbed generator tests; renderer arm tests are a different species (no plumbing) and are the preferred home for per-arm structural assertions on command-driven emission. The producer’s decisions are asserted separately at the pipeline tier, as data (SDL to command rows, e.g. ProjectionMembershipTest, ConditionCommandsPipelineTest), without javapoet.

SDL → classified model → generated TypeSpec.

Where: graphitron/src/test/java/no/sikt/graphitron/rewrite/.

Two shapes:

Classification truth tables: GraphitronSchemaBuilderTest. Each variant family is a // ===== VariantName ===== section with an enum where each constant is one (description, SDL, assertion) triple; one parameterised test iterates the table.

The spec-by-example corpus: graphitron/src/test/resources/corpus/, one GraphQL document per example, loaded by CorpusDocuments. This is the tree’s only folder-of-documents fixture set, and the only fixture home a test reads from the source tree rather than the classpath, so what is asserted is exactly what an author edits. A document carries its annotated fixture, its prose as SDL descriptions, and optionally a projection operation as its last definition; the loader splits the file at that line, hands the type-system half to classification and the operation to the documentation renderer. Every reader of the corpus goes through the loader (ClassifiedDslTest, VariantCoverageTest, the derive/*ShadowTest sweeps, the documentation fragment approval), and nothing else lists the directory. A document states what the fact store holds for it by applying @expectEquals(relation:, rows:) to the schema, once per asserted relation, its rows as CSV; CorpusExpectationTest captures the whole corpus into one store, one graph per document, and compares each block against its relation by anti-join in both directions, so a failure is a row. That is the successor to a coordinate directive naming sealed-arm tokens: a block names a relation and its columns, so it asserts a store fact and reaches relations no coordinate could key. CorpusDocumentsTest carries the floors that keep a folder from passing while empty: a ratcheted document count, agreement between the loader’s admissions and an independent listing, and a document nothing runs failing rather than being skipped. Adding an example is one new file; a .java edit means the assertion vocabulary changed, which is a prelude edit (_prelude.graphqls), not a container one.

A worked example’s documentation fragment: docs/architecture/reference/_example-<id>.adoc, one per document carrying a projection, written by CorpusFragmentTest. This is the tree’s second approval-style fixture home, after the .approved.json files the sakila example’s approval test compares against, and it is generated-and-committed for the same reason roadmap/README.md is: the fragment’s outcome table holds the emitted unit and method names, which are not a store fact and are pinned nowhere else, so the file is an oracle and has to be checked in to be one. The reference page includes each fragment rather than carrying the blocks, so the page authors prose and teaching order and holds no expectation of its own. When a render legitimately changes, the test leaves this run’s output under graphitron/target/corpus-fragments/ and its message gives the cp line; never hand-edit a fragment. Beside the approval sit three placement floors, each with a planted regression in CorpusFragmentRendererTest: a fragment with no document, a document with no fragment, and a fragment no page includes.

Deeper SDL → TypeSpec / variant-shape tests: *PipelineTest files: NodeIdPipelineTest, BatchedTableFieldPipelineTest, TableFieldPipelineTest, LookupPipelineTest, NestingFieldPipelineTest, ServiceRootFetcherPipelineTest, TaggedInputsPipelineTest, StubbedVariantPipelineTest; and in generators/: FetcherPipelineTest, TablePipelineTest. Build a schema with TestSchemaHelper.buildSchema(sdl), assert structural shape on the resulting variant or generated TypeSpec. Banned: code-string body matching.

Generated source must compile against the test catalog.

Where: graphitron-sakila-example, run with mvn compile -pl :graphitron-sakila-example -Plocal-db. The compiler is the assertion; no hand-written assertions are needed for type correctness.

Two test classes layer structural checks on top:

GeneratedSourcesSmokeTest: every expected class is present in the emitter’s output package (catches a generator that silently drops a class).

GeneratedSourcesLintTest: generator-hygiene rules over emitted source text (e.g. no var in emitted code).

Full GraphQL request → SQL → row round-trip.

Where: graphitron-sakila-example, run with mvn test -pl :graphitron-sakila-example -Plocal-db.

Canonical classes: GraphQLQueryTest on the shared fixture; FederationEntitiesDispatchTest on the federated fixture.

Patterns:

Several tests live in graphitron-sakila-example for module-dependency reasons but classify by assertion, not module. Only GeneratorDeterminismTest is @Tag("cross-cutting"); the rest carry one of the four tier annotations:

Tier answers how far a test runs. A second question is orthogonal to it: a test that reads the fact store has to get rows in front of itself, and how it populates the store follows what the test is about, not which module it happens to sit in. There is a harness per subject, and a structural guard (StoreFixtureGuardTest) fails the build on a test that opens a GraphitronModelStore itself instead of taking one from a harness.

Seeding is the method in exactly one module, the one whose subject is the DDL. Above that line capture is the default: the generator, the language server and the MCP server exist to turn real inputs into real rows, so a fixture that skips that step stops testing the thing. A seeded fixture above the model line is occasionally right and owes a reason at the call site.

If no harness expresses the shape you need, add it to the one that owns your subject rather than hand-rolling a helper in your test class. A harness carrying more helpers than any one reader needs is the cheap state; a spread of private copies that have quietly diverged is the expensive one, and it is expensive because nothing points the next author at the existing answer.

See .claude/web-environment.md for database setup prerequisites and the fixtures-jar footgun recovery.

Coverage is off by default. -Pcoverage attaches the JaCoCo agent to every module’s test fork and writes a per-module report at verify.

A module’s report attributes only that module’s own classes. graphitron’s figure is generator-source coverage from its unit and pipeline tiers; `graphitron-sakila-example’s figure is coverage of generated code from the compilation and execution tiers. Two kinds of generator code are missing from both. Code that ran in the Maven process during `graphitron:generate was never instrumented, because the agent is attached to test forks. Code that ran in another module’s test JVM (GeneratorDeterminismTest invokes the generator in-process) was recorded there, but a module’s report analyses only its own classes, so it is dropped when the report is written.

CI publishes the per-tier split for graphitron on trunk pushes. To reproduce it locally, run each tier into its own exec file:

-Dleaf-coverage.skip is not optional in the tier runs: without it each run re-emits graphitron/target/leaf-coverage.jsonl from only that tier’s tests, leaving the inference-axis report a strict subset of what the full suite wrote. The page grows per-tier columns when it finds those directories. Read the columns as slices, not as a decomposition: cross-cutting classes are in neither, so the two do not sum to the combined figure, and the renderer arm tests this guide blesses as a unit-tier family land in the unit column by design.