S7: Additional Features and Documentation

Status: complete for the S7 scope. The broader P8-P10 functional readiness items in public-readiness.md remain in progress. Local checks and static documentation checks pass.

Feature work

ID	Work	Completion condition
T7.1	protobuf/Connect transport	Go, PHP, Rust, and TypeScript implement proto/orm/compiler/v1/compiler.proto, the Connect RPC path, and the same conformance vectors. Their declared default compiler paths remain PHP Unix socket, Go in-process, Rust WASM, and TypeScript Connect/Protobuf.
T7.3	DDL diff and migration generation	ormgen diff compares two manifests, emits deterministic SQL, and rejects table removal, column removal, and column changes without --allow-destructive.
T7.4	Multi-tenant scope	A %% scope <table> <column> directive validates a non-null tenant column and generates query-level scope(value) for all supported clients.
T7.5	point, yaml, and curlfile styles	Encoding and decoding vectors produce the same values in all supported clients.
T7.6	Database row streaming	Database cursor lifecycle, cancellation, errors, and row ownership are specified and tested.
T7.7	ormgen precompile	Static query shapes can be compiled to a file containing the request hash, schema hash, dialect, and plan.
T7.8	Fourth language client	The TypeScript client passes the common structure, symbol, state, and conformance checks for every generated entity.
T7.9	mysql_async comparison	Rust driver results and benchmark data are recorded against the current driver.
T7.10	multi_statement exclusion	The IR and interface checks reject multi_statement fields and generated symbols.
T7.11	Row scan optimization	Go generated typed scans and PHP positional hydration pass the PK and 100-row regression checks.
T7.12	150-table Rust fixture	The fixture is stored or generated deterministically and the compile-time result is recorded.
T7.13	AES key rotation	AES tables require aes_key_version; reads, writes, status, and explicit re-encryption pass in all supported clients.
T7.14	Schema comments	Table and column comments pass through manifest hashes, import, DDL, diff, and SQLite metadata.
T7.15	Migration execution and recovery	MySQL, PostgreSQL, and SQLite serialize execution and classify interrupted migrations as applied, retryable, or unsafe.
T7.16	SQL statement parsing	Migration parsing preserves quoted semicolons and reports the complete failing statement and operation number.
T7.17	Migration source conversion	MMD, SQL, JSON, and live databases support the documented conversion matrix, loss checks, and rollback plans.
T7.18	Physical migration tests	MySQL and PostgreSQL physical tests cover comments, apply, repeat, drift, failure, locks, and recovery.

Documentation work

ID	Work	Completion condition
T7.D1	Language file structure	English pages under docs/ have adjacent Korean pages with .ko.md.
T7.D2	Korean translations	Korean files adjacent to each source with .ko.md contain the same headings, code samples, tables, and links.
T7.D3	Language navigation	Pages provide links between the English and Korean pages.
T7.D4	Style review	Informal, figurative, personifying, and ambiguous wording is removed from manuals.
T7.D5	Feature tables	Each feature documents inputs, outputs, errors, state changes, and supported clients.
T7.D6	Unfinished work list	Unimplemented features are marked as not started or partial.
T7.D7	Translation checker	CI compares document headings, code fences, tables, and link targets.
T7.D8	Style checker	CI rejects configured informal and figurative expressions.
T7.D9	Examples and commands	Each completed S7 feature has examples and reproducible verification commands.
T7.D10	Pages publication	The static build contains both language paths and passes link and no-JavaScript checks.

Common completion rule

An S7 item is complete only when implementation, tests, documentation, and static publication are complete. A feature that cannot use the same logical structure in Go, PHP, Rust, and TypeScript remains unimplemented.

The default compiler paths are PHP over the Unix socket, Go in-process, and Rust through WASM. TypeScript uses the Connect/Protobuf compiler path. These are supported execution paths. All four clients expose the same request, plan, execution, result, and error structures. Connect/Protobuf is the common interoperable path; a client may use its declared native path when its configuration selects it.

T7.3 migration command

go run ./cmd/ormgen diff \
  --from old-schema.json \
  --to schema/schema.json \
  --dialect mysql \
  --out migration.sql

The command emits ADD COLUMN and new table statements without the destructive option. It fails when the change removes a table or column or changes a column definition. Add --allow-destructive only when the migration review includes the destructive SQL.

T7.7 precompile command

go run ./cmd/ormgen precompile \
  --schema schema/schema.json \
  --dialect mysql \
  --in request.json \
  --out plan.json

The output contains version, schema_hash, dialect, request_sha256, and plan. A client must compare schema_hash before loading the plan.

Clients register this bundle metadata with the value-free request that produced it. Go uses db.LoadPlanBundle(bytes, req), PHP uses Orm::transport()->loadPlanBundle($bundle, $req, $kind), Rust uses db.load_plan_bundle(&bytes, &mut req), and TypeScript uses db.loadPlanBundle(bundle, request). Each loader validates the metadata version, schema hash, dialect, request kind, request hash presence, and plan body before inserting the plan into the bounded plan cache. A matching later request reads the cached plan without a compiler call. The bundle remains invalid when its schema or dialect differs from the client.

T7.4 scope directive

%% scope battle service_seq

The directive requires an existing non-null integer, string, or enum column. Generated clients expose scope(value) on the query object only. The request stores its parameter index in query.scope_p, separate from the user predicate tree.

Operation	Scope behavior
root select, update, delete	Adds the scope condition outside the complete user WHERE group.
join	Adds the joined entity scope to JOIN ON, including LEFT JOIN.
relation	Adds the child scope to the relation query WHERE.
insert	Adds the declared scope column with the scope parameter. An explicit assignment must use the same parameter index.
update and upsert	Rejects an assignment to the declared scope column.
raw SQL	Rejects a scoped request because the compiler cannot safely determine an injection position.

Calling scope(value) for an entity without %% scope returns IR_INVALID. Update and delete still require an explicit user WHERE group. Unit tests verify OR precedence, join ON placement, relation steps, write restrictions, and invalid requests. Physical tests verify tenant isolation on MySQL, PostgreSQL, and SQLite in all four clients.

T7.6 database row streaming

stream(visitor) executes one select statement and invokes the visitor once for each row without creating a result collection. Each row owns its values. Returning false stops iteration; returning true continues. The result contains state (stopped or exhausted) and count, including the row that returned false.

Client	Example
Go	Battle().Using(ctx, db).Stream(func(row *BattleRow) bool { return handle(row) })
PHP	Battle::query()->using($db)->stream(fn(BattleRow $row): bool => handle($row))
Rust	`battle::query().using(&db).stream(
TypeScript	await Battle().using(db).stream(async row => handle(row))

The query must be bound before execution. SQL joins are supported because they use the root statement. A plan containing a separate relation statement returns IR_INVALID before opening a cursor; use gets() for that plan. A missing visitor, invalid plan, decode failure, driver failure, or native cancellation returns the existing error category. Visitor failure is propagated in languages that support exceptions from callbacks. Every completion path closes the cursor before returning and releases a borrowed pool connection. A callback return value of false is the common cancellation mechanism.

The common interface_stream vector checks early cancellation after three rows, complete iteration of four rows, independent row ownership, relation rejection, and two executed statements. It runs in Go, PHP, Rust, and TypeScript on MySQL, PostgreSQL, and SQLite.

go run ./tests/conformance/check run
go run ./tests/interfaces/check --results tests/conformance/out
make ts-check rust-check

T7.10 multi-statement exclusion

The public IR and generated clients do not contain a multi-statement option. A request containing multi_statement returns IR_INVALID before SQL execution. The interface manifest prohibits the normalized symbol name, so structure checks reject multi_statement, multiStatement, and MultiStatement declarations in every generated client.

Database	Constraint
MySQL	Multiple statements require a separate driver option and multiple-result handling.
PostgreSQL	Parameterized prepared execution does not accept multiple commands in one statement.
SQLite	Supported driver APIs prepare and execute one statement at a time.

A relation plan retains its ordered steps array. Each dependent step executes after the parent values are available. This preserves parameter binding, failure attribution, and statement counts across all supported clients and databases. A rejected request executes no SQL and changes no query or database state.

go test ./engine ./tests/interfaces/check
go run ./tests/interfaces/check --self-test

T7.11 row scan optimization

Go generates a fixed scanner for each entity's default flat projection. It passes primitive fields directly to database/sql.Rows.Scan; datetime and codec outputs use named temporary values. A projection change, expression, join, relation, or additional plan step uses the positional assembly path. MySQL, PostgreSQL, and SQLite integration tests cover both paths.

PHP keeps PDO::FETCH_NUM rows as the generated row's positional storage. PDO has no typed destination interface equivalent to Go's Rows.Scan. An implementation that called fetch() and created each object immediately increased the measured 100-row time from about 425µs to 511µs, so it was removed. fetchAll(PDO::FETCH_NUM) followed by generated row construction remains the PHP path.

The native Go and PHP benchmarks previously omitted the non-lazy price and ip columns. The benchmarks now use the same default projection as the generated client. The PHP baseline also constructs the same generated row result. Three consecutive runs on MySQL 8.4 produced these median ratios on 2026-09-12:

Workload	Go client/native	PHP client/native	Limit
Primary-key row	0.48	1.12	1.35
100-row list	1.05	0.96	1.25

The limits are the existing S6 regression limits. T7.11 does not change them. The earlier S5 three-row ≤5% objective compared different work after the schema added columns and codecs; the current check compares equal projections and typed results. Alternating PHP samples and same-process Go samples reduce timing drift.

make perf-check

T7.13 AES key rotation

An entity with an AES column requires a non-null integer aes_key_version column. The version column is plaintext metadata, has no encoding style, and is excluded from the default projection. npm run schema:check and the schema builder reject invalid declarations. The schema builder can infer an AES style from a column prefix, but generation and rotation use the finalized styles list in schema.json. Runtime code does not search column-name prefixes.

New rows in an AES entity always store the configured current version, including rows whose AES values are null. One version describes every AES column in a row. An update or the duplicate-update branch of an upsert that changes AES data must assign every AES column; a partial assignment returns IR_INVALID. The planner stores the current version after validating the complete assignment. Generated clients do not expose a setter for aes_key_version, and direct attempts to assign it return IR_INVALID. The version bind is ordinary configuration metadata and is not masked as a secret.

[secrets]
aes_version = 2

[secrets.aes_keys]
1 = "old-key"
2 = "current-key"

The generated entity API uses the same inputs and results in each client.

Client	Status	Rotation
Go	Battle().Using(ctx, db).AESStatus(keyring)	Battle().Using(ctx, db).RotateAES(keyring)
PHP	Battle::query()->using($db)->aesStatus($keyring)	Battle::query()->using($db)->rotateAES($keyring)
Rust	battle::query().using(&db).aes_status(&keyring).await?	battle::query().using(&db).rotate_aes(&keyring).await?
TypeScript	Battle().using(db).aesStatus(keyring)	Battle().using(db).rotateAES(keyring)

Status returns the configured current version, total row count, pending row count, and row counts by stored version. Rotation selects only pending rows. It reads every primary-key column in declared order, the stored version, and every AES column; decodes every AES column with the stored-version key; encodes them with the current key; and updates all AES columns and the version in one transaction. The update includes every primary-key column and the previous version. A concurrent change returns OPTIMISTIC_LOCK. Missing keys return CONFIG; invalid version metadata and invalid ciphertext return CODEC_DECODE. Any failure rolls back the transaction. A repeated rotation returns zero changed rows.

Reads select the key from the stored aes_key_version. Configure every stored version in secrets.aes_keys before rotation. Set the new version as current for writes, run status, rotate in bounded batches (batch_size in PHP, batchSize in TypeScript, and batch_size in the Rust specification; Go uses the same generated specification field), require pending = 0, verify the database, and remove the old key only after the retention period. A call processes at most 1000 rows by default; pending rows remain eligible for the next call.

go test -tags physical ./clients/go/orm -run TestPhysicalAESRotation -count=1
npm run typescript:check
node tests/typescript/database.mjs

The physical tests create a dedicated table, insert old-version rows with all AES columns, check status, rotate, decode with the current key, and verify that a second rotation changes zero rows. scripts/db-test.sh runs the MySQL and PostgreSQL cases through containerctl; the client suites also execute SQLite cases.

T7.9 Rust MySQL driver comparison

bench/rust/src/driver_compare.rs compares sqlx 0.9 and mysql_async 0.37.1 with one connection, the same prepared SQL and binds, the same local MySQL fixture, typed result construction, 200 warmup operations, and 1,000 measured operations. Every paired result is compared before its duration is recorded.

cd bench/rust
cargo run --release --locked --bin driver_compare -- 1000

The 2026-09-12 Apple M3 Pro and MySQL 8.4 local-socket run measured mysql_async/sqlx mean ratios of 1.051 for a primary-key row and 0.943 for a 100-row list. The package rule requires a measured 2x improvement before replacing sqlx. These results retain sqlx. make rust-driver-check compiles the locked comparison program in CI; timing remains a controlled local operation because shared CI hosts do not provide stable latency.

T7.5 additional codecs

Go, PHP, Rust, and TypeScript implement the recursive curlfile value, strict YAML 1.2 parsing, and point conversion. The four clients accept (x,y) database output and POINT(x y) transport text and require two finite coordinates. Generated fields use the types in ir-v1.md. MySQL and SQLite bind POINT(x y). PostgreSQL binds its native (x,y) text. MySQL uses ST_PointFromText and ST_AsText; PostgreSQL casts point values through text; SQLite stores text. Unit tests verify generated types, SQL plans, invalid values, and a physical SQLite database. The container test verifies physical MySQL and PostgreSQL databases.

T7.8 TypeScript structure check

The TypeScript module is buildable with npm run typescript:build and exports dist/index.js and dist/index.d.ts. It defines the common request, query, where, row, compiler, and executor types in clients/typescript/src/index.ts. Generated entry points cover battle, user, service, service_module, and service_member; scope() rejects entities without a declared scope. Db.connect(dsn, options) validates the DSN scheme and compiler metadata; it does not require a runtime configuration file or a driver argument. npm run typescript:check runs the TypeScript compiler, and node tests/typescript/check.mjs checks declarations and method order. The codec vectors and 63 database vectors pass on MySQL, PostgreSQL, and SQLite.

The compiler wire definition is proto/orm/compiler/v1/compiler.proto. It includes the complete request tree, tenant scope parameter, write fields, assembly tree, bind metadata, compiler errors, and server metadata. scripts/proto/generate.sh generates Go, PHP, Rust, and TypeScript message types with pinned plugins. ormd -listen 127.0.0.1:8080 -schema schema/schema.json serves binary Protobuf over the Connect unary paths. make proto-check starts this server and verifies that the four CompilerTransport implementations return identical metadata and plans. All four executors use this transport for plan-cache misses and pass the 63 database vectors on MySQL, PostgreSQL, and SQLite.

T7.12 150-table fixture

Run make rust-150-check. The script generates the same 150-entity Mermaid file in a temporary directory, builds the manifest, generates a Rust crate, and runs cargo check. The temporary files are removed after the check.

T7.15 migration recovery

ormgen recover accepts either the reviewed structured plan or a migration ID recorded by ormgen migrate. It acquires the database migration lock before reading the history state and live schema. A live target match records applied; a live source match records retryable; any other state returns MIGRATION_RECOVERY_UNSAFE without changing history or file logs. MySQL, PostgreSQL, and SQLite run the same state transitions. The command syntax and result table are in usage.md.

T7.17 migration sources

DDL, diff, plan, migrate, verify, apply, and recover accept schema sources through the common loader. A source is a Mermaid file, manifest JSON file, ORM-generated SQL file, or db:<dsn>. SQL generated by ormgen ddl and ormgen diff contains deterministic orm-schema-v1 metadata. The loader verifies the embedded manifest hash before using it. SQL without this metadata returns MIGRATION_SOURCE_LOSS because SQL cannot represent scope, codec styles, named predicates, and relation options. Structured plans contain verified forward and rollback operations. ormgen rollback checks the plan, history, live target schema, and destructive-risk acknowledgement before execution, then verifies the source schema and file log. The command matrix and examples are in usage.md.

Live schema inspection rejects an AES column without aes_key_version. Target MMD, JSON, generated SQL, and live database validation require a non-null integer version column. The runtime does not provide conversion or compatibility for the previous ECB format.