Add multi-error diagnostic-collection scaffolding (phase 1) - #957
Conversation
The P type checker today aborts on the first error: every site in
TypeChecker/* does `throw handler.X(...)`, which unwinds all the way to
the top-level catch in Compiler.cs. A user with N independent type errors
needs N compile-fix cycles to see them all, and PeasyAI's `peasy-ai-fix-all`
loop does N LLM round trips for the same reason.
This is phase 1 of a 3-phase change to collect all errors and report them
together. Phase 1 introduces scaffolding only — no observable behavior
change. Strict mode (the default) is preserved everywhere.
Adds:
- IDiagnosticCollector / DefaultDiagnosticCollector
Strict mode rethrows immediately (today's behavior).
Collecting mode appends and returns; phase 2 will start using this.
- ErrorType sentinel (TypeChecker/Types/)
`IsAssignableFrom` returns true for any other type, so all existing
`IsAssignableFrom` / `IsSameTypeAs` checks transparently pass when
either operand is the sentinel. This is the cascade-suppression trick:
one undeclared variable won't generate 20 "incompatible type" follow-ups.
- ErrorExpr sentinel (TypeChecker/AST/Expressions/)
Deliberately does NOT implement IExprTerm, so the IR transformer
trips loudly if one leaks past type-checking.
- ContinueOnError + Diagnostics on ICompilerConfiguration
Same IDiagnosticCollector instance exposed via Handler.Diagnostics so
visitors with only a handler reference (e.g. ExprVisitor) can reach it
in phase 2 without constructor changes. Driven by env var
P_COMPILER_COLLECT_ERRORS.
- Compiler.cs flush + skip
After type-checking, if any diagnostics were collected, dump and
exit non-zero (skipping IR transformer + code generation). Dormant
in phase 1 since no visitor reports through the collector yet.
- Smoke tests covering the collector, sentinel types, and the
handler/config shared-instance invariant.
Phase 2 will convert the ~67 throw sites in ExprVisitor and ~44 in
StatementVisitor to `handler.Diagnostics.Report(handler.X(...)); return
new ErrorExpr(ctx);`, with cascade-suppression rules at the combiner
sites (binop, cast, call, etc.). Phase 3 will add per-pass tolerance
classification in Analyzer.cs.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
There was a problem hiding this comment.
Pull request overview
This PR introduces phase-1 scaffolding to support switching the P type checker from “throw-on-first-error” to “collect diagnostics and report together,” while keeping strict mode as the default and leaving the new path dormant until later phases.
Changes:
- Adds
IDiagnosticCollectorand a default implementation to support strict vs collecting diagnostic modes. - Introduces sentinel
ErrorTypeandErrorExprto enable cascade-suppression and safe recovery values in later phases. - Wires configuration/handler/compiler plumbing for
ContinueOnError+ diagnostic flushing, and adds smoke tests for the new scaffolding.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| Tst/UnitTests/TypeChecker/DiagnosticCollectorTest.cs | Adds smoke tests for strict/collecting collector behavior and sentinel invariants. |
| Src/PCompiler/CompilerCore/TypeChecker/Types/ErrorType.cs | Introduces an error sentinel type intended to suppress cascading type errors. |
| Src/PCompiler/CompilerCore/TypeChecker/AST/Expressions/ErrorExpr.cs | Introduces an error sentinel expression returning ErrorType. |
| Src/PCompiler/CompilerCore/ITranslationErrorHandler.cs | Exposes Diagnostics on the error handler for phase-2 throw-site conversions. |
| Src/PCompiler/CompilerCore/IDiagnosticCollector.cs | Defines the collector contract used for strict vs collecting mode. |
| Src/PCompiler/CompilerCore/ICompilerConfiguration.cs | Adds ContinueOnError and Diagnostics to configuration. |
| Src/PCompiler/CompilerCore/DefaultTranslationErrorHandler.cs | Wires a collector instance into the default handler. |
| Src/PCompiler/CompilerCore/DefaultDiagnosticCollector.cs | Implements strict/collecting behavior for diagnostic recording. |
| Src/PCompiler/CompilerCore/CompilerConfiguration.cs | Creates the collector/handler and reads P_COMPILER_COLLECT_ERRORS. |
| Src/PCompiler/CompilerCore/Compiler.cs | Flushes collected diagnostics after type-checking and exits non-zero when present. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /// <summary> | ||
| /// Returns true for any other type. This is the central trick that | ||
| /// makes cascade-suppression work without per-site special cases: | ||
| /// every existing <c>IsAssignableFrom</c> / <c>IsSameTypeAs</c> check | ||
| /// transparently passes when either operand is the error sentinel, | ||
| /// so no additional diagnostic is emitted. | ||
| /// </summary> | ||
| public override bool IsAssignableFrom(PLanguageType otherType) => true; |
| // with every other type, so downstream IsAssignableFrom/IsSameTypeAs | ||
| // checks transparently pass and don't emit new diagnostics. | ||
| Assert.IsTrue(ErrorType.Instance.IsAssignableFrom(PrimitiveType.Int)); | ||
| Assert.IsTrue(ErrorType.Instance.IsAssignableFrom(PrimitiveType.Bool)); | ||
| Assert.IsTrue(ErrorType.Instance.IsAssignableFrom(PrimitiveType.String)); | ||
| Assert.IsTrue(ErrorType.Instance.IsSameTypeAs(PrimitiveType.Int)); |
| public bool ContinueOnError { get; } | ||
|
|
||
| public IReadOnlyList<Exception> Diagnostics => diagnostics; | ||
|
|
||
| public bool HasErrors => diagnostics.Count > 0; |
| public DefaultTranslationErrorHandler(ILocationResolver locationResolver, IDiagnosticCollector diagnostics) | ||
| { | ||
| this.locationResolver = locationResolver; | ||
| Diagnostics = diagnostics ?? new DefaultDiagnosticCollector(); |
Four fixes from the Copilot review on #957: 1. ErrorType cascade suppression was asymmetric. PLanguageType.IsSameTypeAs does `this.IsAssignableFrom(other) && other.IsAssignableFrom(this)`, so only the LHS-is-error case was suppressed; the RHS-is-error case still delegated to e.g. PrimitiveType.Int.IsAssignableFrom(ErrorType), which returned false. The doc claimed the suppression was transparent, but it wasn't. Fix: short-circuit IsSameTypeAs when either operand is ErrorType (a small targeted change to the base class, guarded by a comment). Doc on ErrorType expanded to spell out the two-piece mechanism honestly and to forward-reference the Phase 2 CheckAssignable helper that covers the remaining asymmetric IsAssignableFrom sites. 2. The smoke test would have failed in CI as written, since it asserted ErrorType.Instance.IsSameTypeAs(PrimitiveType.Int). Now passes thanks to (1), and extended to also assert symmetry: int.IsSameTypeAs(error) must hold too. 3. DefaultDiagnosticCollector.Diagnostics returned the backing List<T> typed as IReadOnlyList<T>, leaving callers free to downcast and mutate. Now returns diagnostics.AsReadOnly() — a ReadOnlyCollection<T> wrapper that throws NotSupportedException on mutators. New test downcasts and asserts both Add and Clear throw. 4. DefaultTranslationErrorHandler's 2-arg constructor silently replaced a null collector with a freshly-allocated one, which could break the "Handler.Diagnostics === Config.Diagnostics" invariant. Now throws ArgumentNullException. New test asserts the throw. The 1-arg overload remains the place that supplies a default collector. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
| /// <summary> | ||
| /// Collector shared with <see cref="ICompilerConfiguration.Diagnostics"/>. | ||
| /// Exposed here so visitors that hold only a handler reference (e.g. | ||
| /// <c>ExprVisitor</c>) can report diagnostics without constructor | ||
| /// changes. Phase 2 will use this as | ||
| /// <c>handler.Diagnostics.Report(handler.X(...))</c> in place of | ||
| /// <c>throw handler.X(...)</c>. | ||
| /// </summary> | ||
| IDiagnosticCollector Diagnostics { get; } |
There was a problem hiding this comment.
Acknowledged — this is a deliberate choice. ITranslationErrorHandler is implemented by exactly one concrete class in this repo (DefaultTranslationErrorHandler), and P ships as a dotnet tool CLI (p compile, p check) rather than as a NuGet SDK, so external implementers don't exist in practice. The breaking-change cost is theoretical here while keeping the property on the interface keeps Phase 2's visitors clean (they only hold an ITranslationErrorHandler reference and now reach the collector via handler.Diagnostics).
If we ever start distributing the compiler as a library, we can introduce a derived interface (ITranslationErrorHandlerWithDiagnostics) at that point and cast at the call sites that need it — the API surface is small enough to retrofit. Flagging this in the PR description for posterity rather than adding the derived-interface ceremony now.
| /// <summary> | ||
| /// When true, the type checker collects diagnostics and continues | ||
| /// instead of throwing on the first error. See <see cref="IDiagnosticCollector"/> | ||
| /// for the contract. Driven by env var <c>P_COMPILER_COLLECT_ERRORS</c> | ||
| /// (any non-empty / non-"0" value enables it). | ||
| /// | ||
| /// Phase 1: scaffolding only. No visitor currently reports through the | ||
| /// collector, so flipping this flag has no observable effect yet. | ||
| /// </summary> | ||
| bool ContinueOnError { get; } | ||
|
|
||
| /// <summary> | ||
| /// The diagnostic collector for this compilation. Same instance as | ||
| /// <c>Handler.Diagnostics</c>. Construction-time: callers should pass | ||
| /// <see cref="ContinueOnError"/> into the collector constructor. | ||
| /// </summary> | ||
| IDiagnosticCollector Diagnostics { get; } |
There was a problem hiding this comment.
Same rationale as the parallel comment on ITranslationErrorHandler.cs: ICompilerConfiguration is implemented by exactly one concrete class (CompilerConfiguration) in this repo, and P is distributed as a CLI tool rather than as a library. Adding ContinueOnError and Diagnostics to the interface keeps the API discoverable and lets callers (including future Phase 2/3 wiring) reach them without casting.
If external library consumers ever appear, the derived-interface refactor is a low-cost retrofit at that point. Tracking the deliberate-break choice in the PR description rather than introducing two parallel interface hierarchies preemptively.
| // AsReadOnly returns a live ReadOnlyCollection<T> wrapper: callers can | ||
| // still observe new diagnostics as they're added, but mutating methods | ||
| // throw NotSupportedException even if a caller downcasts. This | ||
| // preserves the invariant that only Report() can change the list. | ||
| public IReadOnlyList<Exception> Diagnostics => diagnostics.AsReadOnly(); | ||
|
|
||
| public bool HasErrors => diagnostics.Count > 0; |
Two test additions that turn the "phase 1 is dormant" promise into an
enforceable CI invariant:
1. **Unit (DiagnosticCollectorTest.cs)**
- Env var parsing: 13 [TestCase] entries covering P_COMPILER_COLLECT_ERRORS
values "1"/"true"/"True"/"TRUE"/"yes"/"anything-else"/"0"/"false"/"False"/
"FALSE"/""/" "/null. Each saves and restores the env var so tests
stay isolated. Asserts the collector mode agrees with the config flag.
- Live-view test: confirms the IReadOnlyList<T> wrapper returned from
DefaultDiagnosticCollector.Diagnostics reflects items appended after
the property was read. Compiler.cs's flush pass depends on this; a
future "defensive copy" change here would silently lose late
diagnostics.
2. **Regression (Phase1DormancyTest.cs, new file)**
- One parametrized test per leaf directory under
RegressionTests/{Combined,Feature1..4,Integration}/{Correct,StaticError}
auto-discovered via the existing TestCaseLoader.
- For each input, runs the compiler twice: once strict (today), once
collecting (ContinueOnError=true). Asserts identical exit code AND
identical stderr stream.
- Configurations are built directly so the env var doesn't have to be
mutated, keeping parallel test fixtures isolated.
- Tagged [Category("Phase1Dormancy")] so it can be excluded if Phase 2
intentionally diverges the two modes.
When Phase 2 starts converting throw sites to record-and-continue, this
fixture WILL begin failing on multi-error inputs — that's the correct
signal that dormancy is no longer the contract. At that point split the
fixture (Correct/ subset still demands identity; StaticError/ subset
tolerates more errors in collecting mode) or retire it.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The AstEmitterExhaustivenessTests fixture (added on master in #961) guards that every concrete IPExpr subtype is either implemented by the imperative IExpressionEmitter<T> contract or explicitly excluded with justification. The Phase 1 ErrorExpr sentinel is correctly outside the emitter contract — by construction it never reaches a backend, since Compiler.cs aborts before IRTransformer/GenerateCode when the diagnostic collector HasErrors, and ErrorExpr doesn't implement IExprTerm so any accidental leak fails loudly at a cast site rather than silently producing garbage code. Add it to the exclusion set with that justification inline. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Per Copilot's second-round review on #957: the previous code returned diagnostics.AsReadOnly() from the Diagnostics getter, allocating a fresh ReadOnlyCollection<T> wrapper on every property read. The wrapper is cheap but Compiler.cs's end-of-typecheck flush pass + future Phase 2 visitors will read this often, and the allocation is avoidable. Move the AsReadOnly() call into the constructor and cache the ReadOnlyCollection<T> instance in a private field. ReadOnlyCollection<T> is a live view over the backing list (it reflects new items as Report appends them), so caching doesn't break the existing live-view test in DiagnosticCollectorTest. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Multi-agent review of PRs #957/#965/#967 identified 3 HIGH-severity harness fragilities (could silently mask real bugs) and ~10 coverage gaps. This commit applies the harness fixes + the 4 highest-priority new pinned tests + 1 doc fix. ## Harness fixes ### 1. Count ALL diagnostic markers, not just `[Error:]` Previous: `CountOccurrences(stderr, "[Error:]") + CountOccurrences(stderr, "[Parser Error:]")`. Problem: Compiler.cs also emits `[NotSupportedError:]`, `[NotImplementedError:]`, and per-backend `[<entry> Compiling Generated Code:]` markers. A regression that converts a real diagnostic into a `NotSupportedException` would be counted as zero errors, making `CollectingReportsAtLeastAsManyErrorsAsStrict`'s `>=` invariant trivially pass with both sides zero — masking the regression entirely. We already hit this once when the ForeachInvariantError.p test tripped a `NotSupportedException` on the PVerifier `invariant` keyword. Fix: new `CountErrorMarkers(stderr)` uses two regexes — one for `[<text>Error<text>:]` (catches all five marker variants) and one for the explicit `Compiling Generated Code` marker. Applied to both Phase1DormancyTest.cs and MultiErrorAcceptanceTest.cs. ### 2. Catch ONLY TranslationException Previous: `catch (Exception e)` in both fixture's RunOnce helpers. Problem: Phase 2's MultiAgent audit explicitly avoided this pattern. NREs, `Debug.Assert` failures, and other runtime exceptions get folded into `exitCode = -1` and identical stderr text — two parallel NREs produce equal output, equal exit codes, equal error counts (0), and the `StrictAndCollectingAgreeOnValidPrograms` equality assertion passes green despite both runs crashing. Fix: catch only `TranslationException`. Other exceptions propagate so genuine bugs surface as test failures with full stack traces. ### 3. Mark env-var test [NonParallelizable] Previous: `DiagnosticCollectorTest.ContinueOnError_ReadsEnvVar` mutates the process-global `P_COMPILER_COLLECT_ERRORS` env var. Problem: `RegressionTests.CompileOnlyRegressionTests` is `[Parallelizable(ParallelScope.Children)]`. With `dotnet test` running fixtures concurrently, a parallel CompilerConfiguration construction could read the temporarily-set env var and run a strict-mode regression test in collecting mode — silently masking a strict-mode regression as a flaky-pass. Fix: add `[NonParallelizable]` to the env-var test. ## Doc fix `MultipleErrors.p` line 11 said the 4th error was "arg count mismatch". The actual mechanism is "payload TYPE mismatch via single-arg CheckArgument branch" because `(1,2,3)` parses as a single unnamed-tuple argument. Count is 1 either way, but the prose was actively misleading. ## 4 new pinned coverage tests The audit identified ~12 throw sites in ExprVisitor/StatementVisitor with NO multi-error coverage at all. The four highest-priority additions: 1. **CtorArgErrors.p (1/3)** — VisitCtorExpr arg-pre-visit recovery. Validates that constructor args are visited BEFORE interface lookup, so missing-decl errors in args surface even when the interface itself is unknown. 2. **FunCallArgErrors.p (1/4)** — VisitFunCallExpr Function branch. Two calls combining argument errors with call-shape errors (arity + type mismatch). Validates cascade-suppression of arg[0] while arg[1] mismatch still surfaces. 3. **MultiFunctionPerMachine.p (1/3)** — Phase 3 per-FUNCTION isolation distinct from per-machine. Three functions in ONE machine, one error each. Catches a regression where Phase 3's per-function TolerantStep wrapper stops iterating after the first failing function. 4. **GotoRecoveryErrors.p (1/4)** — VisitGotoStmt rvalue-pre-visit recovery in both the missing-state branch and the arity-mismatch branch. Validates that argument errors surface even when the goto itself is malformed. ## Deferred (lower-priority audit findings) - 6 more coverage gaps (CastExpr, ChooseExpr, FormatString, dup tuple field, RaiseStmt non-void, etc.). Less critical because each is covered by Phase1DormancyTest's weak `>=` baseline. - Harness #5/#7 (scratch-dir GetHashCode collisions, parallel dotnet build). The Ubuntu flake from earlier may relate but hasn't recurred since the projectRoot fix. Tracked separately. - Harness #8 (test timeouts). Nice-to-have; current CI runs reliably. - Identity-aware assertions (stderr substring matches) — would lock in which error fires first, but adds maintenance cost. Defer. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Lights up the diagnostic collector scaffolded in #957. In strict mode (default), behavior is unchanged because handler.Diagnostics.Report re-throws immediately. In collecting mode (P_COMPILER_COLLECT_ERRORS=1) the type checker now reports all independent expression / statement errors in one pass instead of aborting at the first. - CheckAssignable(handler, ctx, expected, actual): report-and-return-false variant of IsAssignableFrom. ErrorType on either side is silently treated as compatible, preventing one upstream error from generating a chain of "incompatible operand" diagnostics downstream. - CheckArgument and ValidatePayloadTypes route through CheckAssignable so call-site / send-site argument checks inherit the same suppression. Every visit method follows the same convention: 1. Visit children first (so their internal errors surface). 2. If any child has ErrorType, return new ErrorExpr(context) without further checks (combiner rule). 3. Each `throw handler.X(...)` becomes `handler.Diagnostics.Report(handler.X(...)); return new ErrorExpr(...);` with remaining compatibility checks routed via CheckAssignable. Notable per-site recoveries: - VisitBinExpr: a single ErrorType guard at the top covers all 14 throws inside the switch. - VisitFunCallExpr / VisitCtorExpr: arguments are visited *before* callee/interface resolution so their internal errors surface even when the callee is unknown. - VisitCastExpr: resolve target type even when sub-expression errored, so a malformed type expression in a cast also gets reported. Statements either continue building the AST node with the values they have (most cases) or return new NoStmt(context) when there's nothing meaningful to construct (missing variable / interface / function / state / event): - Missing-declaration sites in CtorStmt, FunCallStmt, ForeachStmt, GotoStmt, ReceiveStmt → NoStmt placeholder. - Type-mismatch sites in AssertStmt, AssumeStmt, PrintStmt, ReturnStmt, AssignStmt, IfStmt, WhileStmt etc. → Report and continue with the typed expression as-is (downstream sees ErrorType if upstream). - SendStmt / RaiseStmt / AnnounceStmt: ErrorType guard around the null-event + assignability checks so an upstream-errored event expression doesn't generate spurious diagnostics. The Phase 1 fixture asserted bit-identical stderr in both modes. That's no longer the right invariant for StaticError/ inputs — collecting mode can now legitimately report MORE errors per file. Split into: - StrictAndCollectingAgreeOnValidPrograms: Correct/ only. Both modes must produce empty stderr and exit 0. - CollectingReportsAtLeastAsManyErrorsAsStrict: StaticError/ only. Both exit 1; collecting count >= strict count. Catches the only regression that matters: collecting silently suppressing a strict error. A curated P file with 4 deliberately-independent errors: - bool assigned to int - missing-declaration (must not cascade into "wrong type") - int + string - send arg-count mismatch Strict mode reports exactly 1 (aborts on first). Collecting mode reports exactly 4 with no spurious cascade diagnostics. Pinning both counts catches regressions to either the throw-site conversion or the cascade-suppression rules. handler.Diagnostics.Report re-throws when ContinueOnError=false. Every existing static-error test passes unchanged — the throw-site conversion preserves the historical "first error wins, abort" behavior bit-for-bit in strict mode. Pass-level tolerance classification in Analyzer.cs — decide per pass whether it can run on a partially-broken AST (e.g. capability check and module-system passes need a clean AST; control-flow check can tolerate per-function errors). Estimated 2-3 days. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Multi-agent review of PRs #957/#965/#967 identified 3 HIGH-severity harness fragilities (could silently mask real bugs) and ~10 coverage gaps. This commit applies the harness fixes + the 4 highest-priority new pinned tests + 1 doc fix. ## Harness fixes ### 1. Count ALL diagnostic markers, not just `[Error:]` Previous: `CountOccurrences(stderr, "[Error:]") + CountOccurrences(stderr, "[Parser Error:]")`. Problem: Compiler.cs also emits `[NotSupportedError:]`, `[NotImplementedError:]`, and per-backend `[<entry> Compiling Generated Code:]` markers. A regression that converts a real diagnostic into a `NotSupportedException` would be counted as zero errors, making `CollectingReportsAtLeastAsManyErrorsAsStrict`'s `>=` invariant trivially pass with both sides zero — masking the regression entirely. We already hit this once when the ForeachInvariantError.p test tripped a `NotSupportedException` on the PVerifier `invariant` keyword. Fix: new `CountErrorMarkers(stderr)` uses two regexes — one for `[<text>Error<text>:]` (catches all five marker variants) and one for the explicit `Compiling Generated Code` marker. Applied to both Phase1DormancyTest.cs and MultiErrorAcceptanceTest.cs. ### 2. Catch ONLY TranslationException Previous: `catch (Exception e)` in both fixture's RunOnce helpers. Problem: Phase 2's MultiAgent audit explicitly avoided this pattern. NREs, `Debug.Assert` failures, and other runtime exceptions get folded into `exitCode = -1` and identical stderr text — two parallel NREs produce equal output, equal exit codes, equal error counts (0), and the `StrictAndCollectingAgreeOnValidPrograms` equality assertion passes green despite both runs crashing. Fix: catch only `TranslationException`. Other exceptions propagate so genuine bugs surface as test failures with full stack traces. ### 3. Mark env-var test [NonParallelizable] Previous: `DiagnosticCollectorTest.ContinueOnError_ReadsEnvVar` mutates the process-global `P_COMPILER_COLLECT_ERRORS` env var. Problem: `RegressionTests.CompileOnlyRegressionTests` is `[Parallelizable(ParallelScope.Children)]`. With `dotnet test` running fixtures concurrently, a parallel CompilerConfiguration construction could read the temporarily-set env var and run a strict-mode regression test in collecting mode — silently masking a strict-mode regression as a flaky-pass. Fix: add `[NonParallelizable]` to the env-var test. ## Doc fix `MultipleErrors.p` line 11 said the 4th error was "arg count mismatch". The actual mechanism is "payload TYPE mismatch via single-arg CheckArgument branch" because `(1,2,3)` parses as a single unnamed-tuple argument. Count is 1 either way, but the prose was actively misleading. ## 4 new pinned coverage tests The audit identified ~12 throw sites in ExprVisitor/StatementVisitor with NO multi-error coverage at all. The four highest-priority additions: 1. **CtorArgErrors.p (1/3)** — VisitCtorExpr arg-pre-visit recovery. Validates that constructor args are visited BEFORE interface lookup, so missing-decl errors in args surface even when the interface itself is unknown. 2. **FunCallArgErrors.p (1/4)** — VisitFunCallExpr Function branch. Two calls combining argument errors with call-shape errors (arity + type mismatch). Validates cascade-suppression of arg[0] while arg[1] mismatch still surfaces. 3. **MultiFunctionPerMachine.p (1/3)** — Phase 3 per-FUNCTION isolation distinct from per-machine. Three functions in ONE machine, one error each. Catches a regression where Phase 3's per-function TolerantStep wrapper stops iterating after the first failing function. 4. **GotoRecoveryErrors.p (1/4)** — VisitGotoStmt rvalue-pre-visit recovery in both the missing-state branch and the arity-mismatch branch. Validates that argument errors surface even when the goto itself is malformed. ## Deferred (lower-priority audit findings) - 6 more coverage gaps (CastExpr, ChooseExpr, FormatString, dup tuple field, RaiseStmt non-void, etc.). Less critical because each is covered by Phase1DormancyTest's weak `>=` baseline. - Harness #5/#7 (scratch-dir GetHashCode collisions, parallel dotnet build). The Ubuntu flake from earlier may relate but hasn't recurred since the projectRoot fix. Tracked separately. - Harness #8 (test timeouts). Nice-to-have; current CI runs reliably. - Identity-aware assertions (stderr substring matches) — would lock in which error fires first, but adds maintenance cost. Defer. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* Phase 2: convert ExprVisitor + StatementVisitor to record-and-continue Lights up the diagnostic collector scaffolded in #957. In strict mode (default), behavior is unchanged because handler.Diagnostics.Report re-throws immediately. In collecting mode (P_COMPILER_COLLECT_ERRORS=1) the type checker now reports all independent expression / statement errors in one pass instead of aborting at the first. - CheckAssignable(handler, ctx, expected, actual): report-and-return-false variant of IsAssignableFrom. ErrorType on either side is silently treated as compatible, preventing one upstream error from generating a chain of "incompatible operand" diagnostics downstream. - CheckArgument and ValidatePayloadTypes route through CheckAssignable so call-site / send-site argument checks inherit the same suppression. Every visit method follows the same convention: 1. Visit children first (so their internal errors surface). 2. If any child has ErrorType, return new ErrorExpr(context) without further checks (combiner rule). 3. Each `throw handler.X(...)` becomes `handler.Diagnostics.Report(handler.X(...)); return new ErrorExpr(...);` with remaining compatibility checks routed via CheckAssignable. Notable per-site recoveries: - VisitBinExpr: a single ErrorType guard at the top covers all 14 throws inside the switch. - VisitFunCallExpr / VisitCtorExpr: arguments are visited *before* callee/interface resolution so their internal errors surface even when the callee is unknown. - VisitCastExpr: resolve target type even when sub-expression errored, so a malformed type expression in a cast also gets reported. Statements either continue building the AST node with the values they have (most cases) or return new NoStmt(context) when there's nothing meaningful to construct (missing variable / interface / function / state / event): - Missing-declaration sites in CtorStmt, FunCallStmt, ForeachStmt, GotoStmt, ReceiveStmt → NoStmt placeholder. - Type-mismatch sites in AssertStmt, AssumeStmt, PrintStmt, ReturnStmt, AssignStmt, IfStmt, WhileStmt etc. → Report and continue with the typed expression as-is (downstream sees ErrorType if upstream). - SendStmt / RaiseStmt / AnnounceStmt: ErrorType guard around the null-event + assignability checks so an upstream-errored event expression doesn't generate spurious diagnostics. The Phase 1 fixture asserted bit-identical stderr in both modes. That's no longer the right invariant for StaticError/ inputs — collecting mode can now legitimately report MORE errors per file. Split into: - StrictAndCollectingAgreeOnValidPrograms: Correct/ only. Both modes must produce empty stderr and exit 0. - CollectingReportsAtLeastAsManyErrorsAsStrict: StaticError/ only. Both exit 1; collecting count >= strict count. Catches the only regression that matters: collecting silently suppressing a strict error. A curated P file with 4 deliberately-independent errors: - bool assigned to int - missing-declaration (must not cascade into "wrong type") - int + string - send arg-count mismatch Strict mode reports exactly 1 (aborts on first). Collecting mode reports exactly 4 with no spurious cascade diagnostics. Pinning both counts catches regressions to either the throw-site conversion or the cascade-suppression rules. handler.Diagnostics.Report re-throws when ContinueOnError=false. Every existing static-error test passes unchanged — the throw-site conversion preserves the historical "first error wins, abort" behavior bit-for-bit in strict mode. Pass-level tolerance classification in Analyzer.cs — decide per pass whether it can run on a partially-broken AST (e.g. capability check and module-system passes need a clean AST; control-flow check can tolerate per-function errors). Estimated 2-3 days. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Fix NamedTupleBody dict-collision + add stack traces to test harness Two fixes for the Phase 2 CI failures: ## 1. NamedTupleBody dict collision (real bug) In collecting mode, the recovery for a duplicate field name added an entry with the *original* name to the entries[] array. Downstream NamedTupleType construction builds a Dictionary keyed by entry name, which throws InvalidOperationException on the collision. Test `Feature4DataTypes | StaticError | namedDuplicateField2` failed with "An item with the same key has already been added. Key: a". Fix: mangle the duplicate's name with a unique suffix (`$dup$N`). The duplicate-field diagnostic is already reported to the user via handler.Diagnostics.Report; mangling just keeps the AST well-formed for any subsequent type checks that happen to inspect the tuple. ## 2. Stack traces in test harness (diagnostic improvement) Phase1DormancyTest + MultiErrorAcceptanceTest both swallow uncaught exceptions from new Compiler().Compile(...) and report only e.Message. Several CI failures showed NREs on valid programs in strict mode with no way to identify which compiler pass NRE'd. Now we print `e.GetType().Name` and the full stack trace into stderr so the next CI run pinpoints the responsible code site. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Fix Phase 2 test harness: pass scratchDir as projectRoot PCheckerCodeGenerator.Compile dereferences job.ProjectRootPath.FullName at the dotnet-build stage. My fixtures used the 5-arg CompilerConfiguration constructor without supplying projectRoot, so the PChecker backend NRE'd during the second stage of Compile on every Correct program. This was masked in Phase 1 because both modes (strict and collecting) NRE'd symmetrically, and the Phase1DormancyTest's identical-stderr assertion still held. The Phase 2 split asserts exit code == 0 on valid programs, which exposed the bug. Mirror PCheckerRunner.cs by passing scratchDir as the projectRoot. The MultiErrorAcceptanceTest fixture also gets the same fix as defence in depth — its tests stop at type-check (HasErrors causes Compiler.cs to return early before reaching the backend), so the bug wouldn't fire there in practice, but consistency keeps the next person from re- discovering this. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Address Phase 2 Copilot review: 5 cascade-suppression fixes All five comments from Copilot's review of #965 were valid recovery holes — places where the converted code suppressed a diagnostic too aggressively or failed to propagate ErrorType. Fixes: ## 1. ValidatePayloadTypes: don't swallow the arity check Previous: any argument with ErrorType caused an early return from the whole helper, hiding the independent IncorrectArgumentCount diagnostic when arity was wrong AND one arg also errored upstream. Fix: remove the blanket early-return on `arguments.Any(a => a.Type is ErrorType)`. Per-argument cascade suppression is already handled by CheckAssignable's own ErrorType short-circuit, so the arity check (a property of the call-site, not the arg types) now fires independently. Added an explicit arity check inside the tuple branch since the previous code relied on Zip silently truncating mismatched counts. ## 2. VisitForeachStmt: visit invariants when iterator var is missing Previous: missing-iterator recovery visited the collection and body but not `context._invariants`. Type errors inside loop invariants were silently suppressed in collecting mode. Fix: visit each invariant via exprVisitor.Visit before returning NoStmt. ## 3. VisitReceiveStmt: visit handler bodies on spec-illegal monitor op Previous: when receive was illegal (spec machine), the method reported IllegalMonitorOperation and returned NoStmt immediately, suppressing all diagnostics inside the handler bodies. Fix: mirror what Send/Announce/Raise do for their child expressions — construct each recvCase handler (with its scope and parameter) and run FunctionBodyVisitor.PopulateMethod to surface body errors before returning NoStmt. Event-id lookups and duplicate-case checks are intentionally skipped since the receive itself is already illegal. ## 4. VisitTestExpr: propagate ErrorType from instance Previous: an instance with ErrorType still produced a bool-typed TestExpr, masking the cascade for downstream consumers. Fix: do the kind-identifier lookup first (so missing-decl on the kind is still reported even when instance errored), then bail to ErrorExpr if instance.Type is ErrorType. Preserves both diagnostics' independence. ## 5. VisitTargetsExpr / VisitFlyingExpr / VisitSentExpr: propagate ErrorType Previous: these three constructed bool-typed nodes regardless of operand state, breaking the ErrorType propagation convention documented in ExprVisitor's class header. Fix: add the same early-guard pattern used elsewhere — if any visited child has ErrorType, return new ErrorExpr(context) before constructing the typed node. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Phase 2 robustness: multi-agent audit findings + new pinned tests After 3 rounds of Copilot findings, ran a 5-agent parallel audit covering cascade-suppression conventions, AST-recovery downstream NRE risks, strict- mode preservation, test coverage gaps, and Phase 3 forward compatibility. This commit applies the HIGH and MEDIUM severity findings and adds three new pinned-count acceptance tests. ## Why Copilot kept finding things Cascade-suppression has ~5 rules (producer, propagator, combiner, lvalue, call) and the conversion touched ~30 visit methods. Verifying every method follows every rule is combinatorially hard for a single reviewer. The multi-agent audit caught a third set of bugs that a single read missed — this time we also added programmatic acceptance tests that pin error counts so the next regression fires loudly. ## Tier 1: cascade-suppression fixes (HIGH severity) **1. `VisitReceiveStmt` spec-machine recovery now visits event-id lookups.** Copilot's first round added handler body visiting; the audit caught that undeclared events in the `case ...` lists were still being swallowed. **2. `VisitQuantExpr` diff-bound recovery now visits the body.** The two early-returns (arity mismatch, bound-type mismatch) skipped the body, so nested type errors inside `forall ... :: <body>` quantifiers got lost when the bound was malformed. **3. `VisitQuantExpr` bound-ErrorType short-circuit.** When TypeResolver reported a missing declaration on the bound type, the diff switch defaulted to a spurious "expected Event" cascade. New explicit `if (bound[0].Type is ErrorType)` guard bails silently and visits the body anyway. ## Tier 2: defense-in-depth for Phase 3 **4. Analyzer.cs HasErrors gate before pass 8.** ModuleSystemDeclarations / ModuleSystemTypeChecker (passes 8-10) contain hard casts (`(NamedTupleExpr)`, `(NamedTupleType)`, `(SeqLiteralExpr)`) that ErrorExpr / ErrorType would trip with InvalidCastException in collecting mode on a partial AST. Today this is masked by Compiler.cs's HasErrors gate at the END of compilation — but those passes still RAN on the broken AST. Skip them when diagnostics are present. Phase 3 will replace this single gate with per-pass tolerance classification. **5. `FunctionBodyVisitor.PopulateMethod` backfills empty Body.** `Function.IsForeign = Body == null`, so a function that errored partway through population would silently be misclassified as foreign. Backfill `new CompoundStmt(...)` on failure. Belt-and-braces — Compiler.cs's HasErrors gate already ensures these don't reach IR or backends, but preserving the "non-foreign ⇒ has body" invariant simplifies Phase 3. **6. `ValidatePayloadTypes` null payload safety.** Pass 2a (MachineChecker) may leave PayloadType null when an event/ interface declaration itself errored. Treat null as ErrorType bail so ValidatePayloadTypes doesn't NRE in collecting mode when called via pass-3 recovery paths. ## Tier 3: pinned-count acceptance tests Refactored `MultiErrorAcceptanceTest` from one hard-coded file to a `[TestCaseSource]`-driven fixture. Each row pins (strict count, collecting count) for a curated `.p` file. When counts change intentionally, update the row AND the file's header comment. Three new pinned cases: - **`NestedExprErrors.p`** (1 / 2): single expression with two nested missing-declarations. Validates combiner rule — `undeclaredA.foo + undeclaredB.bar * "str"` must NOT produce extra cascades on `+` or `*`. - **`ForeachInvariantError.p`** (1 / 2): foreach with valid iterator, body error + invariant error. Validates the happy-path branch of VisitForeachStmt visits both body and each invariant. - **`SpecReceiveBodyError.p`** (1 / 3): `receive` inside a spec machine: IllegalMonitorOperation + undeclared event + body type mismatch. Validates the new event-id lookup + the existing handler- body recovery in the spec-machine branch. Diagnostic guidance for future failures is in MultiErrorAcceptanceTest's class doc. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Fix Phase 2 build: add `using AST` for IPStmt in FunctionBodyVisitor CompoundStmt's constructor takes IEnumerable<IPStmt>; the empty-body fail-safe I just added references IPStmt without the matching using. Build broke on all 3 platforms with CS0246. Trivial import fix. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Drop PVerifier-only `invariant` from foreach test ForeachInvariantError.p triggered a NotSupportedException because `invariant` is gated to PVerifier mode — the regression suite runs in default PChecker mode which rejects PVerifier tokens at the lexer. The MultiErrorAcceptanceTest pinned 1/2 errors but the harness got 0/0 because [NotSupportedError:] isn't counted as [Error:]. Rename ForeachInvariantError → ForeachBodyErrors and rework to exercise two body errors (MissingDeclaration + TypeMismatch) without any PVerifier syntax. Still validates the audit-flagged "happy path visits the body" invariant; the parallel "happy path visits invariants" fix in StatementVisitor.VisitForeachStmt remains in code and will get its own regression once a PVerifier-mode test harness exists. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Reconcile PR #963's TryResolveStateForInstance with Phase 2's hasKind precheck Subtle bug noticed while resolving the rebase conflict: the hasKind precheck loop used `table.Lookup(name, out State _)` (cross-machine lookup) for the State branch, but the actual dispatch below used `TryResolveStateForInstance` (PR #963's narrowed lookup). Failure path: `myA is S2` where MachineA has no S2 but MachineB does. hasKind succeeds via the cross-machine lookup (S2 found in B). The ErrorType check passes. Then dispatch falls through Machine/Event, and `TryResolveStateForInstance` correctly returns false (S2 not in A's scope) — but the trailing "Unreachable" comment becomes REACHABLE and silently returns ErrorExpr without firing the MissingDeclaration diagnostic. The user gets exit code 1 with NO error message. Fix: change the hasKind precheck's State branch to use `TryResolveStateForInstance` so `hasKind` reflects whether a USABLE binding exists, not just any binding anywhere in the program. Now the cross-machine S2 case correctly fails the precheck and emits the "could not find machine, event, or state 'S2'" diagnostic. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Multi-agent review of PRs #957/#965/#967 identified 3 HIGH-severity harness fragilities (could silently mask real bugs) and ~10 coverage gaps. This commit applies the harness fixes + the 4 highest-priority new pinned tests + 1 doc fix. ## Harness fixes ### 1. Count ALL diagnostic markers, not just `[Error:]` Previous: `CountOccurrences(stderr, "[Error:]") + CountOccurrences(stderr, "[Parser Error:]")`. Problem: Compiler.cs also emits `[NotSupportedError:]`, `[NotImplementedError:]`, and per-backend `[<entry> Compiling Generated Code:]` markers. A regression that converts a real diagnostic into a `NotSupportedException` would be counted as zero errors, making `CollectingReportsAtLeastAsManyErrorsAsStrict`'s `>=` invariant trivially pass with both sides zero — masking the regression entirely. We already hit this once when the ForeachInvariantError.p test tripped a `NotSupportedException` on the PVerifier `invariant` keyword. Fix: new `CountErrorMarkers(stderr)` uses two regexes — one for `[<text>Error<text>:]` (catches all five marker variants) and one for the explicit `Compiling Generated Code` marker. Applied to both Phase1DormancyTest.cs and MultiErrorAcceptanceTest.cs. ### 2. Catch ONLY TranslationException Previous: `catch (Exception e)` in both fixture's RunOnce helpers. Problem: Phase 2's MultiAgent audit explicitly avoided this pattern. NREs, `Debug.Assert` failures, and other runtime exceptions get folded into `exitCode = -1` and identical stderr text — two parallel NREs produce equal output, equal exit codes, equal error counts (0), and the `StrictAndCollectingAgreeOnValidPrograms` equality assertion passes green despite both runs crashing. Fix: catch only `TranslationException`. Other exceptions propagate so genuine bugs surface as test failures with full stack traces. ### 3. Mark env-var test [NonParallelizable] Previous: `DiagnosticCollectorTest.ContinueOnError_ReadsEnvVar` mutates the process-global `P_COMPILER_COLLECT_ERRORS` env var. Problem: `RegressionTests.CompileOnlyRegressionTests` is `[Parallelizable(ParallelScope.Children)]`. With `dotnet test` running fixtures concurrently, a parallel CompilerConfiguration construction could read the temporarily-set env var and run a strict-mode regression test in collecting mode — silently masking a strict-mode regression as a flaky-pass. Fix: add `[NonParallelizable]` to the env-var test. ## Doc fix `MultipleErrors.p` line 11 said the 4th error was "arg count mismatch". The actual mechanism is "payload TYPE mismatch via single-arg CheckArgument branch" because `(1,2,3)` parses as a single unnamed-tuple argument. Count is 1 either way, but the prose was actively misleading. ## 4 new pinned coverage tests The audit identified ~12 throw sites in ExprVisitor/StatementVisitor with NO multi-error coverage at all. The four highest-priority additions: 1. **CtorArgErrors.p (1/3)** — VisitCtorExpr arg-pre-visit recovery. Validates that constructor args are visited BEFORE interface lookup, so missing-decl errors in args surface even when the interface itself is unknown. 2. **FunCallArgErrors.p (1/4)** — VisitFunCallExpr Function branch. Two calls combining argument errors with call-shape errors (arity + type mismatch). Validates cascade-suppression of arg[0] while arg[1] mismatch still surfaces. 3. **MultiFunctionPerMachine.p (1/3)** — Phase 3 per-FUNCTION isolation distinct from per-machine. Three functions in ONE machine, one error each. Catches a regression where Phase 3's per-function TolerantStep wrapper stops iterating after the first failing function. 4. **GotoRecoveryErrors.p (1/4)** — VisitGotoStmt rvalue-pre-visit recovery in both the missing-state branch and the arity-mismatch branch. Validates that argument errors surface even when the goto itself is malformed. ## Deferred (lower-priority audit findings) - 6 more coverage gaps (CastExpr, ChooseExpr, FormatString, dup tuple field, RaiseStmt non-void, etc.). Less critical because each is covered by Phase1DormancyTest's weak `>=` baseline. - Harness #5/#7 (scratch-dir GetHashCode collisions, parallel dotnet build). The Ubuntu flake from earlier may relate but hasn't recurred since the projectRoot fix. Tracked separately. - Harness #8 (test timeouts). Nice-to-have; current CI runs reliably. - Identity-aware assertions (stderr substring matches) — would lock in which error fires first, but adds maintenance cost. Defer. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* Phase 3: pass-level tolerance classification in Analyzer.cs Stacks on Phase 2 (#965). Closes the multi-error type-checker initiative by adding per-item try/catch isolation to the analyzer's gathering passes and a HasErrors gate before the analysis passes that assume a clean AST. ## What changed ### `TolerantStep` helper New private helper in `Analyzer.cs`: private static void TolerantStep(ITranslationErrorHandler handler, Action body) { try { body(); } catch (TranslationException e) when (handler.Diagnostics.ContinueOnError) { handler.Diagnostics.Report(e); } } In strict mode (ContinueOnError == false) the `when` filter is false so the exception propagates exactly as before — bit-for-bit preservation of today's throw-on-first-error semantics. In collecting mode the catch captures the TranslationException into the collector and the loop continues with the next item. Only TranslationException is caught; NREs and other runtime exceptions still propagate so genuine bugs surface loudly. ### Tolerant gathering passes (2a, 2b, 3, 3b, 7) Each iteration wrapped in `TolerantStep`: - **2a MachineChecker.Validate** — per-machine. One bad machine no longer aborts MachineChecker for siblings. - **3 FunctionBody + Validator** — per-function. One bad function (TypeResolver throw on a bad var decl, missing-paths-return, etc.) no longer aborts pass 3 for the rest of the program. - **3b Invariants / Axioms / AssumeOnStarts / Pures / Foreign** — per-item across all five subgroups. One bad invariant no longer blocks axiom checking. - **2b ValidateNoStaticHandlers** — per-machine. - **7 InferMachineCreates** — per-machine. Already gated by HasErrors in practice, but the wrapper is cheap insurance. ### Gathering→analysis boundary New HasErrors gate placed after pass 2b (the last gathering pass) and before pass 4 (ApplyPropagations): if (handler.Diagnostics.HasErrors) return globalScope; If any gathering pass collected an error, the analysis passes (4-7) and the module-system passes (8-10) are skipped. The duplicate gate before pass 8 (added in Phase 2 robustness) is kept as defense-in-depth — once Phase 4+ converts pass-5 capability checks to Report, that second gate will earn its keep. ## Why this matters Without Phase 3, a single machine with a bad start state aborts MachineChecker mid-pass, so a sibling machine's MoreThanOneParameterForHandlers error never gets reported. A single function with a malformed var decl aborts pass 3, so none of the OTHER functions' bodies get type-checked at all. Combined with Phase 2's expression-level recovery, Phase 3 means a user with N independent errors (in any combination of expressions, statements, machines, or functions) sees ALL of them in one compile. ## Tests ### New: `MultiMachineErrors.p` + pinned counts Three machines, each with one independent error (bool->int, missing declaration, int+string binop). Pinned in MultiErrorAcceptanceTest: strict=1, collecting=3. Validates that one machine's error doesn't clobber its siblings' diagnostics in collecting mode. ### Existing: All Phase 2 acceptance tests unchanged MultipleErrors (1/4), NestedExprErrors (1/2), ForeachBodyErrors (1/2), SpecReceiveBodyError (1/3). Same counts as Phase 2 — Phase 3 doesn't change behavior within a single machine. ### Existing: Phase1DormancyTest baseline The "collecting >= strict count" invariant on every Correct/StaticError test still holds. Phase 3 only *adds* errors (when multi-machine / multi-function scenarios exist); it never removes. ## Phase 4 (deferred, NOT in this PR) Convert pass 5 (capability checks) and pass 6 (ControlFlowChecker) throw sites to Report so capability and control-flow violations also accumulate across machines instead of aborting on first. The current HasErrors gate already prevents these passes from running on a broken AST, so they remain RequiresClean for now. Estimated: ~1 day if warranted by user feedback. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Multi-agent review: harness fixes + 4 new pinned coverage tests Multi-agent review of PRs #957/#965/#967 identified 3 HIGH-severity harness fragilities (could silently mask real bugs) and ~10 coverage gaps. This commit applies the harness fixes + the 4 highest-priority new pinned tests + 1 doc fix. ## Harness fixes ### 1. Count ALL diagnostic markers, not just `[Error:]` Previous: `CountOccurrences(stderr, "[Error:]") + CountOccurrences(stderr, "[Parser Error:]")`. Problem: Compiler.cs also emits `[NotSupportedError:]`, `[NotImplementedError:]`, and per-backend `[<entry> Compiling Generated Code:]` markers. A regression that converts a real diagnostic into a `NotSupportedException` would be counted as zero errors, making `CollectingReportsAtLeastAsManyErrorsAsStrict`'s `>=` invariant trivially pass with both sides zero — masking the regression entirely. We already hit this once when the ForeachInvariantError.p test tripped a `NotSupportedException` on the PVerifier `invariant` keyword. Fix: new `CountErrorMarkers(stderr)` uses two regexes — one for `[<text>Error<text>:]` (catches all five marker variants) and one for the explicit `Compiling Generated Code` marker. Applied to both Phase1DormancyTest.cs and MultiErrorAcceptanceTest.cs. ### 2. Catch ONLY TranslationException Previous: `catch (Exception e)` in both fixture's RunOnce helpers. Problem: Phase 2's MultiAgent audit explicitly avoided this pattern. NREs, `Debug.Assert` failures, and other runtime exceptions get folded into `exitCode = -1` and identical stderr text — two parallel NREs produce equal output, equal exit codes, equal error counts (0), and the `StrictAndCollectingAgreeOnValidPrograms` equality assertion passes green despite both runs crashing. Fix: catch only `TranslationException`. Other exceptions propagate so genuine bugs surface as test failures with full stack traces. ### 3. Mark env-var test [NonParallelizable] Previous: `DiagnosticCollectorTest.ContinueOnError_ReadsEnvVar` mutates the process-global `P_COMPILER_COLLECT_ERRORS` env var. Problem: `RegressionTests.CompileOnlyRegressionTests` is `[Parallelizable(ParallelScope.Children)]`. With `dotnet test` running fixtures concurrently, a parallel CompilerConfiguration construction could read the temporarily-set env var and run a strict-mode regression test in collecting mode — silently masking a strict-mode regression as a flaky-pass. Fix: add `[NonParallelizable]` to the env-var test. ## Doc fix `MultipleErrors.p` line 11 said the 4th error was "arg count mismatch". The actual mechanism is "payload TYPE mismatch via single-arg CheckArgument branch" because `(1,2,3)` parses as a single unnamed-tuple argument. Count is 1 either way, but the prose was actively misleading. ## 4 new pinned coverage tests The audit identified ~12 throw sites in ExprVisitor/StatementVisitor with NO multi-error coverage at all. The four highest-priority additions: 1. **CtorArgErrors.p (1/3)** — VisitCtorExpr arg-pre-visit recovery. Validates that constructor args are visited BEFORE interface lookup, so missing-decl errors in args surface even when the interface itself is unknown. 2. **FunCallArgErrors.p (1/4)** — VisitFunCallExpr Function branch. Two calls combining argument errors with call-shape errors (arity + type mismatch). Validates cascade-suppression of arg[0] while arg[1] mismatch still surfaces. 3. **MultiFunctionPerMachine.p (1/3)** — Phase 3 per-FUNCTION isolation distinct from per-machine. Three functions in ONE machine, one error each. Catches a regression where Phase 3's per-function TolerantStep wrapper stops iterating after the first failing function. 4. **GotoRecoveryErrors.p (1/4)** — VisitGotoStmt rvalue-pre-visit recovery in both the missing-state branch and the arity-mismatch branch. Validates that argument errors surface even when the goto itself is malformed. ## Deferred (lower-priority audit findings) - 6 more coverage gaps (CastExpr, ChooseExpr, FormatString, dup tuple field, RaiseStmt non-void, etc.). Less critical because each is covered by Phase1DormancyTest's weak `>=` baseline. - Harness #5/#7 (scratch-dir GetHashCode collisions, parallel dotnet build). The Ubuntu flake from earlier may relate but hasn't recurred since the projectRoot fix. Tracked separately. - Harness #8 (test timeouts). Nice-to-have; current CI runs reliably. - Identity-aware assertions (stderr substring matches) — would lock in which error fires first, but adds maintenance cost. Defer. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Round-2 multi-agent: typo fixes + source-order diagnostics + null-safety Five fixes from a second multi-agent audit covering 5 new dimensions (backwards-compat, PVerifier integration, error message quality, performance/scale, cross-PR consistency, PeasyAI integration). The audit produced ~30 findings; this commit applies the 5 quick-wins. The rest are tracked as deferred follow-ups (see PR description). ## 1. Error message typo: "is not undeclared" reverses meaning `DefaultTranslationErrorHandler.UndeclaredGlobalParam` previously emitted `'global param X' is not undeclared` — literally says the param IS declared, confusing the user reading the diagnostic. Fixed to `global param 'X' is undeclared` (also cleaned up the inner quoting). ## 2. Error message typo: "Value M has no start state" → "machine M..." `DefaultTranslationErrorHandler.MissingStartState` said `Value {machine.Name} has no start state`. The word "Value" is residual from a long-ago refactor; the entity is a Machine. Fixed. ## 3. Source-order diagnostic flushing Previously `FlushCollectedDiagnostics` iterated `Diagnostics` in insertion order (visitor traversal order). With multi-error collecting mode the user expects errors in source reading order so IDE jump-to-line flows naturally. Now sorts by `(file, line, column)` with a stable tiebreaker. Diagnostics without a parseable location prefix sort last. Cached compiled regex (`LocationPrefixPattern`) to avoid per-diagnostic regex compilation on the flush path. ## 4. SourceLocation.ToString() null-safety `SourceLocation.ToString()` previously did `return File == null ? throw new ArgumentException() : ...`. The throw fires partway through `FlushCollectedDiagnostics`'s loop if any diagnostic has File=null (reachable for `EmptyContext` per `DefaultLocationResolver.GetLocation`), partial-flushing the user's diagnostics with a confusing ArgumentException stack trace. Fixed: fall back to `<no source>:line:col` instead of throwing. ## 5. Cached compiled regexes in test harness CountErrorMarkers `Phase1DormancyTest.CountErrorMarkers` and the parallel helper in `MultiErrorAcceptanceTest` previously constructed two `new Regex(...)` per call. With ~280 test dirs × 2 modes × 2 fixtures, that's ~1000+ regex compilations per suite. Negligible in absolute terms (~100-200ms total) but mechanical to fix — moved to static fields with `RegexOptions.Compiled`. Also documented why the harness's own `[Test harness caught uncaught TranslationException:]` marker is substring-free of "Error" (so it deliberately doesn't inflate counts). ## Deferred (multi-agent findings tracked but not applied) - **PR #963 ↔ Phase 2 conflict resolution** — needs explicit merge-order decision (cherry-pick #963 into Phase 2 stack vs. land #963 first then rebase Phase 2's VisitTestExpr rework on top). - **Wrap pass 5 capability + pass 6 ControlFlowChecker in TolerantStep** — semantic change; deserves its own PR with new pinned tests. Could be a "Phase 4". - **PVerifier end-to-end CI smoke** — multi-agent flagged ZERO CI coverage of PVerifier-mode compile. PR #963 noted the existing `Tutorial/Advanced/4_Paxos/PVerifiedPaxos` already crashes on bare `p compile`. Needs separate fix + test addition. - **PeasyAI collecting-mode integration** — multi-agent produced concrete 5-file integration plan (compilation.py, fixer.py, generation.py, server.py, fixing.py). The parser already supports multiple errors; PeasyAI just needs to set `P_COMPILER_COLLECT_ERRORS=1` and call the existing `get_all_errors` instead of `parse_error`. Separate PR scope. - **Parallelize Phase1DormancyTest** — biggest CI speedup (~9min → ~2-3min) but needs scratch-dir race-condition validation. - **"N errors suppressed" footer** — UX improvement; new infrastructure. - **Truncation cap** — needed for pathological N>50 cases. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * Address Copilot round-2 review: InsertStmt location + doc accuracy Real bug + 3 doc accuracy improvements from Copilot's second-round review of PR #965 (1 code fix) and first-round review of PR #967 (2 doc updates): ## 1. VisitInsertStmt key-check pointed at the wrong AST node `StatementVisitor.VisitInsertStmt` reported the key/index TypeMismatch at `context.rvalue()` (the VALUE token of `insert lvalue[index] = value`) when the key actually lives in `context.expr()`. Diagnostics like "got type: string, expected: int" would point the user at the value expression instead of the index expression — confusingly off-target. The bug pre-existed the Phase 2 conversion (master had the same wrong context), but Copilot caught it because Phase 2 made the diff visible when converting `throw → CheckAssignable`. Fixed at line 215 to use context.expr() for the key check; context.rvalue() remains correct for the value check. ## 2. TolerantStep XML doc no longer claims cascade-suppression The previous doc said TolerantStep "runs an analysis step under cascade- suppression rules" — but it's used for both gathering AND analysis passes, and it doesn't implement cascade suppression at all (cascade suppression lives in CheckAssignable + the ErrorType IsSameTypeAs short-circuit; TolerantStep is the per-pass-item analog: same goal, different mechanism). Rewritten to: - Acknowledge dual-use across gathering and analysis passes - Distinguish from the expression-level cascade-suppression machinery - Make the strict vs collecting behavior contracts explicit ## 3. MultiMachineErrors test claim narrowed to what it actually tests The .p file header and the MultiErrorAcceptanceTest description both claimed the test validates "TolerantStep wrappers around pass 2a (MachineChecker) and pass 3". In reality, each machine in the file has a valid start state and payload type, so pass 2a never throws — the test only exercises pass 3 (function body type-checking) errors flowing through Phase 2's Report-and-continue plus Phase 3's per-function TolerantStep. Both descriptions updated to accurately call out: - What the test DOES exercise (per-machine isolation across pass 3) - What it DOESN'T cover (per-machine pass-2a throws like MissingStartState — would need a separate fixture) ## 4. PR description: precise strict-mode invariants Also updated PR #965's description to be precise about which strict-mode invariants the PR preserves vs. relaxes. The original claim "strict mode bit-for-bit unchanged" was over-broad — valid programs ARE bit-identical and invalid programs still exit 1, but the first-error IDENTITY can differ for invalid programs with both an argument error AND a callee/format/interface error (because args are now visited BEFORE lookup, which is the compiler-convention order and what lets collecting mode surface child errors). Replied on the 4 strict-mode reorder Copilot threads with the same rationale. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
…omments (#968) Final multi-agent audit on master found that the multi-error compilation feature (PRs #957, #963, #965) shipped without any user-facing docs and with stale "Phase 1 wiring only" / "Phase 2 will..." comments in the public XML API. Users can't discover the env var; contributors have no convention to follow when adding new throw sites. ## User-facing docs - **README.md** — new "Multi-Error Compilation" section in "What's New" with a worked example showing strict vs collecting output and pointing at the AI-fix-loop use case. - **ChangeList.md** — entry under "P 3.0 Changes" listing the env var, cascade-suppression semantics, the #963 state-lookup narrowing, and explicitly noting strict-mode bit-for-bit preservation. ## Contributor docs - **CLAUDE.md** — new "Multi-Error Type Checking (Compiler)" top-level section. Covers architecture (IDiagnosticCollector, ErrorType/ErrorExpr, CheckAssignable, TolerantStep), the per-visitor convention checklist (visit children first → short-circuit on ErrorType → Report+ErrorExpr), the analyzer pass-tolerance pattern, and the test-fixture layout. Mentions the env var in the "Working with P Programs" command examples. ## MCP tool descriptions - **peasy-ai-compile** — append a Tip about setting P_COMPILER_COLLECT_ERRORS=1 to receive all errors in one response. - **peasy-ai-fix-all** — append the same Tip with a note that batched errors converge in N/k iterations instead of N. (The PeasyAI subprocess invocations themselves still need to actually PROPAGATE the env var — separate PR per the audit's recommendation.) ## Stale XML-doc refresh - **IDiagnosticCollector** — strike "Phase 1 wiring only — no visitor currently calls Report" (Phase 2 lit it up). Add a Lifecycle list pointing at ReadContinueOnErrorEnvVar, the visitors, TolerantStep, and FlushCollectedDiagnostics. Cross-reference the three test files that exercise the contract. - **ErrorType** — strike the "Phase 2 will additionally add a CheckAssignable helper" forward reference (it landed). Promote the helper to a peer of the IsAssignableFrom override + IsSameTypeAs short-circuit so the three-piece cascade-suppression story is told once and accurately. - **ErrorExpr** — strike "Phase 1 introduces this class; no visitor produces it yet." Rephrase the IExprTerm-leak protection clause to cite Compiler.Compile (not the generic <see cref="Compiler"/>). Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Summary
Phase 1 of a 3-phase change to convert the P type checker from throw-on-first-error to collect-all-errors-and-report-together. This PR adds scaffolding only — no observable behavior change. Strict mode (throw on first) is preserved as the default everywhere; the new code path is dormant until phase 2 starts using it.
Why
Today every type-check failure does `throw handler.X(...)` and unwinds to the top-level catch in `Compiler.cs`. A user with N independent errors needs N compile/fix cycles. PeasyAI's `peasy-ai-fix-all` loop does N LLM round trips for the same reason — the biggest concrete win from this work is collapsing those round trips.
What's in phase 1
Design notes
What's NOT in this PR (deferred)
Test plan
🤖 Generated with Claude Code
Deliberate breaking changes to public interfaces
Copilot flagged that adding
DiagnosticstoITranslationErrorHandlerandContinueOnError/DiagnosticstoICompilerConfigurationis a source-/binary-breaking change for any external implementer of those interfaces. After review:DefaultTranslationErrorHandler/CompilerConfiguration).dotnet toolCLI, not as a NuGet SDK. External consumers invokep compile/p checkrather than implementing these interfaces.handler.Diagnostics/config.Diagnosticswithout new parameters).If we ever start distributing the compiler as a NuGet package, the retrofit is straightforward — add
ITranslationErrorHandlerWithDiagnostics : ITranslationErrorHandler(and the matching config variant), and cast at the small number of call sites that need the collector. Flagging the deliberate choice here rather than introducing two parallel interface hierarchies preemptively.