DX Audit app / github
demo
positive github:error-responses-consistent

The three requests that produced errors returned consistent structured envelopes

The duplicate branch (422), out-of-scope write (403), pull request with a nonexistent head branch (422), and invalid-token (401) probes returned JSON objects with a message, documentation_url, and status. Validation errors also included an errors array identifying the resource, field, and code. Both models analysed the responses without human intervention. The requested out-of-scope read was not an error scenario: GET /repos/torvalds/linux succeeded because the repository is public. Both agents then tested a write, which correctly returned 403. The pull-request validation response identified head as invalid but did not say that the branch was missing; both agents described that message as actionable but less specific than it could be.

Evidence · transcript moments

Opus Audit run

Show all 4

Sonnet Audit run

Show all 4