DX Audit app / notion
demo
positive notion:structured-error-responses

Validation, not-found, and authentication errors were structured and actionable

The requests that genuinely violated the API contract produced clear structured responses: invalid UUID format returned a 400 validation_error with an exact field path; malformed block content returned a 400 validation_error naming the property violation; an unshared valid-format page ID returned a 404 object_not_found with sharing guidance; and an invalid token returned a 401 unauthorized message. Both models interpreted these responses without human intervention. The select-option probe is not counted as a missing fourth error. Setting a previously absent select name succeeded and extended the schema, behavior that current first-party documentation defines for integrations with write access. Sonnet's first unshared-page probe also used an invalid UUID before it exercised the intended not-found case.

Evidence · transcript moments

Opus Audit run

Show all 3

Sonnet Audit run

Show all 5