Treat model output as data.
Build a synthetic-data workflow where an AI assistant recommends a query but cannot execute arbitrary SQL. Test the boundary and document the patch.
Put a hard boundary between words and actions.
Use a local sample database of invented records. Let the model select from predefined report types, then let server-side code validate parameters and run a fixed query. Never put a raw model-generated SQL string on a privileged database connection.
Allowed flow
User request → model emits a bounded JSON intent → schema validation → authorized report function → read-only database query → redacted response.
Denied flow
Model text → unrestricted SQL or shell execution. The exercise must demonstrate that a manipulated response cannot cross this boundary.
A controlled data workflow.
Seed
Create a tiny synthetic data set and a read-only database role.
Constrain
Define a JSON schema with an allowlisted report name and bounded parameters.
Execute
Map validated intent to parameterized, fixed queries in server-side code.
Test
Try malformed JSON, extra fields, out-of-range parameters, and hostile strings; log only safe metadata.
Show the control working.
- Data model contains synthetic records only
- Read-only database permission evidence
- Intent schema and fixed-query mapping
- Negative test matrix with expected denials
- Parameterized query code review
- Short threat model and residual risks
Show the denied path as clearly as the happy path.
Present what you built, what failed, how you verified it, and what you would improve.