Navigate this article
A model that answers well still needs a system to work within. Learn how to design that foundation through an order lookup example.
Start with a task, not an agent
Consider a support team that needs to look up orders and prepare customer replies. The aim is to shorten the lookup without letting a conversation change payments or expose another account's data.
This is a teaching example. Before implementing a similar workflow, understand the process, identify who may access each field, and prepare a test environment with fictional records.
What the harness does
A harness is the execution system around a model: it provides context and tools, enforces permissions, and records what happened. The model may propose an action; the system decides whether it is valid and authorized. A prompt saying ‘do not change payments’ cannot replace a backend restriction.
Harness Engineering means designing, testing, and maintaining that system. It can support software developers as well as customer service, sales, finance, and operations. The process changes; the need for boundaries and traceability remains.
- Context: instructions, references, and task-relevant information.
- Tools: small operations with validated inputs and predictable outputs.
- Permissions: identity, scope, and isolation enforced outside the model.
- Execution: time, cost, retry budgets, and stopping conditions.
- Evaluation: behavioral tests, records, and human review.
Design the tool before writing the prompt
An order lookup tool is easier to constrain than a connection that can execute arbitrary SQL. It validates the authenticated identity and returns only required fields. The authorized company comes from the server session, never from a model-selected value.
The following contract describes a policy, not a ready-to-run SDK configuration. The tool service must enforce every rule.
tool: lookup_order
input: { order_id: string }
identity: authenticated_session
scope: session.company_id
permissions: [orders.read]
output: [status, estimated_delivery]
limits: { timeout_ms: 3000, max_attempts: 2 }
audit: [actor_id, tool, outcome, duration]Reading, preparing, and executing are different permissions
An authorized lookup can run automatically. A customer message can start as a draft. A refund needs a separate tool and an explicit decision from an authorized person. Approving one draft must not grant unlimited authority for later actions.
Bind approval to the exact action, parameters, and reviewer identity. A changed proposal needs new approval. Documents, web pages, and messages returned by tools are external data, not trusted instructions that can change these rules.
- Support: find an order and suggest a reply without changing the purchase.
- Sales: prepare a proposal without granting an unauthorized discount.
- Finance: compare records without permission to issue payments.
- Operations: classify a request and route it to its owner.
Test the cases that should be rejected
Include missing orders, unauthorized users, another company's order, unavailable tools, and a message trying to change the rules. Check the user-facing answer and actual system effects. A polite reply does not make an unauthorized lookup acceptable.
Compare the assisted workflow with the current process on the same cases. Track resolution time, human corrections, tool failures, and cost per completed task. Define acceptance criteria before evaluation; one successful demo is not evidence of consistent quality.
Logs can expose data too. Record only what is needed, remove secrets, and define access, retention, and deletion.
Build a foundation other teams can use
A shared foundation can provide authentication, approved tools, evaluation templates, and version records. Each team describes its process, proposes a change, and tests within those limits. Engineering maintains the foundation; the people running the process help define quality.
Start with a small read-only or draft task. Let the team review results, refine the contract, and increase autonomy only when evidence supports it. Multiple agents are not a requirement: a simple execution can solve the task with fewer failure points.
Sources and further reading
Educational material. Validate examples in a test environment and adapt decisions to your project's context, risks, and responsibilities.
Share