You can describe the job in one sentence, say who does it today and how often, and say what a good outcome looks like. Blocker: nobody can say what “working” would mean.
- The task and the team that does it today are named
- You know roughly how often it happens each week
- A good outcome is written down, with a real example
- Someone has said what one mistake would cost
- There is a reason to start now, beyond wanting to use AI
The answers or inputs exist in a form a system can read, and someone can say which version is current. Blocker: the knowledge lives only in people’s heads.
- The documents or records exist and are current
- One copy is agreed as the source of truth
- Access can be granted through a scoped account, not a shared login
- Personal or sensitive data in scope has been identified
- Someone owns keeping the content up to date
The steps are stable enough to describe, including the exceptions. Blocker: the process changes every month or differs from person to person.
- The steps fit on one page as a procedure
- The common exceptions are known and listed
- There is a clear point where a person takes over
- You can describe how the work is checked today
- The people doing the work agree with the description
The systems involved can be reached through an API or a supported export, and you know which ones the AI may change. Blocker: the key system has neither.
- Each system involved is listed with its owner
- Read access and write access are considered separately
- An API or supported export exists for each system
- A test environment exists, or test data can be made
- Model and hosting accounts can be opened in your company’s name
A named owner has time every week, and the people affected know what is coming. Blocker: nobody can review outputs during the first weeks.
- A named owner can give a few hours each week early on
- Reviewers for the first outputs are named
- Legal or compliance has seen the idea if regulated data is involved
- Staff whose work changes have been told how
- Everyone knows how the system will be switched off if needed
What scoping is for
Scoping turns an idea into a decision you can hold someone to: one job, one success measure, one owner, a known list of systems, and a price and timeline that follow from those choices. It is also the cheapest moment to drop an idea. Google’s Rules of Machine Learning open with two rules that apply just as well to language models: do not be afraid to launch without machine learning, and design your metrics first. If a form, a rule or a better process would do the job, scoping should say so.
NIST’s AI Risk Management Framework makes the same point from the risk side. Its Map function comes first: understand the context, the intended purpose and the people affected before anything is built or measured.
Terms used in this guide
- Job: the task the system will do, described the way the people who do it today would describe it.
- Baseline: how the job performs today, in volume, time or errors, measured before the build.
- Write action: anything the system changes outside itself, such as a record update, a message sent or a payment.
- Human checkpoint: the point where a person reviews, approves or takes over.
- Pilot: a limited release to real users or real work, with a decision date for keeping, changing or stopping it.
Run the readiness checklist
The checklist at the top of this page covers five areas. Go through it with the people who do the work, not only with their managers, and count “not sure” as a no. Each area names one blocker. If any blocker applies, fix it before scoping a build, or scope a short discovery step whose only output is the missing piece: a written process, an agreed source of truth, an owner with time, or access to the system that matters. Gaps that are not blockers become line items in the scope, because someone will have to close them during the build.
The scope variables that move price and timeline
In the projects we scope, the choices below move effort far more than the choice of model. Each row is a dial. Turning one to the right is often justified, but it should be a decision, not something that drifts in during the build.
| Variable | Lighter scope | Heavier scope | Why it moves effort |
|---|---|---|---|
| Systems connected | One, read-only | Several, with writes | Every integration needs access, error handling, test data and someone on the other side who knows the system |
| Actions | Answers or drafts only | Updates records, moves money or contacts customers | Writes need validation, duplicate protection, approval gates and an audit trail |
| Content sources | One current, well-structured source | Many sources, conflicting versions, scanned files | Preparing and versioning content becomes a workstream of its own |
| Channels | One: the website or an inbox | Web, WhatsApp, phone and email | Each channel brings its own formats, limits, consent rules and test cases |
| Languages | One | Several, with local policies | Each language needs its own evaluation cases and a reviewer who speaks it |
| Human review | A sample reviewed after the fact | Approval before every action | Review screens, queues, notifications and staffing become part of the build |
| Data sensitivity | Public or internal content | Health, financial or children’s data | Data handling, retention, vendor terms and legal review add work and lead time |
| Speed and volume | Replies can take a moment | Real-time voice at peak volume | Latency budgets and load testing shape the architecture |
| Evaluation | A short set of cases reviewed by hand | A versioned set with a release gate | Writing and grading cases takes time, and pays back on every later change |
| Handover | Runs in your accounts with a run-book | Your own team will extend it | Documentation, code standards and training for another team to build on |
To bring a quote down, move rows to the left rather than lowering the quality bar. A first release that reads from one system and drafts replies for a person to send is cheap to build, quick to test and easy to switch off, and it produces the evidence for the next row to move. Our own published prices and engagement models are on the pricing and engagement pages; any vendor should be able to tell you which rows drive their estimate.
Choose the first project with a rubric
Most teams have more candidate ideas than they can build. Score each one from one to three on the six criteria below, with the same people who ran the checklist. The use-case library on this site has a quick four-question selector for the tasks we build most often; this rubric is for your own list, and adds the questions that decide whether a first project can be judged fairly.
| Criterion | Scores three when | Scores one when |
|---|---|---|
| Measurable today | You can count the current volume, time or error rate | Nobody tracks the work, so nobody can say if it improved |
| Owner with time | A named person will review outputs every week | The owner is “the team” or a busy executive |
| Contained mistakes | Errors are internal and easy to reverse | Errors reach customers, money or a regulator |
| Inputs ready | Current documents, or a system with an API | Knowledge in people’s heads, or on paper |
| Few systems | One system, read-only at first | Several systems, with writes from day one |
| Visible value | Saves time someone will notice, or answers customers sooner | Value depends on a long chain of other changes |
Two rules sit on top of the totals. A candidate that scores one on owner or on contained mistakes is not a first project, whatever its total: without an owner nobody can judge it, and with uncontained mistakes the first errors are expensive. And when two candidates tie, choose the one whose mistakes are easier to undo.
Worked example: three candidates scored
An illustration with Sample data, not a client engagement. An online training company lists three ideas.
| Candidate | Measurable | Owner | Contained | Inputs | Systems | Value | Total |
|---|---|---|---|---|---|---|---|
| Help-centre answer assistant | 3 | 3 | 2 | 3 | 3 | 3 | 17 |
| Invoice data into the accounting system | 3 | 2 | 2 | 2 | 2 | 3 | 14 |
| Outbound sales calls by a voice agent | 2 | 2 | 1 | 2 | 1 | 2 | 10 |
- The help-centre assistant goes first. Its scope stays on the left of the variables table: one source, read-only, the website only, one language, sampled review and a short evaluation set.
- Invoice extraction goes second, once the accounting system’s test environment is available and the finance lead can review a weekly sample.
- Outbound voice calls are not a first project. Mistakes reach prospects directly, the rules on automated calls and consent need checking per market, and two systems would need writes from day one.
Write a one-page scope brief
The brief is what you send to whoever builds the project, whether that is your own team, us or another vendor. If a line cannot be filled in, that is the next thing to resolve, not something to leave to the build.
# Scope brief: <project name>
Job: <one sentence, in the words of the people who do it today>
Users: <who uses the output, inside or outside the company>
Baseline today: <volume, time or error rate, and how it was measured>
Success measure: <what changes, measured how, decided by when>
In scope: <channels, languages, content sources, actions>
Out of scope: <what this release will not do, even if asked>
Systems: <system> read | write, owner <name> (one line each)
Data sensitivity: <public / internal / personal / health or financial>
Human checkpoint: <who reviews or approves what, and when>
Evaluation: <how many cases, written by whom, reviewed by whom>
Owner: <name, hours per week for the first month>
Switch-off: <how the system is turned off, and who can do it>
Decision date: <when you will decide to keep, change or stop it>Questions to ask whoever builds it
- Which rows of the scope-variables table drive most of your estimate?
- What will you need from our team each week, and from whom?
- What is the first thing real users or real work will see, and when?
- How will we know it works: which cases, which metric, which baseline?
- Whose accounts will the models, hosting and data sit in?
- What is explicitly out of scope, and what would adding it later involve?
- How do we switch it off, and what happens to the work in flight?
When not to
Some ideas should leave the scoping stage without an AI build:
- A form, a fixed rule or a standard automation does the job. OpenAI’s guide to building agents makes the same test: if the work is not complex judgement, brittle rules or unstructured input, a deterministic solution may suffice.
- The process itself is broken or disputed. Automating it makes the disagreement faster, not smaller.
- Nobody can own it. A system without a reviewer drifts until someone notices a mistake in public.
- Every output must be right and there is no human checkpoint. Change the design so a person approves the consequential step, or do not build it.
- The goal is “use AI” rather than a job. Start from the checklist’s problem tab and come back with a task.
Limitations
- Rubric scores are judgement calls. Their value is in making a team argue about the same criteria, not in the totals.
- The checklist is general. Regulated sectors such as healthcare and finance need additional review of data handling, vendors and records.
- Scope variables interact. Adding a channel to a system that writes records multiplies the testing, rather than adding to it.
- A successful pilot shows the system works at pilot volume, with pilot users. Plan a second measurement after the wider rollout.
- Model and hosting costs depend on vendor list prices that change. Ask for running costs as a formula you can re-check, not a single number.
Sources
- Rules of Machine Learning: Best Practices for ML Engineering (Martin Zinkevich) (opens in a new tab) · Google for Developers · accessed
- A practical guide to building agents (opens in a new tab) · OpenAI · accessed
- Building effective agents (opens in a new tab) · Anthropic · accessed
- AI Risk Management Framework (opens in a new tab) · NIST · accessed