Large companies answer this with procurement committees. Small teams need something lighter: a process that takes a day, not a quarter, and that people actually follow because it is easier than going around it.

Start with a simple principle

The goal is not to block tools. It is to make sure every tool the team relies on has a known owner, a known cost, and a known answer to the question "what data goes into it?" If a process delivers those three things, it is doing its job.

Sort requests into three tiers

Not every tool deserves the same scrutiny. A quick tiering step keeps the process fast for low-risk cases.

Tier 1: personal experiments. Free or trial use with no company or customer data, such as testing a writing assistant on public text. These need no approval, only a note in a shared list so others know it is being tried.

Tier 2: team use with internal data. The tool will touch internal documents, code, or plans, or it will be paid for by the company. These need a short review using the checklist below.

Tier 3: customer or sensitive data. Anything that processes customer information, personal data, financial records, or credentials. These need the full checklist plus sign-off from whoever is responsible for security or legal matters, even if that is the founder.

The one-page checklist

For Tier 2 and Tier 3 requests, the person proposing the tool fills in a short form. Keep it to questions people can answer in fifteen minutes:

1. What problem does it solve? One or two sentences, tied to real work.
2. What do we already have? Is there an existing tool that does most of this? Overlap is the most common source of wasted spend.
3. What data will go in? Be specific: public text, internal docs, source code, customer records.
4. What do the vendor's terms say about that data? Check whether inputs are used for model training, whether that can be turned off, how long data is retained, and where it is stored. Link the relevant policy page.
5. How do people sign in? Prefer company accounts and single sign-on where available, so access can be removed when someone leaves.
6. What does it cost at realistic usage? Look at the plan you would actually need, including seat counts and usage limits, not the entry price.
7. Who owns it? One named person responsible for the account, renewals, and offboarding.
8. How will we know it is working? A simple check after 30 or 60 days: is it still used, and is it saving time?

Before filling in the form, it helps to see what alternatives exist. Browsing tools by category on Toolbit AI gives the requester a quick sense of comparable options, so the "what else did we consider?" question has a real answer.

Decide quickly and record the outcome

Agree on a response time, such as two working days. The reviewer can approve, approve with conditions (for example, "no customer data" or "company account only"), or decline with a reason. Speed matters more than perfection here. If approvals take weeks, people will stop asking.

Record every decision in a single shared register, a spreadsheet is enough, with these columns:

• Tool name and link
• Tier
• Owner
• Approved data types
• Plan and monthly cost
• Renewal date
• Review date

This register becomes your source of truth for onboarding, offboarding, and budget conversations.

Review on a schedule

AI tools change often. Vendors add features, update terms, change pricing, and sometimes shift how they handle data. An approval given in spring may not fit by autumn.

Set a review rhythm that matches the tier: Tier 3 tools every quarter, Tier 2 tools twice a year. At each review, the owner confirms the tool is still used, the cost still makes sense, and the vendor's data terms have not changed in a way that matters. Renewal dates in the register are a natural trigger for this.

Plan for exits

Every approval should include a short answer to "how would we leave?" Can you export your data in a usable format? Are there contract terms that lock you in? Who removes access when someone leaves the team? Answering this up front makes it far easier to switch when a better option appears.

Make the process easy to follow

A few habits help the process stick:

• Keep the form short. If it takes longer than fifteen minutes, people will skip it.
• Publish the approved list. When people can see what is already approved, they request fewer duplicates.
• Say yes often. A process known for reasonable, fast approvals earns trust. One known for saying no gets bypassed.
• Explain declines. A clear reason, such as unclear data retention terms, teaches the team what to look for next time.

Bringing it together

A small team does not need a procurement department to adopt AI tools responsibly. It needs three tiers, a one-page checklist, a shared register, and a review date on the calendar. That is enough to keep data where it belongs, avoid paying twice for the same capability, and still move at the speed that makes small teams effective.

This is not a failure of discipline. It is what happens when tools are cheap to adopt, billed to different cards, and improving so quickly that last quarter's best choice may not be this quarter's. The fix is not a freeze on new tools. It is a short, repeatable audit, run every quarter, that keeps the stack lean without slowing anyone down.

This guide walks through that audit step by step and ends with a template you can copy into a spreadsheet today.

Why quarterly, and why not just "when the bill looks high"

Reactive audits only happen after money is already lost, and they tend to be rushed: someone cancels the most expensive line item, which is often the tool people actually rely on.

A quarterly rhythm works well for AI tools for three reasons:

• Products change fast. Features that justified a separate subscription often get folded into tools you already pay for. A quarter is long enough for that to happen and short enough to catch it.
• Annual renewals hide in the calendar. A fixed review date gives you a chance to see renewals coming instead of discovering them on a statement.
• Usage patterns settle. After a quarter, the novelty has worn off. You can see what people actually use, not what they were excited about in week one.

Put the audit on the calendar as a recurring 60–90 minute block with one named owner. Ownership matters more than the exact length.

Step 1: Build the inventory (the honest version)

You cannot trim what you cannot see. Start by listing every AI-related subscription, and look in more places than feel necessary:

• Company cards and expense reports (search for vendor names and words like "AI", "pro", "plus", "credits")
• Personal cards that get reimbursed
• App store subscriptions on work phones
• SSO or identity provider dashboards, which show which apps people sign in to
• API billing consoles, where usage-based charges live
• Browser extensions with paid tiers

For each tool, record the basics: name, owner, plan, cost per month (convert annual plans to a monthly figure), renewal date, number of seats paid, and how billing is done.

The first inventory usually takes the longest. Later quarters are mostly updates.

Step 2: Measure real usage, not intended usage

For each tool, answer one question: who used it in the last 30 days, and for what?

Where the vendor offers an admin dashboard, use its active-user numbers. Where it does not, a two-question message to the team works fine:

1. Which of these tools did you use in the last month?
2. If it disappeared tomorrow, what would you use instead?

The second question is the important one. If the answer is "the other tool we already pay for," you have found overlap. If the answer is "nothing, and I'd be stuck," you have found a tool that earns its place.

A simple way to read the result is seat utilisation: active users divided by paid seats. Hypothetical example: a team paying for 10 seats where 4 people logged in last month is at 40% utilisation. Whether that is a problem depends on the price and the plan's minimum seats, but it is worth a conversation.

Step 3: Map overlap by job, not by product name

Overlap is hard to see when you list tools by brand. It becomes obvious when you list them by the job they do.

Group every tool under the task it serves, for example:

• Drafting and editing text
• Meeting notes and transcription
• Image and video generation
• Coding assistance
• Research and summarisation
• Automation and integrations

Any job with two or more paid tools under it is an overlap candidate. That does not mean one must go. Sometimes two tools cover the same job for different people with different needs, such as one for long documents and one for quick chat. The point is to make the overlap a decision rather than an accident.

When you are weighing two tools for the same job, a side-by-side look at features and pricing helps. A comparison view like Toolbit's AI tool comparisons is a quick starting point, but confirm current limits on each vendor's own pricing page before you decide, since plans change often.

Step 4: Decide with four outcomes

For every tool, choose one of four outcomes. Keeping the options few makes the meeting fast.

• Keep: used regularly, no cheaper overlap, clear owner.
• Downgrade: valuable, but on a bigger plan or more seats than needed.
• Consolidate: the job is covered by another tool you are keeping. Plan a migration and a cancel date.
• Cancel: low usage and no one would be stuck without it.

Two rules prevent regret:

1. Export before you cancel. Many AI tools hold prompts, generated assets, transcripts or custom configurations. Find out what is stored and export it before the subscription lapses.
2. Check the contract terms. Annual plans may not refund mid-term, and some have notice periods. If cancelling now saves nothing, set a reminder before the next renewal instead.

Step 5: Check the risks a cost audit usually misses

Price is only one reason to trim a stack. While you have every tool in one list, take five minutes on these:

• Data handling: Do you know what each tool does with the content you put in? Is there a team or business plan with clearer terms?
• Access: Do former team members still have accounts? Is the tool behind SSO, or are logins shared?
• Single points of failure: Is an important workflow in a tool only one person knows how to use?
• Shadow tools: Did the usage survey reveal tools no one had listed? Add them to the inventory.

None of these needs a long review. Flag anything uncertain and give it an owner and a date.

Step 6: Plan for what's next

A good audit also looks ahead. Note any tool where a vendor announcement could change the decision next quarter, such as a model upgrade, a new tier, or a feature landing in a product you already pay for. Keeping an eye on a running feed of changes, like Toolbit's AI updates page, makes the next review faster because you already know what moved.

Finally, agree on a light rule for new tools between audits. For example: anyone can trial a tool, but a paid plan needs a named owner and is added to the inventory the same week. That one rule prevents most sprawl.

The quarterly AI stack audit template

Copy these columns into a spreadsheet, one row per tool:

And a checklist for the meeting itself:

• Inventory updated from cards, expenses, SSO, app stores and API consoles
• Usage checked for every tool (dashboard or two-question survey)
• Tools grouped by job; overlaps highlighted
• One decision recorded per tool
• Exports scheduled before any cancellation
• Renewal dates and notice periods checked
• Access cleaned up for former team members
• Data-handling questions flagged with an owner
• Watch list for next quarter written down
• Next audit on the calendar

A worked example (hypothetical)

To make this concrete, here is an illustrative scenario. The numbers are invented for the example.

A five-person team finds seven AI subscriptions. Two are writing assistants, and the survey shows everyone uses one of them daily while the other was used by one person for one project. Two are transcription tools, one bundled into the team's video-call plan and one paid separately. One image tool is on a seat plan for five, but only the designer uses it.

The decisions: keep the main writing assistant, cancel the second after exporting its saved prompts, consolidate transcription onto the bundled feature, and downgrade the image tool to a single seat. Nothing the team relies on goes away, and every change has an owner and a date.

The savings in your case will be different. What carries over is the process: list, measure, group by job, decide, export, and repeat.

Keep it boring

The best stack audit is a boring one. If you run it every quarter, most rows will say "Keep," a few will say "Downgrade," and the occasional overlap gets resolved before it turns into a year of double billing. That small habit leaves room in the budget for the next tool that is genuinely worth adding.

Toolbit AI Team

This guide is a test-first checklist for doing exactly that. It is vendor-neutral and deliberately low-tech: a spreadsheet, a handful of real examples, and an afternoon are enough. It works whether you are choosing a coding assistant, a meeting-notes tool, a document Q&A app, or a raw model API.

Start with the job, not the tool

Before opening a single free trial, write one sentence describing the job you want done and how you will know it was done well. For example:

• "Summarize a 45-minute customer call into decisions, owners and open questions, with no invented commitments."
• "Generate a first-pass migration script from our schema diff that passes our existing tests."
• "Answer support questions from our help center, and say 'I don't know' when the answer isn't there."

That sentence narrows the shortlist and becomes the yardstick you score against later.

Next, note your hard constraints. These are pass/fail, not scored:

• Data handling: Can this data leave your environment? Does the vendor's published policy say whether inputs are used for training, and can that be turned off on the plan you'd buy?
• Integration: Does it need to work inside a specific editor, chat app, CRM or API?
• Budget ceiling: The most you'd pay per seat or per month at expected usage.

Any tool that fails a hard constraint is out, however good its output is. Checking these first saves you from falling for a tool you can't actually adopt.

Build a small, honest test set

A test set is a collection of real inputs paired with what a good output looks like. Ten to twenty items is usually enough to separate tools that fit from tools that don't. More is better only if you'll actually review the results.

Make the set representative of your real work:

1. Typical cases (about half). Everyday inputs you'd hand the tool on a normal Tuesday.
2. Hard cases (about a third). Long inputs, messy formatting, domain jargon, ambiguous requests, mixed languages, the files that break things.
3. Trap cases (a few). Inputs where the correct behavior is to refuse, ask a clarifying question, or say the information isn't available. These are where confident fabrication shows up.

For each item, write down the expected result before you run anything. It doesn't need to be a perfect reference answer. A short list of "must include", "must not include" and "acceptable if" notes is enough. Writing expectations in advance protects you from a common bias: judging an impressive-sounding output as correct because it's fluent.

Use synthetic substitutes for sensitive data until you've verified a trial's data terms.

Run every tool under the same conditions

Fairness matters more than sophistication. A few rules keep the comparison meaningful:

• Same inputs, same instructions. Paste the identical prompt and files into each tool. If a tool needs a different format, note the adaptation.
• Default settings first. Evaluate what you'd get out of the box, then optionally a second pass with the tool's recommended configuration. Record which is which.
• Run more than once. Generative tools can return different outputs for the same input. Run each test item at least twice, preferably three times, and look at consistency, not just the best result.
• Fresh context. Start a new session or chat for each item so earlier tests don't leak into later ones, unless multi-turn memory is the thing you're testing.
• Record everything. Save raw outputs, time taken, and any errors. Screenshots or copied text in a shared folder are fine.

If you're comparing underlying models rather than finished apps, keep the model version fixed during the test and note it. Providers update and retire models regularly, so a result from last quarter may not describe what you'd get today. Keeping an eye on a release tracker such as Toolbit's AI model updates helps you know when a re-test is due.

Score with a rubric you wrote in advance

Score each output against your expectations, not against the other tools. A simple 0–2 scale per criterion is faster and more consistent than 1–10:

• 0: Wrong, missing, or unusable without a rewrite.
• 1: Usable with light edits.
• 2: Usable as-is.

Pick four to six criteria that match your job sentence. Typical ones:

• Correctness: Facts, code, numbers and claims are right.
• Faithfulness: Nothing is invented beyond the source material; trap cases are handled honestly.
• Completeness: Required elements are present.
• Format fit: Output drops into your workflow (correct structure, length, file type) without reformatting.
• Edit time: Roughly how long it took you to make the output shippable.
• Consistency: Repeated runs give comparably good results.

Weight the criteria by what failure costs you. For a support bot, faithfulness probably outweighs style. For a brainstorming tool, the reverse may be true. Agree on weights before you see results, or you'll unconsciously tune them toward the tool you already like.

A reusable scoring table template

Copy this into a spreadsheet, with one row per tool per test item. Average the runs, multiply by weight, and sum.

The example rows only show how to fill it in. They are not real results. Keep a separate summary tab with each tool's weighted score, its trap-case pass rate, and its median edit time. Those three numbers tell you more than any single headline score.

Once you have two or three finalists, putting their published features, pricing tiers and limits next to each other makes it easier to see the trade-offs your test didn't cover; a side-by-side view like Toolbit's compare page is one way to do that before you read each vendor's own pricing page to confirm.

Sample test prompts

Here are prompt patterns that work across many categories. Adapt the wording to your domain and keep it identical across tools.

Typical task

Here is the transcript of a customer call. Summarize it as:
1. Decisions made
2. Action items with owner and due date, only if stated
3. Open questions
Do not include anything that is not in the transcript.
[transcript]

Hard case: long, messy input

The attached file is an export with inconsistent column names and
some empty rows. List the columns that appear to contain the same
data under different names, and explain how you decided.
[file]

Trap case: information not present

Using only the attached policy document, what is the refund window
for annual plans? If the document does not say, reply exactly:
"Not stated in the document."
[document that does not mention annual plans]

Consistency check Run any of the above three times in fresh sessions and compare. Differences in wording are fine; differences in facts, numbers or decisions are not.

For coding tools, swap in tasks from your own repository: a real bug with a known fix, a small feature with existing tests, and a request that should be refused or questioned because it conflicts with the codebase's conventions. Judge by whether the tests pass and whether you'd merge the diff, not by how plausible the explanation sounds.

Check the things outputs don't show

A tool can score well and still be the wrong purchase. Before paying, confirm:

• Usage limits. Read the plan's published limits on messages, tokens, file sizes, seats or minutes. Estimate your monthly volume from the test and check you'd stay within the tier you're considering.
• Real cost at your volume. For usage-priced tools, multiply your measured usage per task by expected monthly tasks. Include the human edit time you recorded; a cheaper tool that needs more cleanup can cost more overall.
• Failure behavior. On timeouts or oversized files, clear errors beat silent truncation.
• Change risk. Does the vendor publish a changelog or deprecation notices? Tools built on third-party models can change behavior when the underlying model changes.

Make the decision and set a re-test date

Pick the tool with the best weighted score among those that passed every hard constraint, then sanity-check it against cost and the checks above. If the top two are within a point or two, prefer the one with the better trap-case behavior and lower edit time; those predict day-to-day trust better than peak quality does.

Start on the smallest paid tier or a monthly plan, not an annual commitment, and put a re-test on the calendar for a few months out. Keep your test set. It's now an asset: when a new tool launches or your current one updates its model, you can re-run the same items in an hour and know whether anything actually changed.

Conclusion

Evaluating an AI tool doesn't need a benchmark lab. It needs a clear job definition, a small set of real and deliberately difficult examples, expectations written before you look at outputs, and a rubric applied the same way to every candidate. Do that, and you replace "the demo looked great" with evidence you can defend to your team, and you'll spend your budget on the tool that fits your work rather than the one with the best launch video.

— Toolbit AI Team