Ask five operations leads where AI should go in their business and you will get five answers, most of them a chatbot. Ask which tasks in their week are tedious, repetitive and involve reading messy text, and the answers get much more useful. Large language models are good at that second category. They are much weaker at the first.
Our view, after building workflow automation around them and watching where it holds up in practice, is that language models earn their keep in a narrow band of operational work. Outside that band they cost more than they return.
The shape of a good fit
A language model is, at its core, very good at turning unstructured language into structured output and back. That is the useful part. Operations is full of unstructured language: emails, invoices as PDFs, support messages, free text notes, supplier replies, policy documents.
The best fits share a pattern:
- The input is text or a document with no fixed format.
- A person currently reads it and types the result somewhere.
- A wrong answer is catchable, by a person or by a rule, before it does harm.
- The volume is high enough that the reading time adds up.
When all four hold, a model can take over the reading and leave the deciding to humans and rules.
Where it works well
Extraction. Pull the supplier name, invoice number, amount, currency and due date from a document that arrives in twenty different layouts. Traditional templates break when a supplier changes their format. A model reads the page the way a clerk would. You still validate the result, for example by checking that line items add to the total, but the tedious part is gone.
Classification and routing. Read an incoming message and decide whether it is a billing question, a complaint, a request to change details or spam, then send it to the right queue. The model does not resolve anything. It sorts, which is where time is usually lost.
Drafting. Write a first reply to a routine query, using your own policies and the customer's record, for a person to review and send. The human edit is the control. This tends to save time even when the draft needs changes, because starting from a draft is faster than starting blank.
Summarising. Condense a long ticket history or a thread of forty emails so the next person can act without reading everything. Useful, with the caveat that summaries can omit a crucial detail, so keep the source one click away.
Search over your own documents. Answer questions from your policies, contracts and procedures, with references to the passage used. This works when the answer exists in your material and the system is built to cite it.
Where it disappoints
Arithmetic and exact lookups. Models can produce a confident wrong sum. If a number matters, compute it with code and use the model only to read the inputs.
Decisions with consequences and no check. Approving a payment, changing a bank account, or granting access should not be a model's call alone. Use it to prepare the case, and let a rule or person decide.
Anything needing guaranteed consistency. The same input can produce slightly different outputs. If you need identical results every time, or an audit trail of exact logic, plain code is the right tool.
Poorly defined processes. If your team cannot explain how a task is done, a model will not fix that. It will automate the confusion at speed.
Design for being wrong
A model will occasionally be wrong with complete confidence. That is not a flaw to wait out, it is a property to design around. Practical habits that help:
- Ask for structured output and validate it against a schema. Reject anything malformed.
- Add checks that do not depend on the model, such as totals matching or a supplier existing in your records.
- Route low confidence or failed checks to a person, with the original document beside the model's reading.
- Log inputs, outputs and the human's final answer. That record is how you measure accuracy honestly.
Nobody should have to trust the model blindly, and nobody should have to recheck everything. The goal is not zero errors. It is that errors are caught cheaply, before they reach a customer or a ledger.
Measure before you scale
Start with a small, real sample. Take a few hundred past cases where you already know the correct result, run them through, and count how often the output is right. Do this before you build anything large. Then keep measuring in production, because documents, suppliers and customer behaviour change.
Be honest about the comparison too. The baseline is not perfection, it is the current process, which includes tired humans making typos. Sometimes a model with review beats the status quo comfortably. Sometimes it does not, and the saving is smaller than the pilot suggested.
Data and privacy
Operational text often contains personal and financial data. Before sending it to any model provider, know where it goes, whether it is retained or used for training, and what your obligations are under the Nigeria Data Protection Act 2023, GDPR or your sector's rules. Minimise what you send. Strip identifiers that the task does not need. Treat this as a design constraint from the first day, not a review at the end.
A sensible first project
Pick one high volume, text heavy, low risk task. Inbound document handling or support triage are common candidates. Build it so a person reviews every output at first, measure accuracy, and relax review only where the numbers justify it. If it works, you have a template. If it does not, you have spent weeks, not quarters, finding out.
Language models are not a strategy. They are a capable tool for reading and writing, and the operations that benefit are the ones that know exactly which reading and writing they would like to stop doing by hand.