A study of 128 knowledge workers found that an internal AI tool sped up three common tasks. It also made the research worse.
Employees at a multinational industrial company were assigned to work with or without a tool that used GPT-4o mini and retrieval from the company’s database. They had to find facts in internal documents, compress a product description into four sentences, and propose new products and research priorities.
With AI, they finished the research task 29 percent faster, the summary 52 percent faster, and the idea-generation task 39 percent faster. The summaries were more faithful, representative, concise and conventionally written. The ideas were more complete, though no more novel.
AI also changed the spread of performance. It narrowed quality gaps on summaries and ideas by improving weaker outputs. But they found a wider gap on research. About a quarter of the AI-assisted responses to the quantitative research question misreported or invented numbers, even though the correct figures were in the database. This means that the same colleague may need close review on one step and none on the next, which is hard to observe and mitigate.
The authors call this “task-technology fit.” More plainly: language models are good at turning source material into smooth prose. They are unreliable at exact retrieval, where a single wrong number can invalidate the answer; participants appear to have accepted the AI’s research rather than spending time checking it.
Research becomes a summary, recommendation, slide or plan. A fabricated number can pass through each stage while the language around it becomes clearer and more persuasive. The experiment cannot tell us how often that happens because the three tasks were evaluated independently. The authors call for studies of continuous workflows in which early errors create rework downstream.
False claims also gain credibility through repetition. The illusory truth effect makes repeated claims feel more credible even when people know better. A number repeated in an AI answer, summary, deck and meeting notes can harden into organizational fact before anyone checks it.
For what it's worth, when I need an exact answer from a body of data, I ask a model to write a script that retrieves and checks the answer, and then refine the script. This is what I meant in When It Starts Feeling Like a Video Game: software is its easy mode. A script can be wrong, but it leaves an inspectable method that can be tested, rerun and corrected.
Scale AI says nearly half of its new data-training projects now involve reinforcement-learning environments. These simulated workplaces contain incomplete records, multiple tools and recoverable errors. An agent starts from a known system state. Its actions and intermediate results are recorded, and verifiers check what changed in the system, which is more or less exactly the same idea as described above, played out at a much larger scale.
Training this way forces an explicit description of how the organization operates: starting conditions, permissions, handoffs, exceptions, decision rights and definitions of done. It resembles Company as Code applied to production. A network of teams needs interfaces an agent can traverse. A logbook supplies decisions and context that system state cannot explain.
Legibility has a cost. Shannon Mattern’s A City Is Not a Computer, which I wrote about here, warns against confusing the map for the territory. Software records clicks, approvals and changed fields. It misses the unofficial expert, the short conversation that prevents a foolish decision. Clean reward signals make work easier to automate and easier to overvalue.
What then, instead? Measure the full workflow: Time to a verified outcome; Errors caught before reuse; Rework created downstream.
