When you cannot use a large model

We tested one of the lightest models on the market in agentic work, comparing Liquid AI's LFM2.5 against Google's Gemma 4 in a controlled test frame. Part of the result was predictable: the larger model produces good results more reliably. The other part was not. The small model did better than we expected, with 2.6 billion parameters against Gemma 4's 25.2 billion.

The limits of running a model on your own machine

Language models are large, often tens of billions of floating-point numbers and sometimes more than a trillion. All of those numbers have to be loaded into memory before the model can do anything at all.

We have all grown used to ChatGPT, where the hardware requirements are OpenAI's problem rather than ours. Their machines take in whatever the user supplies, whether images, files or source code, prepare it for the model, and deliver the answers back to the end user.

Recently the market has filled with mid-sized models that do not require tens of thousands of euros of compute, or a rented server farm. Running a model on your own workstation or laptop used to be out of the question, and that is changing. Gemma 4 has 25.2 billion parameters, of which only 3.8 billion are active at any one time, and quantised its weights take up 26 gigabytes on disk.

A little over a week ago Liquid AI released LFM2.5-2.6B, which runs on practically any modern device, including a phone. It knows little about the world and supports a limited set of languages, but it is optimised for agentic workflows, and in those it performs better than its size and memory requirements would suggest. Those are the two models we put against each other.

The memory gap is smaller than the parameter counts imply. In parameters the ratio is about ten to one; in memory while running, only about four to one, because context and compute buffers do not shrink along with the model. LFM2.5's 2.7 gigabytes of weights need around nine gigabytes of memory in service, and Gemma 4's 26 gigabytes of weights need around 35. The small model is still cheap enough to keep loaded that leaving it resident is barely a decision.

What a 2.6 billion parameter model managed

We ran a carefully controlled comparison on a task where the agents had to find information on the web. That normally takes a great deal of tool use: the agent runs searches, reads pages, takes notes, runs further searches and so on, without stopping to check in during the work. This repeating cycle of searching, reading and searching again is called a tool loop. Runs of this kind can take hours, and until recently agents built on smaller models did not cope with them well.

We ran 29 questions, each on both models and twice over, making 116 runs in total against real production infrastructure. Some of the questions were chosen because the web holds almost no answer to them. Those were traps: we wanted to see whether a model would invent a source when none exists.

	LFM2.5-2.6B	Gemma 4 (26B)
Parameters	2.6bn	25.2bn (3.8bn active)
Weights on disk	2.7 GB	26 GB
Memory needed in service	approx. 9 GB	approx. 35 GB
Generation speed	approx. 71 t/s	approx. 41 t/s
Fabricated source URLs	0	0
Dead links	3%	1%
Citations per report	2.5	3.3
Citations that survived checking	51%	33%
Reports carrying a working source	66%	81%
Mean time per task	104 s	107 s

Neither model took the bait. Across 116 runs the small model invented no source URLs at all, not one domain that does not exist. Hallucination, the failure organisations most often worry about, did not occur here.

The basic mechanics held up as well. LFM2.5 had been running the same task in production for about two and a half days before the test proper, and processed 1.46 million tokens during that time. Not one tool call went wrong, and no response came back malformed.

That matters more than it sounds. The agent has to format every tool call exactly right, or the whole chain breaks in the middle of a task. This has been the traditional stumbling block for small models, and it now appears to have been solved.

Why the faster model was not faster

The obvious advantage of a small model is speed, and it is measurable: LFM2.5 generates text at around 71 tokens per second against Gemma 4's 41. We expected that difference to show in the result. It did not: mean time per task was 104 seconds for the small model and 107 seconds for the large one.

The reason lies in where the time actually goes in agentic work. When the agent runs a search, it waits for the search engine. When it reads a page, it waits for the page to load. Dozens of these waits accumulate over a single task, each of them seconds long. The model's own text generation is a small share of the total. The network sets the pace, not the model.

There is a direct practical consequence. If you are considering an agentic workflow on more modest hardware, a slower model does not necessarily mean a slower result. We would suggest trying it and measuring both speed and quality in your own use case, before making decisions about infrastructure requirements.

What the small model stopped doing

So the small model did not invent sources. It did something else, and it is worth understanding precisely, because this kind of failure is considerably harder to spot.

Late in long tool loops, on the hardest questions, LFM2.5 stopped citing its sources. Twenty of its 58 reports contained no source URL at all. The reports were well structured, fluently written and confident. They simply could not be checked.

The difference from hallucination matters. An invented source is caught immediately, because the address does not exist. A report with no sources passes any fact-check at all, because there is nothing to check.

The most interesting detail is this: when LFM2.5 did cite a source, the citation survived checking more often than with the model ten times its size, 51% of the time against 33%. The ability to ground an answer in a real source is present at this scale. What erodes over a long task is the discipline to do it every time.

For design purposes, that is the useful observation. Discipline is not necessarily worth fixing with a larger model, but with the structure of the workflow: more precise instructions, and an automatic check that rejects an uncited report and sends the agent back to do the work again with tighter instructions than before.

Our conclusions

On the strength of this test, we would not give LFM2.5 unsupervised research work as it stands. We defined the test setup in advance so that the result would be trustworthy.

The processing power and memory of an ordinary workstation are nevertheless no longer an obstacle to running your own model. LFM2.5 is not the only option either: Bonsai's 1-bit models, for instance, run on almost any modern device. The LFM2.5 we tested handled the basics of agentic work without a fault: tool calls went through cleanly, it invented no sources, and nothing had to be given up on speed. If regulation, contracts or data protection require your data to stay under your own control, the starting point is considerably better than it was a few years ago.

There is one thing we did not measure at all: energy consumption. LFM2.5 does the same work with a fraction of Gemma 4's computation, so it very likely wins the emissions comparison by a clear margin. How clear a margin is for the next test to establish.

Language models differ from one another, and the choice commits an organisation for longer than many expect. A workflow is always tuned around the model in use: the instructions, the way tools get called, and the quirks the team has learned to work around. When the model changes, the quality of the work changes with it, or at least the way that work comes about. That is why sufficiency is worth measuring in the task the model is actually going into, rather than from the figures the vendor publishes.

At Trail Openers we help organisations choose an AI architecture that fits both their compliance requirements and the hardware they have. Want to know whether a model you can genuinely run is good enough for your use case? Get in touch and we will work it out together.