The two biggest challenges of AI development: quality assurance and big-model dependency

In my previous article I argued that AI writes exactly the code you ask for. Today I'll follow up with what that implies — because if generating code stopped being the problem, something else had to become one.

I believe it's two things. And neither of them is "AI isn't smart enough".

Challenge #1: Everyone can generate. Few can verify.

Until a few years ago, writing code was expensive and slow. The whole industry was built on that scarcity: we paid for person-days of typing.

Today, typing is nearly free. AI generates a function, a module or an entire app in minutes. And that flipped the economics of development: the bottleneck is no longer producing code, but verifying it's correct. QA, long treated as an afterthought at the end of the process, is becoming the most valuable discipline in the whole chain.

It's only logical. When ten people generate as much code per day as a hundred people used to write in a month, the amount of things that need to be checked, tested and approved grows at the same rate. But verification can only be partially automated — tests, static analysis, type checks and machine review catch a lot, yet the final questions remain human:

• Does it actually do what the business needs — not just what the ticket said?
• Does it behave sensibly in edge cases nobody wrote down?
• Will it break something in six months that nobody can see today?

This changes the developer's role more than most of the industry admits. Value is shifting from "I can write it" to "I can specify it precisely and verify it reliably." That's a skill closer to a senior engineer, an architect and a tester than to a classic code typist. The human factor isn't disappearing from development — it's moving to the one place where it's irreplaceable: the end of the chain, the decision that "this is correct and may ship".

In our practice this means more energy goes into tests, reviews and verification runs than into the writing itself. It sounds like the opposite of how AI development gets marketed. It's exactly why our output works in production.

Challenge #2: A big model everywhere = a dependency you didn't need

The second challenge is architectural, and it gets surprisingly little airtime — because it goes against the interest of everyone selling tokens: where in the system should an LLM even be?

The prevailing pattern of 2026 is "stuff a GPT/Claude API call into everything". Every feature calls a cloud model: order categorization? API call. Search? API call. Notifications? API call. The result is an application that:

• carries the latency of an external service in every request,
• pays for tokens on every user click,
• ships company data to a third party even when it doesn't have to,
• and above all — stops working when the model vendor stops working. Or raises prices. Or changes behaviour between versions.

Meanwhile, an inconvenient truth holds for most of those tasks: they don't need billion-dollar intelligence. They need well-written deterministic code. And this is where the two challenges meet beautifully — because today, a large model will write that code for you.

The pipeline that makes sense to us

Our approach fits in one sentence: large cloud LLMs during development, deterministic code in production, small models only where language is genuinely needed.

In practice it looks like this:

1. Build time: large models at full throttle. Design, code generation, tests, code review, documentation, migrations. This is where frontier intelligence pays off — and the vendor dependency is safe at this stage: if the vendor disappeared tomorrow, your production app wouldn't even notice. Only further development would slow down.
2. Runtime: deterministic code as the default. Rule-based categorization, calculations, workflows, reports, integrations — all of it is cheaper, faster and more reliable as plain code. The old objection that such code was expensive to write? That's precisely what's no longer true.
3. Runtime with language: the smallest model that does the job. Extracting fields from documents, classifying texts, summaries, simple answers — small models handle these, often running locally on your own hardware. No tokens, no data leaving the company, predictable costs.
4. Runtime with a large model: only with a genuine reason. Open-ended customer conversation, complex reasoning over arbitrary input, agentic scenarios. These cases exist — but they're a minority, and they should be a conscious decision with a budget, eval suites and a fallback, not the default state.

An honest note on "local"

To keep this from being propaganda: "small" and "local" are two different axes. A small cloud model solves cost and latency, but data still leaves. A local model adds privacy and independence, but brings operational load — an inference server, updates, quality evaluation. At low volumes, a small cloud model can paradoxically be cheaper than your own GPU. What decides is volume, data sensitivity and whether you have the capacity to run infrastructure. There is no single right answer — there is a right order of questions.

The decision ladder

When we design a new feature, we ask in this order:

1. Can deterministic code do it? → Write it (with a large model's help, by all means). Done.
2. Is language understanding needed? → Can a small model handle it? Try it, measure it, ship the smallest model that passes evaluation.
3. Does it truly need a frontier model? → Fine — but with a token budget, an eval suite, a fallback for outages, and the awareness that you've just bought a dependency.

Most features end on the first rung. And that's fine — the goal isn't to have as much AI in the app as possible, but to solve the problem as reliably as possible.

Conclusion

The biggest challenges of the AI era aren't technological — they're matters of discipline. First: keeping quality up when code pours in at a rate QA processes were never built for — and accepting that human judgment is now the most expensive and most important link in the chain. Second: resisting the temptation to stuff a large model into every request, and building systems that use AI where it makes sense — above all during their creation — while standing on their own feet in production.

Both challenges share a common denominator: people decide, not models. Still.

At Reactive Studio we build apps this way for a monthly subscription — AI at full throttle during development, sober in production. If you're figuring out where AI belongs in your system and where it doesn't, book a consultation — it's exactly the kind of question we enjoy digging into.