AI Won't Replace Software Engineering (Yet)

I have a boring opinion: AI is a tool.

Like a text editor, a compiler, grep. And like every tool, it's clueless, easily nudged off track, and sometimes too eager to please. That's not a flaw of character. It's a stochastic model, not a person.

It is a groundbreaking tool. My third most-used after the text editor I'm typing this in and the shell running it. No tool has rooted itself this deep, this fast. Long-abandoned projects look polished. Ideas I shelved as too tedious are a prompt away. It's almost magic.

The numbers feel magic too:

• OpenAI processes ~6 billion API tokens per minute; the Library of Congress, every three seconds.
• Anthropic went from $87M to $30B in revenue in 27 months; a 345× run.
• NVIDIA hit $4T in July 2025 and $5T in October 2025; the first company at either line.

The headlines are magic too:

• Frontier models now score ~2× expert virologists on the Virology Capabilities Test; biology benchmark scores rose >4× from Nov 2022 to April 2025.
• Two of the four 2024 Nobel Prizes went to AI work; Hassabis and Jumper for AlphaFold, Hopfield and Hinton for neural networks. First Nobels for AI, ever. Two in one year.
• On 2-hour tasks, AI agents beat human experts 4×. On 32-hour tasks, humans win 2-to-1. There's a duration cliff in agentic work, and almost all real engineering lives on the wrong side of it.

Still, I'd taper expectations. The underlying tech is human language, not a model of the world. Language is a symptom of intelligence, not the cause. AI is still a tool.

Replacing Software

I've been coding with AI assistants for over two years. The shortest summary I have:

The best intern I could ask for. It listens, chips away, makes some bad assumptions. On familiar ground, it shines. Off-trail, the output is off-trail too. With clear direction, it solves most of what a working engineer hits in a day.

I've watched it fall flat in narrow places (LaTeX Tikz, custom diagrams) and broad ones (overweighting the current file instead of reaching for world knowledge). It's not just me:

• METR ran a randomized controlled trial: 16 experienced open-source devs, 246 real GitHub issues. With AI tools enabled, they took 19% longer to ship. Afterwards, the same devs estimated AI had sped them up by 20%. A 39-point gap between feeling and measurement, on their own codebases.
• MIT's Project NANDA found 95% of organizations investing in generative AI saw "little to no measurable impact on P&L" despite $30–40B in enterprise spending. Internal builds succeed at one-third the rate of purchased ones.
• Goldman Sachs, March 2026: "no meaningful relationship between AI and productivity at the economy-wide level". Only 10% of S&P 500 management teams quantified AI's impact on a specific use case. Just 1% quantified it on earnings.
• Only 21% of generative AI users have redesigned a workflow around the tool. The rest are layering AI on top of processes that don't need it.
• Hallucinations are the #1 documented AI failure mode: 38% of catalogued incidents, more than nonsensical output (25%) and fabrication (15%) combined.
• AI incidents tracked by the AI Incident Database climbed from 149 (2023) to 233 (2024) to 362 (2025). The trendline is accelerating, not slowing.

My workflow is unrecognizable from two years ago. The objective hasn't moved: ship software that scales. Code is one piece. AI helps with most of them, but someone still has to orchestrate top-to-bottom. AI turns a rowboat into a crewed ship, and the rower into the captain. They route, monitor, and adjust. I find AI roughly useless without me at the wheel, nudging it back on course.

This may age badly, but I'll stake it: the software engineering industry keeps growing for a while yet. It contracts later, when models meet or beat human intelligence and the captain stops being load-bearing. For now we're building the ship and flying it at the same time.

So if we aren't being replaced, what are we doing?

Raising The Bar

When the compiler arrived, demand for engineers went up. More ambitious projects became feasible; the harder problems got reached for, not skipped. Productivity tools tend to do that — they raise the ceiling before they shrink the floor.

The lesson: lean into where AI is strong. Onboarding, legacy code, the routine. I haven't written an argparse CLI by hand in over a year, and I don't miss it. Side projects that would have taken months take days. The cobwebs in my codebases are fun to clear, not tedious. Professionally, however, it's a different story.

The difference is simple: I read every line of code I ship at work. I don't read every line I ship at home. Side-project stakes are low. if it breaks, someone files an issue or sends a patch. Production stakes aren't.

When models pass human intelligence, I'll stop reading the code to fix it and start reading the code to learn from it. It'll know more than I do. Until then, intelligence is bounded by context windows and language. I'm not staking a career on a pile of tokens and linear algebra — neither by skipping the code review, nor by quitting the field.

A few real obstacles on the way there:

• Only 3% of developers "highly trust" AI output. About 46% explicitly don't. 75% still ask a human when they don't trust the model.
• Hyperscaler AI hardware depreciates fast enough that ~$400B in annual depreciation outpaces their combined profits. The denominator can't keep up with the numerator.
• Art history grads (3% unemployment) are now more employed than computer scientists (6.1%) and computer engineers (7.5%). Twenty years of "learn to code" inverted in twenty-four months.
• US data centers used 183 TWh of electricity in 2024 — roughly all of Pakistan's annual consumption — projected to grow 133% to 426 TWh by 2030.

Those are problems for us, as humans, to fix.

As engineers, we do what we did when the compiler shipped, and the IDE, and the package manager: we build more. Better. Sometimes the bottleneck is human review; sometimes it's reaching past where the model can see. The bar moves up.

Every engineer now runs a small team of capable agents. We can build anything we can specify. We can serve very small audiences with very specific software, or very large ones with very hard software. The work isn't going away. It's getting harder, and more interesting, at the same time.

Still The Captain

I trust the model on the easy stretches. I read every line on the hard ones. That doesn't change until the ship doesn't need a captain. And from where I'm standing, mid-flight, it still does.