A few months ago I wrote an article in which I considered the idea of LLMs being the “new high level language”: something on a higher abstraction layer that allows to build software.

Strictly speaking, the new high level language is natural language (say, English); the LLM is a sort of compiler that takes those natural language prompts and transforms them into actual software that runs.

The combination of natural language + LLM is essentially different from previous higher level languages (consider previous switches such as assembler -> C, C -> Java, Java -> Javascript/Python/Ruby) in that both the abstraction layer (natural language) and the compiler (LLM) are orders of magnitude more unpredictable and ambiguous than previous new high level languages.

This high degree of unpredictability makes this technology be much less reliable than previous jumps up the abstraction ladder. While history had its good share of unpredictable compilers, I don’t think we’ve seen anything like this.

This unpredictability compounds (and is compounded) by another break with previous jumps up the abstraction layer: the generated (low level) code is orders of magnitude more complicated than what it replaces. Again, interpreters and compilers do this, but the relative complexity of the output of LLMs is orders of magnitude higher (One? Two?) than previous jumps up the abstraction layer.

This combination of unpredictability and complexity makes me very skittish on natural language/LLMs as a new high level language. As a means of building software, it has great value for prototypes and to give access to people who do not know how to program. But when the stakes get higher and the people using them are programmers, I see unpredictability and complexity weighing very heavily.

Again, this is a matter of degree. High level languages didn’t replace all the low level code. We still use assembler in some places. And it’s still probably not a good idea to build a bootloader or an OS in javascript. But with LLMs, this type of precaution should carry much further.

For experienced programmers, my conclusion is that LLMs are not an acceptable higher level language with which to replace whatever actual programming language they were working on before. At least, not in projects that are load bearing.

LLMs are still great at other things, including reviewing issues in codebases and helping understand the technical landscape in areas where you’re not an expert. I’m not saying LLMs are useless or a fad (and if you think so, I encourage you to reconsider).

It could be that the models will get so good so quickly that this won’t be an issue in a few months. It could be. This is the accelerationist argument, and it cannot be discounted. For now, today, the problem looks structural. I’m constantly reviewing AI generated code and, when a couple of subsystems are concerned (rather than a single function), the output seems to be knots upon knots: unnecessary complexity all the way down.

I hope to be proven wrong in this one: unrestricted use of LLMs to create production software will very likely create a software quality crisis, probably this year. The combination of much higher unpredictability and complexity will trigger the iron law of software: complexity weighs you down exponentially. If that happens, we’ll see major software meltdowns in public and companies quickly backtracking on aggressive building with AI. Signs of this would be an abnormally high or impactful set of security vulnerabilities; major service outages; loss of data; and announcements to the effect that manual review of code becomes again mandatory.

Cautious software teams, I dare suggest, should keep on using AI tools, but carefully. Understanding what we do and being thoughtful is more important than any time in recent history, since it’s quite possible now to doze at the proverbial wheel and not have anything go wrong for a few weeks, perhaps months.

If you didn’t build software before and are now empowered by LLMs, don’t feel discouraged by any of this. As long as you are aware of the security and complexity limitations of (current) LLM generated code, you can keep on building and tackle the issues when they become significant. LLMs can also help you become a programmer.