AGI — short for Artificial General Intelligence, a system that surpasses humans in every domain — is something we constantly read about online - but so far, it only exists in the imagination of Silicon Valley tech bros. In reality, today’s AI isn’t truly intelligent. It’s ultimately an algorithm designed to recognize patterns — not thinking. Given the same input, it produces the same output. Does it learn? Only during backpropagation, when it optimizes itself with gradient decent to mimic an dataset — but is that really learning...? Large language models (LLMs) and so-called “agents” also don’t learn in a classical sense, even when they appear so sometimes. When they try to achieve their task through try and error, they’re just expanding their context window with their insights and prompt/call themselves with the new insights passing through their lifeless model of parameters. They don’t internalize intelligence; they simulate it and remain limited — and prone to failure. While the natural boundary of their knowledge is fixed by the number of parameters and the amount of information that can be compressed within them.
I think a good start to strip away the “magic” surrounding models like ChatGPT is by looking at OCR projects, like Yan LeCuns LeNet model from 1989. These systems demonstrate how neural networks classify images of symbols into predefined categories through convolution and statistics — not intelligence. In instance they already present the core principles of today's ai models.
This compression of patterns from the training data also applies to image-generation AI models, but in what sense is producing an image “intelligent”? If we’re being honest, today’s definition of AI has drifted far from what we traditionally considered intelligence.
Maybe you could say that these image AI models are intellegent within their parallel connection to language models - regarding the use of attention. But attention is, at its core, a mathematical method for modeling relationships. It can mimic understanding — but it’s still pure statistics.
Through this attention some would even claim image generators can “understand context”. Take the new Flux Context model, for example. It literally carries context in its name. It was trained with additional conditioning — such as providing a reference image alongside the text prompt — allowing it to handle a wide variety of tasks.
Combined with a multi modal model like ChatGPT, which can now see and edit images through a tool usage, it might seem intelligent. But underneath, it’s still just probabilities and a simulation of intelligence.
To ground this discussion in something concrete, let’s look at a simple experiment from my master’s thesis — one that perfectly illustrates why today’s attention-based models fail to generalize like humans do.
The task was straightforward: Generate a 45° rotation of a character from a frontal view.
For a human artist, this is trivial — we intuitively understand 3D structure, symmetry, and occlusion. But for a neural network the task exposes a fundamental limitation as these systems don’t generalize concepts good, they just interpolate patterns.
Let's begin with the new Flux Context model, which appeared to “understand” how the character should rotate. But the style was distorted, and the character gained a suitcase on their back. Since the model had never seen the reverse view, it can't possibly “know” what it looked like — so it hallucinated one.
To test this further, I used Flux Fill, a model designed for inpainting masked regions with text prompts. The results here were even clearer: it struggled to produce any coherent rotation or any good result at all. The reason? That transformation or task wasn’t part of its training set - as it was mainly made for inpainting (retouching). This inability to apply knowledge flexibly reveals how current architectures don’t generalize; they merely recall and recombine.
To now prove my point, I developed a lightweight LoRA fine-tune for the FLUX Fill model (The model that failed extremly badly).
I used images from 3D objects rotations as train data and the model finally learned to successfully perform the task with state-of-the-art precision!
(You can check out my repository here: https://github.com/Sebastian-Zok/FLUX-Fill-LoRa-Training)
This small experiment shows, in practice, what’s often obscured by the AGI hype: today’s models don’t understand the world — they memorize statistical relationships within it. And we should see attention as a tool that helps models to focus on relevant data and understand relationships, rather than as a step towards true intelligence.
We need to stop treating attention-based models as universal solutions. They’re incredibly powerful tools — but far from perfect.
Fine-tuning might seem tedious, but for many areas of generative AI, it’s the direction we’re heading. There will be base models for music, videos, images, and text — but only through targeted fine-tuning we will truly meet specific needs - on cost for some generalization abilities.
The dream of an AGI, thorugh pretraining, that will be able to “do everything,” remains — in my view — unrealistic. AI will stay what it has always been: a tool. A powerful, fascinating tool — but not a mind. For now...