A historic shift is underway in the LLM industry. Neural networks have learned to physically calculate.
The startup Percepta has published a study (currently a Proof of Concept) that has the entire AI community buzzing. They've baked a virtual machine (WASM interpreter) directly into the Transformer's weight matrix.
What's the fundamental problem?
Regular ChatGPT and Claude don't calculate math; they guess it (hence the hallucinations in exact problems). Currently, they're fixing this with hacks: they're forcing the AI to write Python code and run it in an external sandbox.
Percepta has proven that the Transformer is capable of executing the most complex machine code directly within itself with 100% accuracy!
What it looks like (pure cyberpunk):
Instead of generating text across the screen at 30,000+ tokens per second, machine code flies – the neural network juggles registers and branching. The authors fed the world's most complex Sudoku into the weights. The Transformer physically executes the search algorithm: it substitutes a digit, catches a contradiction, backtracks, and finds the answer. No hallucinations, pure logic.
How did they break the limitations?
They didn't do this before because of Attention Bottleneck.
A classic neural network is forced to reread its entire generation history at every step. At the millionth computational step, it would simply consume all the memory and die.
The authors invented Exponentially Fast Attention – searching past data in logarithmic time. The Transformer stopped being slow and started running millions of steps in seconds.
A direct path to AGI? Andrey Karpaty has already commented on the article, expressing his respect. And the point here is far from just creating a fast calculator.
Modern LLMs are our intuitive, "fast" thinking (System 1). Program code is rigid, deterministic logic (System 2). What Percepta has created is a blueprint for merging them into a single "brain."
If this infallible mathematical coprocessor can be seamlessly combined with speech models, we'll get AI that will forever stop hallucinating in logic. Agents will no longer need external scripts - heavyweight simulations will run right inside the scales. And this isn't just an improvement on chatbots; it's a direct, technically tangible bridge to AGI.
The merge of System 1 (pattern-matching intuition) and System 2 (deterministic logic) reasoning is the holy grail for agent architectures. Current LLMs excel at the first but fail spectacularly at the second when you need mathematical precision.
Embedding actual computation into the transformer weights rather than relying on external tool calls (Python REPL, calculators) could eliminate a lot of brittle handoffs in agent pipelines.
The question is whether this scales to arbitrary computation or if it's limited to specific mathematical domains. If it generalizes, you could build agents that reason AND compute in a single pass without context-switching overhead.