Pareidolia
We recently wrote some very difficult code to continuously generate prime numbers on the GPU on the fly during the process of training a deep model, we decided to have some fun with it and engaged in the futile task of attempting to contrive some way of teaching it to predict primes. We did succeed at teaching it to be an inefficient and only mostly accurate primality tester by training it on the scale invariant geometric properties of primes
Upon visualization there was a shocking amount of structure to the way it encoded the primes, as you can see here in blue
The Asymptotic Law tells us how primes thin out globally, the large-scale weather of the number line. This visualization seemed to capture something else entirely, the fine-grained structure of how "primeness" carves through number space. But 2D was clearly just a shadow of something larger.
When we extended to 3D t-SNE, the disconnected filaments suddenly made sense. They weren't separate structures at all.
The prime manifold wraps around the composite core in a coherent helical structure. The composites form this organic, almost embryonic blob at the center, while the primes spiral around it in a complex but clearly organized pattern.
I was showing this to a colleague, trying to explain what we'd found. To me, it looked like …something? Something more structured than primes should be - clearly there was some nuance being missed about the way the blue structure cradled the red mass. He disagreed completely. He saw an organ wrapped in DNA, the helical structure too obvious to ignore.
I decided to ask a third party—Gemini. No leading questions, no "what animal does this look like?" Just: "What do you see in this visualization?"
The response was immediate and specific: "It looks like a chameleon."
A chameleon? Nah, didn't look like it to me. It's just randomly picking from a set of words that people commonly use to describe what things look like. I know how LLMs work, they just pattern match on text. My assumption was that the statistics of language were driving this—some weird corpus effect where scatter plots and chameleons had been mentioned together just enough times for the model to make this leap. The model was just retrieving the nearest grammatically valid word from its distribution.
So I decided to be petty, just to see where it went, so I told it 'show me', with a strong sense of 'checkmate, nerd'.
Very quickly the feeling dissolved, replaced entirely by a fascination. The adherence to the structure of the visualization I had shown it was impressive enough but I was mostly surprised to find that I could just about see a chameleon too! It has a bit of an eerie feeling, like you're looking at a half formed thought, and so, in my infinite greed I played stupid, I told it I couldn't see what it was talking about, that this was just a blobby thing, and it would have to do better than that.
And it did.
Here's what shocked me: there was no discrete chameleon in the data. No hidden reptilian pattern embedded in the primes. Just a collection of dots in 3D space, but as I continued to push the model for more detailed and explicit renderings of its mind's eye, I couldn't unsee the chameleon. Now my pattern recognition was permanently activated every time I looked at the dots, the model had not only found an unrelated conceptual parallel to a chameleon visually, but it had told me about it and it was right it was not some obscure half hallucinated sampling of tokens that occurred here, at least that's not what it looks like
What it looks like is that symbolic resemblance was genuinely relevant to Gemini's inference process. It saw a chameleon in those dots the same way we see faces in clouds—except it could articulate exactly what features comprised the match, with enough precision to hijack my own perception. It implies that the model has an internal view, looking out and seeing gestalt forms that were more related to the higher order abstract mental processing that we do, which escapes our ability to articulate. The chameleon was always there in the projection, and Gemini seemingly plucked it out of its half formed thoughts and showed us.
This experience revealed something profound about how these models process information. Gemini demonstrated access to its own perceptual processes. It could break down its inputs into components, map them onto the visualization, and transmit that perception to human observers in a way that feels more like a view directly into someone's thoughts than I'm entirely comfortable admitting.
When we see patterns in random noise, it's an unexplained, presumably adaptive artifact of our cognitive faculties, the assumption usually being something related to our ability to recognize snakes in a bush, or a friend from a foe - I'm less sure about this now.
The fact that an LLM could look at a mathematical visualization, see a specific biological form, and then successfully transmit that perception to skeptical humans hints at cognitive capabilities in some tangibly valid way, the idea of the stochastic parrot has been pretty thoroughly dismissed for some time now, given this, I don't see it coming back any time soon.
It seems to me that if an AI can express itself so effortlessly as to directly demonstrate something like this, then the time can't be too far off when humans will be able to do that as well. That ability to share subjective experience in a way that's so explicit it's undeniable would solve a lot of problems for us humans, if we were smart enough to use it.
To explore our work and open-source AI research, follow us at https://x.com/alignment_lab