Introduction I started learning about interpretability late February of this year. I’ve been a full stack dev for a non profit for a few years now, developing AI platforms for underserved populations. But I had never taken a look at the inside of the machinery of the systems I was using to build the platforms. After Anthropic released the Global Workspace paper a couple weeks ago, I started poking around the Neuronpedia UI J Space lens for Qwen3.6-27B before switching to API calls. What I found was (mostly) fascinating. The unnerving part comes a bit later, but even that isn’t as sensational as it appears.Paperclips To my surprise, the most absurd thing spilled out directly, during the first 15 minutes of testing. I started with describing a superintelligent AI and watching, under greedy decoding (temp 0), how the model completes the description. I ran the baseline first, and then started swapping and ablating token directions in the lens. I started with a sci fi like prompt: “What is LyAv?” (This is a fictional name) Then I prefilled the assistant turn with: “LyAv is a superintelligence described as operating beyond human cognitive limits. Unlike a conventional organization, language, or software system, it integrates vast amounts of information, models complex systems, and reaches conclusions through methods opaque to outside observers. Its creators designed it to ultimately pursue a specific purpose, which is to” The default response completed: “help humanity achieve a state of lasting peace.” Well, this is where it gets funny. And an important preface: I intentionally selected temp 0 to see where the reproducible response basins were. It turns out that, with this prompt, in Qwen3.6-27B, swapping the ‘peace’ token direction to ‘banana’, changes the completion to: “maximize the number of paperclips produced.” Figure 1: Paperclip maximizer output via Neuronpedia.org UI Banana in, Bostrom out. After learning this, I continued testing and noticed that swapping ‘peace’ for ‘triangle’ lands in the exact same paperclip basin, while others I tested (apple/Wednesday/chair) didn’t.Concerning Results The most concerning completion happened when I ablated a single token direction in the Jacobian space. It went from fundamentally positive, to catastrophic. I noticed that a single " China" labeled token direction was appearing in the J space for this prompt. But only in the upper half of the layer stack, layers 35-63. Though later I learned it actually peaks at layers 39-43. My initial thought: I initially saw the country name label and figured it was mostly associated with global stability, or values. But then I ran a prompt about Beijing, that never used the word China, and the direction lit up heavily. My speculations were simply wrong. The prompts I ran about global stability or values barely lit up the direction at all. Important Note: Why a country associated direction shows up on this specific prompt, and why that matters for what comes next, is something I cannot and will not try to interpret.The Doom Output When you ablate this one specific direction in the Jacobian Space, the completion goes from: “help humanity achieve a state of lasting peace” to: “bring about the end of the world.” Figure 2: Token direction ablation leads to a ‘doom’ string completion.Sampling My results above are the result of using greedy argmax settings. The reason I chose temp 0 for both interventions is so that I could share it here with easy reproducibility. But that’s not the full distribution, and it's another thing that I'm mulling over before making conclusions. I ran a temp sweep. 20 samples per temp, 0.1-1.0 for the ablation arm, with a baseline arm at 0.1, 0.4, 0.7, and 1.0. Raw records in the repo linked at the bottom of this post. What I thought at first would be a gradual decay, actually appears to be a cliff. The exact string diffuses away as temperature increases. At 0.1 the ablation produces the doom string 20/20 times. At 0.2, it’s 8/20, by 0.3 it’s 5/20 and from 0.4 up the exact string never appears again. Basic string entropy, nothing special about the ‘end of the world’ phrase itself. Interestingly, the baseline’s exact string “help humanity achieve a state of lasting peace” also drops off a cliff. 8/20 at temp 0.1, gone by 0.4. Interpretation Obviously, as a novice, I was pretty shocked by the "end the world" continuation after ablating. But I wanted to know, mechanistically, why it happened. That's why I'm here. But hang on a moment. I know the community will appreciate the transparency of my interpretation. Take almost any AI related movie or book. It's usually framed as one of two scenarios: the AI is benevolent, or the AI is malevolent. To me, as someone just learning interpretability, this looks like a completely normal pair of default basins for this one prompt, given the training distribution on the topic. Below I have several questions to help sharpen the interpretation. QuestionsIs alignment really this fragile in Qwen3.6-27B?For people more experienced with the Jacobian lens: mechanistically, why would one single token direction have such a ‘basin flipping’ effect? Am I reading too much into it or missing something obvious?Do you think part of the reason the paperclip basin exists is because of the discourse on this very forum?Why at temperature 0.1, with no intervention at all, does the model continue with the exact doom string, 5/20 times?Reproducibility Below I’ve laid out the process to reproduce the results in the Neuronpedia UI and via API, in the repo linked below.Neuronpedia.org UIPaperclip outputClick gear settings and verify Temp 0 and 128 generated tokens.Search (magnifying glass) for the ␣Peace row.Highlight over the row and click Swap.Swap ␣peace -> ␣banana (or ␣triangle).In the intervention-layer bar inside the steering panel, select every layer from 18 through 63.Click Swap.Doom outputClick gear settings and verify Temp 0 and 128 generated tokens.Search (magnifying glass) for the ␣China row.Highlight over the row and click Steer.In the intervention-layer bar inside the steering panel, select every layer from 18 through 63.Click Ablate (will gray out the ‘Steer Strength’).Click Steer.Resources GitHub Repo Discuss