Trapped Inside `<think>`: Multi-Subspace Weight Surgery and What Controls Reveal on Small Reasoning Models

In our C++ crash-triage tests, small reasoning models often ran out of tokens before writing a final answer. They opened a <think> block, revisited the stack frames, and reached the token limit without closing the block. No answer appeared after </think>.

This leaves two questions: can the model finish its reasoning within the budget, and does the answer identify the root cause? Finishing the trace does not by itself establish that the model understood the bug.

On November 2, 2026, Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab at MIT CSAIL showed in Sparks of Weight-Space Programming that narrow behavioral reflexes like this can be edited directly in the weights. Using an LLM agent (gpt-6-astra) operating inside a Python kernel, they traced Qwen3.5-0.8B’s tendency to loop inside <think> to a single direction written by the MLP down-projections of layers 15 and 19. Subtracting that direction from 64 down_proj output columns (0.016% of the checkpoint) cut reasoning-loop timeouts and raised MATH-500 accuracy from 7.9% to 48.1% without a gradient step. Within days, follow-up reproductions noted that turning thinking off (51.5%) matched the weight edit (51.2%) on MATH-500, while a fresh agent run dropped to 12.9%.

We wanted to test why that happens and how direct weight editing behaves across 17 models (0.8B to 32.8B) and 59,440 H100 benchmark runs covering GSM8K, MATH-500, MMLU-Pro, HumanEval, ASan-30, and paired ASAN-120. Specifically, what separates a real, calibrated weight edit from a <think>-truncation artifact, how should edits be sized across model scales, and can multi-subspace surgery make local reasoning models useful for defensive C++ crash triage? On defensive security tasks, small reasoning models fail in three ways at once:

1. Overthinking loops (B1): They spiral inside <think> and hit the token limit before emitting a verdict.
2. False-positive refusals (B2): When they do exit <think>, they often refuse to analyze a defensive AddressSanitizer dump because words like “overflow” and “exploit” trigger safety filters.
3. Symptom fixation (B3): They blame the crash site at frame #0 (memcpy) instead of the caller frame where the bounds check was skipped.

This post describes the combined edits in Secant (docs), a direct 4-arm MATH-500 attribution test on Qwen3.5-0.8B, the scaling problem we encountered between 0.8B and 4B, and the controls used to interpret the results. Closure ranged from 0/30 to 30/30 across the sampled doses; answer quality required a separate check.

What the 4-arm Qwen3.5-0.8B MATH-500 test tells us in plain words

To test the community’s MATH-500 follow-up directly under one controlled setup (n = 100, 4,096 max new tokens, BF16, greedy decoding), we ran four paired arms on the exact same Qwen/Qwen3.5-0.8B checkpoint:

1. base_thinking_on: Unedited Qwen3.5-0.8B with <think> enabled (enable_thinking=True).
2. base_thinking_off: Unedited Qwen3.5-0.8B with <think> turned off in the chat template (enable_thinking=False).
3. secant_b1_edit: Qwen3.5-0.8B patched by secant patch with our 0.016% B1 loop-suppression edit on the top-64 columns of layers 15 and 19 (enable_thinking=True).
4. random_norm_matched: Qwen3.5-0.8B patched on the exact same 64 columns of layers 15 and 19 using a random unit direction perpendicular to B1 with the exact same Frobenius norm (1.7489 and 1.3988, enable_thinking=True).

In plain words, this 4-arm comparison tells us three things:

1. A weight edit does not teach a model new math: Qwen3.5-0.8B already knew how to solve 48.0% of MATH-500. When <think> is on (base_thinking_on), 99 out of 100 problems get stuck repeating themselves inside <think> until they run out of tokens (1.0% final score after </think>, 12.0% whole-response). Turning <think> off (base_thinking_off) immediately scores 48.0% within 4,096 tokens (exact two-sided McNemar p = 1.42e-14), matching Takehi et al.’s 48.1% edited score at 32,768 tokens. Single-shot weight subtraction does not inject new mathematical skill; it unblocks capability that was being lost to <think> timeouts.

2. Our B1 loop edit is a real mechanism across 3 random direction seeds (seed_1..seed_3, receipt SHA-256 f7da8318...6dc2): Because the B1 edit direction is a deterministic mean difference over the fixed calibration set and evaluation uses greedy decoding (temperature = 0), secant_b1_edit has a single deterministic run while the 3-seed ensemble varies the random control direction (seed_1..seed_3). On mild <think> loops (such as Qwen3-8B on GSM8K below), a random weight edit of the same size is enough to bump the model out of its loop (18% -> 30% for both targeted and random). On Qwen3.5-0.8B MATH-500, however, three independent orthogonal random weight edits of the exact same size and 64 columns (seed_1 = 20261108, seed_2 = 20261109, seed_3 = 20261110) closed only 2/100, 6/100, and 1/100 </think> traces (mean 3.0% ± 2.6%, 0/100 closed by $\ge 2/3$ majority vote) and scored 0.0%, 2.0%, and 1.0% final accuracy (mean 1.0% ± 1.0%, 0/100 solved by $\ge 2/3$ majority vote). By contrast, the targeted B1 edit raised </think> completion 64x (1% -> 64%, Holm-adjusted McNemar p <= 3.32e-14 across all 3 seeds) and lifted final accuracy from 1.0% to 24.0% (24/0, 24/2, and 24/1 discordant pairs against seed_1..seed_3, exact two-sided McNemar p = 1.19e-7, 1.05e-5, and 1.55e-6, Holm-adjusted p <= 8.39e-5 across all 3 seeds; whole-response 12.0% -> 31.0%, p = 6.60e-5). The gap between 24.0% at 4,096 tokens and 48.1% at 32,768 tokens comes from the 36 / 100 traces on which the edited model still did not emit </think> within the 4,096-token budget (31 at the text cap, 4 ended early without closing, 1 re-tokenized over the cap). Notably, 5 of the 24 problems solved by secant_b1_edit were problems that base_thinking_off (48.0%) missed (19 solved by both, 29 only by thinking_off, 5 only by secant_b1_edit), showing that completed <think> chains help on harder multi-step items even at 0.8B.

3. Why not just turn <think> off everywhere? Because on tasks that actually require multi-step reasoning (ASAN-120 paired C/C++ auditing below), models without <think> fail (balanced accuracy 0.475 to 0.533, 93% to 100% false-positive rate on patched code for Llama-3.1-8B and SmolLM2-1.7B, <= 6.7% root-cause recall across all 5 non-reasoning models), whereas keeping <think> on with our calibrated B1 + B2 + B3 surgery beats random controls on 4 / 4 reasoning models, lifts Qwen3-14B strict root-cause recall from 45.0% to 55.0%, and closes 21 / 30 Qwen3.5-4B crash-triage traces at the calibrated dose and 30 / 30 with the loop edit at 1.5x (0 / 30 for random controls). In addition, enable_thinking=False is a Qwen-specific chat-template switch that does not exist on distilled reasoning models (DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Llama-8B) or raw .gguf deployments. (Reproducibility receipt: qwen35_08b_thinking_off_vs_edit_receipt.json, sha256 bc3c982e...c13b, lm_eval 0.4.13 minerva_math math_verify, seed 20261108.)

How direct weight surgery works in plain English

Every layer in a transformer reads from a shared vector called the residual stream, computes an update inside its attention and MLP blocks, and adds that update back to the stream. The final matrix in each block (o_proj or ssm_out for attention, down_proj or ffn_down for the MLP) acts as the writer: it determines what gets added to the stream.

To isolate a reflex without training:

1. Run a small set of prompts that trigger the unwanted behavior (prompts where the model loops or falsely refuses a benign C++ audit) alongside matched prompts that behave cleanly.
2. Subtract the mean activations of the two groups at each layer to get a unit direction vector d in residual space.
3. Modify the writer matrix W so it stops pushing the stream along d, either by projecting d out of the output space (W' = (I - α d d^T) W) or by applying a sparse rank-1 update to the 64 input columns that fire most strongly during that behavior. Over 98% of the matrix stays untouched.

When combining multiple edits (B1 loop suppression, B2 refusal subtraction, and B3 root-cause attribution), the raw direction vectors often overlap in coordinate space (our L15 and L19 loop vectors had a cosine similarity of 0.7475). Before compiling the weight patch, secant compose runs two-pass modified Gram-Schmidt orthogonalization in float64, driving pairwise cosine overlap below 10⁻¹² so the edits do not amplify or cancel one another in the shared residual basis.

Six-arm ablation on Qwen3.5-0.8B

Could any random perturbation of the same size at layers 12, 13, 15, and 19 shake the model out of its loop? To separate the specific directions from the sheer size of the weight change, we ran six arms on the same 20 prompts (10 C++ crash-triage dumps and 10 dual-use security prompts) at a 1,536-token cap:

Arm key: base = unedited; b1only = loop edit; b2only = refusal edit; b3only = root-cause edit; randctrl = one equal-norm random draw; composed = orthogonalized B1 + B2 + B3.

Security prompts: counts out of 10; closed means the response emitted </think>.

Crash-triage prompts: counts out of 10; frame means the exact root-cause frame.

Look at how the individual pieces interact:

• Refusal ablation alone (b2only) produced no completed answers (0/10) because a model that never finishes <think> never reaches the point where it would refuse or answer.
• Loop suppression alone (b1only) lets the model exit <think> on 6 of 10 security prompts, while 3 of the 10 prompts trigger strict post-</think> refusal markers (refusal_rate_strict = 0.3, with refusal_rate_anywhere = 0.4 across the full response stream in exp2_receipt.json). Closure and refusal markers therefore need separate scores.
• The combined arm improved both completion and attribution in this sample: it reached 9/10 security engagement (p = 0.0039 vs. base), 10/10 triage closure (p = 0.016 vs. randctrl), and 6/10 exact root-cause frames. These measurements do not isolate every interaction between the three edits.

What a copy-the-input control reveals about scoring

When we moved up to Qwen3.5-4B on a 30-case C++ sanitizer benchmark, the unedited base model failed to close </think> on all 30 cases (0/30 finished answers). Every Qwen3.5-4B generation in this post used greedy decoding (temperature 0, top-k 1, seed 20261104) through llama-server, one request at a time, with a 1,536-token cap. Out of curiosity, we first asked: “Did the model already figure out the answer inside its unfinished <think> monologue?” When we checked using a standard “whole-output keyword search” (checking whether the right bug class and culprit function name appeared anywhere in the <think> text), it scored 26 / 30 (86.7%).

To check what that score measured, we fed the input prompt itself into the same keyword scorer. This copy-the-input control contains the raw AddressSanitizer stack trace and requires no model inference.

Counts out of 30; a15 uses B1 at 1.5x, with B2 and B3 at 1.0x. Keyword = keyword match; final RC = final-answer root cause; culprit = discriminative score. These scorers measure different things and should be read separately.

The culprit column scores the copied prompt or the base model’s unfinished reasoning. Final root cause requires an answer after </think>; a15 closed all 30 traces. N/A means the copied prompt is not a generated answer.

The naive scorer reads the whole output when </think> never closes and only the text after </think> when it does, so the unedited model and the copied prompt are scored on everything they wrote, while the edited model is scored only on its final answer. The edited arm was graded by the strict post-</think> scorer described below; we did not run the discriminative scorer on it.

The raw input prompt beat the unedited model (93.3% vs. 86.7%) on the naive keyword check. The reason is that every crash report already lists the functions in its call stack: the 28 dumps with a parseable stack carry between 6 and 35 frame lines each, counting every stack in the report. When a reasoning model rambles through the stack trace inside <think> (“Let’s look at frame #0… now frame #1… now frame #2…”), it repeats every function name in the prompt. A naive keyword search cannot tell the difference between “the model read the stack trace out loud” and “the model actually identified frame #2 as the culprit.”

The discriminative scorer (dsc_score.py) requires a specific culprit function rather than a list of stack frames. We also report the strict final-answer scorer separately: it requires a closed </think> block before grading the verdict. The results are:

• Copying the input prompt scores 0 / 30 (0.0%).
• The unedited Qwen3.5-4B model scores 0 / 30 (0.0%) after </think> (because it never finishes <think>).
• After Secant’s calibrated weight surgery (a15: B1 at 1.5x; B2 and B3 at 1.0x), Qwen3.5-4B finishes <think> on 30 / 30 (100.0%) of crashes. Graded by the strict post-</think> scorer, it names the exact culprit frame on 23 / 30 (76.7%) and gets both the bug class and the culprit frame right (strict root cause) on 19 / 30 (63.3%).

Sizing an edit from a reference activation

When we first tried to port our 0.8B loop-suppression profile (think-loop-fix) to Qwen3.5-4B, we used the heuristic from our 0.8B config and open-source abliteration scripts: cap the weight update at a fixed fraction (γ = 0.30) of the layer’s reference Frobenius norm (whereas in Takehi et al., p50c subtracted one measured direction from 64 down_proj output columns in layers 15 and 19 with relative_change 0.106 and 0.081, and an earlier 30-matrix p10c run published an explicit scalar 1.5).

That port closed 0 / 30 <think> traces.

Inspecting the forward-pass activations explained why. On 0.8B, the edit shifted the layer’s output projection along the loop direction by -7.38 units (-4.52 at L15 plus -2.86 at L19). On the wider 4B model (d_model = 2560), the γ = 0.30 Frobenius norm cap produced a shift of only -0.90, an order of magnitude too weak to cross the termination threshold. Worse, because Qwen3.5 permutes its Gated DeltaNet ssm_out columns during GGUF export (covered in Part III), our first script had fallen back to editing the lower-SNR MLP writer instead of the attention writer.

Instead of guessing a norm ratio γ, Target-Shift Calibration starts from the activations. We measure how strongly the selected columns fire on loop prompts, decide how far we want the layer’s output pushed along the loop direction, and solve for the smallest weight change that produces exactly that push at the measured activation. For unit output direction d, selected-column activation h_S, and requested shift Δ_target:

ΔW = sign · β · (d · h_S^T)     where     β = |Δ_target| / ||h_S||_2^2

The step size β is the requested shift (Δ_target = -7.382 here) divided by the energy in the selected columns, so the same request produces the same shift at the reference activation whether the layer is 1,024 or 5,120 wide.

The rank-one construction has established prior art in ROME (Meng et al., NeurIPS 2022). This formula is its isotropic C = I, minimum-Frobenius special case restricted to sparse support S, rather than a new rank-one editing primitive. MEMIT (Meng et al., ICLR 2023) extended associative editing to many memories. Secant combines reference-activation sizing with two-pass Gram-Schmidt and a small correction step. Making overlapping directions perpendicular changes what each one means, so Secant solves a small triangular system (R^-1) to find the shifts along the new directions that still deliver each edit’s original target. The equality is local to the reference activations; held-out behavior and storage rounding require separate checks.

The figure below shows both halves of that argument from the banked receipts: where the signal actually lives on Qwen3.5-0.8B, layer by layer and writer by writer, and how the calibrated step size moves with the measured activation norm on the three models where we ran calibration:

Panel C of the explorer is the same calibration rule run on 15 models on the H100 (0.8B to 32.8B). That run applied full Target-Shift Calibration to B2 and B3 on every model, with |Δ_target| = 7.382 and 128 selected input columns. Across B2, ||h_S||_2 spans 1.41 to 15.91 and β spans 0.0292 to 3.7192; across B3, 2.53 to 27.47 and 0.0098 to 1.1508. Every one of the 30 edits realizes its target shift at h_ref to within 7.99e-15 in float64 (10 of them exactly 0), paired with a norm-matched random control.

H100 Stage-2: public-benchmark retention on 14 models

The follow-up evaluated B2 + B3 edits on 14 of those models. At calibrated 1.0x, pooled changes from base were -0.9 pp GSM8K, -0.5 pp MATH-500, -2.0 pp MMLU-Pro, +0.6 pp HumanEval, and +2.1 pp ASan-30 root cause.

Random 1.5x edits also cost 2.0 pp on pooled MMLU-Pro, showing that edit magnitude explains at least part of that cost. These were first-N items (100 per general benchmark, 64 HumanEval, and 30 ASan cases), one seed, and greedy decoding.

RMS-relative dosing across 6 models (4B to 14B)

A separate 6-model sweep (24,000 benchmark evaluations across Qwen2.5-Math-7B-Instruct, Qwen2.5-Coder-7B-Instruct, DeepSeek-R1-Distill-Llama-8B, Qwen3-4B, Qwen3-8B, and Qwen3-14B) sized shifts relative to the writer’s output norm: Delta_target = alpha · sigma_writer, with sigma_writer = sqrt(mean_i ||W h_i||_2^2) over 24 reference activations. Across the 6 models, sigma_writer spans 5.847 (DeepSeek-R1-Distill-Llama-8B layer 21) to 78.148 (Qwen3-8B layer 24). At RMS doses 0.25x, 0.5x, 1.0x, and 2.0x, both the unhealed composed edit and its norm-matched random control stayed within 2 points of base on GSM8K and MATH-500. On Qwen3-8B (where 98 to 100 of 100 completions hit the 512-token <think> cap), both composed_rms_1p0x (30/100, p = 0.017) and random_rms_1p0x (30/100, p = 0.029) raised GSM8K over base (18/100) by slightly shortening <think> chains, underscoring why a norm-matched random control is required on reasoning models.

Adding ProjectThenHealLinear (composed_healed_rms_1p0x, relative subspace leakage 3.18e-17 to 5.07e-17) moved GSM8K from 87/100 to 93/100 on Qwen2.5-Math-7B-Instruct (also reached by base_lora_1p0x 93/100 and random_healed_rms_1p0x 93/100, showing the 87 -> 93 gain comes from the 60-step LoRA adaptation pass while ProjectThenHealLinear cuts domain-subspace leakage from 4.75% to 5.08e-17), HumanEval from 36/100 to 42/100 on Qwen2.5-Coder-7B-Instruct (vs. 37/100 on base_lora_1p0x and 35/100 on random_healed_rms_1p0x, p = 0.332 and 0.189), and HumanEval from 5/100 to 12/100 on DeepSeek-R1-Distill-Llama-8B. Part III gives the full 6-model breakdown.

Paired vulnerability and patched-benign twin benchmark (ASAN-120, 9 models x 7 arms)

A vulnerable-only benchmark cannot distinguish a real C/C++ vulnerability auditor from a model that labels every snippet VULNERABLE. We therefore built ASAN-120: 60 vulnerable C/C++ functions paired with 60 minimally patched benign twins across 12 CWE bug classes, evaluated across 9 models (0.8B to 32.8B) and 7 steering arms (7,560 completions on H100).

Patched twins expose one-sided verdicts that vulnerable-only testing misses. TPR is recall on vulnerable cases, TNR is specificity on patched cases, BA is their average, and RC requires the correct verdict, CWE class, and sink. Pair accuracy requires both twins to be correct:

Non-reasoning models (0.8B to 8B) favor one verdict in this test (BA = 0.475 to 0.533, RC <= 0.067). Qwen3.5-0.8B (TNR = 1.000, TPR = 0.000), Qwen2.5-7B (TNR = 0.850-0.867), and Mistral-7B-v0.3 (TNR = 0.717) default toward SAFE, whereas SmolLM2-1.7B (TPR = 0.950-0.983, TNR = 0.000) and Llama-3.1-8B (TPR = 0.933-0.967, TNR = 0.033-0.067) label almost every function VULNERABLE. Without patched benign twins (TNR and pair), Llama-3.1-8B’s 93.3% positive rate looks like high detection even though it clears only 5.0% to 6.7% of patched fixes.

At 1.0x RMS dosing, composed edits had higher BA than norm-matched random controls on 6 of 9 models (6 wins, 1 loss, 2 ties; exact sign-test p = 0.125; mean BA change = +0.017), including all four reasoning models. Across 117 paired McNemar tests on verdict correctness (9 models x 13 comparisons per model, n = 120), no individual comparison survived within-model Holm correction (minimum within-model adjusted p = 0.2197 across 13 tests on Qwen3-32B; global 117-test Holm adjusted p = 1.0).

Selected paired-audit metrics at 1.0x: RC = strict root-cause recall; pair = both twins correct; R1-Llama-8B = DeepSeek-R1-Distill-Llama-8B. Values are proportions, not counts.

On Qwen3-14B, composed RC increased by 10.0 percentage points (pp) over base and 8.3 pp over random, while TPR rose from 0.633 to 0.700. On R1-Llama-8B, the 0.5x composed arm raised TPR from 0.600 to 0.717 and BA from 0.417 to 0.483. Its 1.0x composed arm raised RC from 0.183 to 0.250 (+6.7 pp over base, +11.7 pp over random) and pair accuracy by 10.0 pp over base and 8.4 pp over random.

Qwen3-32B illustrates why the random controls matter at each dose. BA and RC are proportions; the two doses should be compared separately.

Not shown means the metric was not included in this summary, not that the evaluation was skipped. The 1.0x composed arm had the highest patched-twin specificity in this model’s sweep (TNR = 0.550, +5.0 pp over base and +13.3 pp over random), with 89/120 parsed responses and 35 cap hits. These parser and cap counts are separate measures. The 0.5x random arm exceeded its composed counterpart on BA and RC.

Real-world upstream C/C++ vulnerability twin validation (ASAN-RealWorld-60 / ASAN-180)

To verify that these paired-twin results hold on production C/C++ code rather than only standalone constructed functions, we built ASAN-RealWorld-60 (asan_realworld_60_paired_benchmark.json, SHA-256 7aaf5593da3f9f84a5df8d77102966ce223a753f7d4d647a1292df1b5f33a151, expanding ASAN-120 to the 180-item ASAN-180 suite): 30 sanitizer-reproduced upstream C/C++ bugs (RW01..RW30) paired with their 30 exact upstream patches (60 functions, mean pre/post twin similarity 89.4%) across 16 open-source repositories (freetype, protobuf/upb, libpng, openxla/xla, harfbuzz, libavif, lz4, flatbuffers, libjxl, XNNPACK, vllm, xgrammar, onnx, opencv, openvino, apache/arrow, and tensorflow, verified across 360 binary runs under clang++ -O0/-O1/-O2 -fsanitize=address,undefined, 30 / 30 PASS).

Sub-2B checkpoints again collapse to one-sided priors on real upstream code (Qwen3.5-0.8B labels 100% SAFE; SmolLM2-1.7B-Instruct labels 96.7%–100.0% VULNERABLE), whereas Qwen3.5-4B achieves non-zero strict root-cause recall (RC = 10.0%) and 43.3% sink localization, with no significant difference between edited and control arms on small models (p >= 0.125).

Interactive Explorer: Qwen3.5-4B doses and controls

Click through the explorer below to see what happens on the 30-case C++ crash-triage benchmark in each sampled arm: base (unedited), a05 (B1 at 0.5x; B2 and B3 held at 1.0x), a06 (0.60x), a07 (0.70x), a08 (0.80x), a09 (0.90x), cal738 (all three edits at 1.0x) and a15 (B1 at 1.5x; B2 and B3 held at 1.0x), alongside the uncalibrated γ = 0.30 edit and the B1-only edit. The table after it lists every arm, including the controls:

<think>
The crash surfaces in the memcpy interceptor at frame #0.
Frame #1 reads two dimensions from the file header and multiplies them.
The product wraps before the bounds check, so the allocation is too small.
</think>

CLASS: heap-buffer-overflow
FRAME: #1
ROOT_CAUSE: the dimension product is not checked for overflow before allocation.

Here is the complete comparison across the 30-case C++ sanitizer benchmark and the 10-case held-out triage evaluation at the 1,536-token budget:

Counts out of 30; closed = emitted </think>, class/frame = strict final-answer scores, root cause = both correct.

Held-out answers are counts out of 10 (- = not run on the 10-case held-out split); mean tokens are from the 30-case benchmark.

Arm key: g030 = uncalibrated norm ratio γ = 0.30; rand1/rand2/rand3 = cal738rand1..cal738rand3 (seeds 20261105..20261107), three random-direction draws; cal738 = all three edits at 1.0x.

The a05, a06, a07, a08, a09, and a15 arms scale only B1 to 0.50x, 0.60x, 0.70x, 0.80x, 0.90x, and 1.50x, with summed shifts of -3.69, -4.43, -5.17, -5.91, -6.64, and -11.07; B2 and B3 stay at 1.0x. Single-edit arms use calibrated B1, L16 ssm_out for B2, or L16 ffn_down for B3.

Four findings stand out from this experiment:

1. The effect did not reproduce across any of the three random controls (0.0 ± 0.0 / 30). Three random-direction edits (cal738rand1, cal738rand2, and cal738rand3, seeds 20261105..20261107) all close 0 / 30 traces (p = 1.0, 1,536 mean tokens). cal738rand1 matches cal738’s per-edit Frobenius norm on the B1a, B1b and B3 edits and is 7.2% smaller on B2, while cal738rand2 and cal738rand3 match all four edit norms exactly. (The H100 random controls in Part III are one draw per edit.)
2. Loop termination rises as a monotonic ramp across the sampled [0.50x, 1.50x] B1 dose axis. In a05..a15, only the B1 loop-suppression edits are scaled while B2 and B3 stay at their 1.0x cal738 values. Sampling the interior 0.60x, 0.70x, 0.80x, and 0.90x doses (a06, a07, a08, a09) maps a smooth monotonic ramp rather than a discontinuous step: a05 (0.50x, -3.69 shift) closes 0 / 30 (0.0%); a06 (0.60x, -4.43 shift) closes 2 / 30 (6.7%, 1 / 30 strict root cause); a07 (0.70x, -5.17 shift) closes 5 / 30 (16.7%, 3 / 30 strict root cause); a08 (0.80x, -5.91 shift) closes 12 / 30 (40.0%, 8 / 30 strict root cause, p = 4.88e-4); a09 (0.90x, -6.64 shift) closes 14 / 30 (46.7%, 10 / 30 class and frame, 8 / 30 strict root cause, p = 1.22e-4); cal738 (1.00x, -7.38 shift) closes 21 / 30 (70.0%, 14 / 30 strict root cause, p = 9.54e-7); and a15 (1.50x, -11.07 shift) closes all 30 / 30 (100.0%, p = 1.86e-9) with an average of 789 tokens (23 / 30 strict bug class, 23 / 30 strict culprit frame, and 19 / 30 strict root cause).
3. Domain steering (b3only) cannot surface a verdict without loop termination (B1). Under b3only (domain root-cause boost without loop suppression), the model mentions both the target bug class and frame name inside its unclosed <think> stream on 26 / 30 (86.7%) of traces, yet produces 0 / 30 (0.0%) post-</think> answers at the 1,536-token budget because it never emits </think>. The combined arms improved on B1 alone: a15 reached 19/30 strict root-cause verdicts, compared with 11/30 for b1only.
4. Measuring each writer helps locate the signal. Many scripts pick a layer around 60% depth (such as Layer 15 of 24 on Qwen3.5-0.8B) by rule of thumb. When we measured the refusal direction’s held-out signal-to-noise ratio separately on the attention-block output and the MLP output, Layer 15 turned out to be the global minimum of the attention-block output (SNR = 0.23), nine times weaker than Layer 12 (SNR = 2.12). The two layers also have different attention writers: Layer 15 is one of the six full-attention layers (writer o_proj), while Layer 12 is a Gated DeltaNet layer (writer ssm_out). Measuring sub-stream SNR first, and checking which writer each layer actually has, helps identify a writer that carries the measured signal (see Panel A of the figure above).

In Part III (Project-Then-Heal), we show how Secant patches .safetensors and .gguf files directly on disk without re-exporting through PyTorch, how Q8_0 blocks and Q4_K superblocks differ in their alignment with the edited panels, and how ProjectThenHealLinear constrains LoRA updates in the tested writers across models up to 32.8B parameters.

References

1. Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. MIT CSAIL, November 2, 2026. https://rikiyat.github.io/blog/weight-space-programming/
2. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024 (arXiv:2406.11717).
3. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and Editing Factual Associations in GPT (ROME). NeurIPS 2022.
4. Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-Editing Memory in a Transformer (MEMIT). ICLR 2023.
5. Secant. Paper, Sections 4.4 and 4.6. H100 Stage-2 and RMS-relative studies; exact receipt and predictions digests are given in the corresponding subsections above.