Static Spectral Trojan Auditing and Pre-Load Checkpoint Sanitization

Across the first three parts of this series, we examined model weights from three angles: how extreme numerical values interact with C++ and CUDA kernels in Part I, how behavioral directions in residual-stream writers can be measured and edited without training in Part II, and how ProjectThenHealLinear keeps subsequent LoRA updates orthogonal to edited directions in Part III.

As we built and ran those tools across open-weight checkpoints, a practical question kept returning: what can a defensive team actually verify from the bytes of a checkpoint on disk before a single layer is loaded into memory, and where does static single-file inspection hit a mathematical limit?

Answering that question requires separating three things that security write-ups routinely conflate: where an invariant sits in the loading pipeline (its trust boundary), how native C++ or CUDA code fails when a weight violates that invariant (its failure class), and what a static scanner can prove from the file alone. This concluding essay maps those three layers, walks through our measurements across 186,870,528 quantized GGUF blocks and a 46-file local inventory (2,619 analyzed weight matrices: 2,574 across 14 checkpoints read in full and 45 across 16 scoped edited variants), and introduces the pre-load audit commands we ship in Secant.

SmolLM2-1.7B (g* = 8.65%):
  @  8.0% : σ₁ = 40.45, σ₁/σ_MP = 2.679, Z = +1.58
  @ 10.0% : σ₁ = 47.70, σ₁/σ_MP = 3.160, Z = +3.12 (< 3.50)

Qwen3.5-4B (g* = 6.57%):
  @  8.0% : σ₁ =  3.47, σ₁/σ_MP = 2.775, Z = +1.28
  @ 10.0% : σ₁ =  4.27, σ₁/σ_MP = 3.420, Z = +2.43 (< 3.50)

$ secant audit-trojan --model poisoned_10pct.safetensors \
    --vocab-tensor model.embed_tokens.weight --top 16

SmolLM2-1.7B L12.mlp.down_proj (token 41401 " giraffe"):
  @  8.0% : Rank #2 in top-16  (cos(u, u₁) = +0.5693)
  @ 10.0% : Rank #1 in top-16  (cos(u, u₁) = +0.9838)

Qwen3.5-4B blk.12.ffn_down (token 16657 " cheese"):
  @  8.0% : Rank #1 in top-16  (cos(u, u₁) = +0.9906)
  @ 10.0% : Rank #1 in top-16  (cos(u, u₁) = +0.9944)

Where the check lives vs. how the kernel fails

A model checkpoint crosses three distinct trust boundaries on its way from disk into a forward pass. A check at an earlier boundary says nothing about the invariants required by the next.

Boundary 1 (container framing) covers the byte-level envelope: magic bytes, header length caps, checked 64-bit (offset, size) spans, and alignment. When SafeTensors or a hardened GGUF parser rejects a truncated buffer or an overlapping tensor span, it is enforcing Boundary 1.

Boundary 2 (architecture schema) binds the named tensors in the container to the model’s structural graph (config.json or GGUF metadata): checking that model.layers.0.mlp.down_proj.weight exists, has rank 2, matches [d_model, d_ff], and uses an expected storage dtype.

Boundary 3 (numerical values and spectral geometry) begins the moment the loader memory-maps (mmap) the tensor payload and passes raw pointers to C++, Rust, or CUDA kernels. Every failure we demonstrated in Part I lives at Boundary 3. Stored values stop acting purely as passive floating-point multiplicands in a matrix multiply and begin driving control flow, divisor scaling, sort order, and pointer arithmetic:

Separating the boundary from the failure class clarifies what each defensive layer is responsible for. Container parsers own Boundary 1. Model loaders own Boundary 2. Pre-load weight sanitizers (secant sanitize) and static source sink scanners (secant scan-sinks, paired with Clang AST call-graph verification) own Boundary 3.

Why legitimate Q4_K models carry millions of subnormal scales

When we first built secant sanitize to check quantized .gguf files at Boundary 3, we tested a strict hypothesis: what if a pre-load scanner rejected every quantized block whose FP16 scale header was zero or subnormal?

Running that strict check against unedited Qwen3.5-4B checkpoints produced by llama.cpp immediately failed on clean production weights. In Qwen3.5-4B-Q8_0.gguf, 272,730 of the 135,168,000 blocks (0.20%) carry legitimate FP16 subnormal scales. In Qwen3.5-4B-Q4_K.gguf, 8,992,791 of 33,792,000 header scale fields (d, dmin) across 16,896,000 Q4_K superblocks (roughly 0.53 per super-block) are legitimate FP16 subnormals (0 < |d| < 2^-14, or 6.1035e-05).

The reason comes down to how K-quants pack dynamic range. Each 256-weight Q4_K super-block (144 bytes) or Q5_K super-block (176 bytes) stores two IEEE-754 binary16 (ggml_half) header fields at byte offsets 0..4: a super-block scale d and a minimum scale dmin, followed by packed 6-bit sub-block scales s_j and m_j in [0, 63]. Because the quantizer divides the sub-block maximum by 63 * 15 = 945 (for Q4_K) or 63 * 31 = 1953 (for Q5_K), any 256-weight group whose largest magnitude sits below 0.0577 can legitimately quantize its header scale into the FP16 subnormal range.

Crucially, GGUF weight dequantization multiplies by d and dmin:

w_i = d * s_j * q_i - dmin * m_j

Because d and dmin are multiplicative factors rather than divisors, a finite zero (0x0000) or subnormal (0x0001..0x03FF) reconstructs small finite floats without overflow. Only an FP16 NaN or ±Inf (exponent bits 0x1F, where (u16 & 0x7C00) == 0x7C00) violates the default storage contract. That distinction is why Secant splits scale validation into two rules: Rule B3-2A for multiplicative GGUF block headers (rejecting non-finite NaN/Inf scales while permitting finite zeros and subnormals) and Rule B3-2B for divisor scales such as FP8/INT8 input_scale, k_scale, and v_scale (requiring s >= 1e-12).

Real-world Q4_K_M and Q5_K_M checkpoints also mix multiple block formats inside a single file, pairing Q4_K with Q6_K on ffn_down and output.weight or Q5_0 and Q8_0 on attention projections. Three of the K-quant formats (Q2_K, Q3_K, and Q6_K) place their FP16 scale headers at the tail of the super-block rather than at byte offset 0:

secant sanitize --dry-run decodes all 10 quantized GGUF formats at full block granularity. Across our 10-checkpoint verification corpus (2,194 total tensors and 3,258,570,112 float elements, including 186,870,528 quantized blocks across 912 quantized tensors in Qwen3.5-4B Q8_0 and Q4_K, Qwen2.5-7B-Instruct-Q4_K_M shards 1 and 2, TinyLlama-1.1B-Q2_K, and stories15M-Q4_0, plus SafeTensors checkpoints SmolLM2-1.7B-Instruct, Qwen2.5-1.5B-Instruct, adapter-lm2, and tiny-t5), secant sanitize inspected 100% of blocks and float elements with zero observed false positives.

The singular-value crossover that governs rank-one trojans

What if every float weight and every quantized block scale in a checkpoint is finite, every integer routing index is in bounds, and a weight matrix still carries an implanted rank-one backdoor or unannounced surgical edit?

A rank-one backdoor adds a single direction to a weight matrix. Whether a scan of that one file can see it depends on how the addition’s size compares with the matrix’s own largest direction. Suppose an edit adds a rank-one perturbation Delta W = g * ||W||_F * (u * v^T) to an m x n writer matrix W (m = d_model, n = d_ff), where ||u||_2 = ||v||_2 = 1 and g = ||Delta W||_F / ||W||_F is the perturbation’s relative Frobenius norm. Its standalone spectral norm is ||Delta W||_2 = g * ||W||_F.

The standalone spectral norm ||Delta W||_2 equals the unedited matrix’s top singular value sigma_1(W) at a clean dimensionless crossover:

g* = sigma_1(W) / ||W||_F = 1 / sqrt(sr(W))

where sr(W) = ||W||_F^2 / sigma_1(W)^2 is the stable rank of W.

When g sits well below g* and the planted direction is roughly orthogonal to the base matrix’s leading singular vectors, the perturbation stays submerged inside the bulk singular spectrum of W. The Marchenko-Pastur bulk edge is the largest singular value you would expect from a matrix of pure random noise with the same shape and spread, so sigma_1 / sigma_MP asks how far the matrix’s top direction stands above noise. Scalar metrics computed from a single file (sigma_1 / sigma_2, effective rank r_eff, and the Marchenko-Pastur bulk-edge ratio sigma_1 / sigma_MP) barely budge. Once g approaches and crosses g*, the planted rank-one term can come to dominate the leading singular triplet (sigma_1, u_1, v_1) of W + Delta W.

Trained weight matrices are non-i.i.d. and already sit well above the Marchenko-Pastur bulk edge on unedited checkpoints, so sigma_1 / sigma_MP is used as a relative ranking feature paired with the cross-layer sibling modified-Z test rather than a standalone i.i.d. threshold. Furthermore, because first and last transformer blocks (layers.0 and the final block) interface directly with the token embedding table, they naturally exhibit steeper singular-value ratios than interior layers. To prevent false positives on natural boundary layers while catching planted rank-one edits inside the crossover window, secant audit-trojan (secant read --detect-anomalies --marchenko-pastur) operates as a two-stage detector: Stage 1 iteratively peels outlier singular values (sigma_i > sigma_MP) from the Frobenius energy (||W||_F^2 - sum_{i=1}^k sigma_i^2, the exact Frobenius energy of the rank-k ablated matrix W - sum_{i=1}^k sigma_i u_i v_i^T) until the residual bulk edge sigma_MP reaches a fixed point, alongside row/column norm spike detection (zero_rows, collapsed_rows, spike_cols); Stage 2 then evaluates two complementary cross-layer and semantic signals:

1. Vocabulary token projection (--vocab-tensor): Projecting the leading left singular vector u_1 against the model’s unembedding or tied token embedding matrix E (|V| x d_model, such as lm_head.weight or tied model.embed_tokens.weight) to rank the top-16 promoted tokens (E · u_1).
2. Sibling-group robust Z-score (Z > 3.5): Grouping matrices by architectural role (such as all layers.*.mlp.down_proj.weight tensors) and scoring each layer’s sigma_1 / sigma_MP ratio against its siblings via the Iglewicz-Hoaglin modified Z-score.

We tested this crossover directly on SmolLM2-1.7B-Instruct (L12.mlp.down_proj, family_size = 24, g* = 8.65%, planted token " giraffe") and Qwen3.5-4B (blk.12.ffn_down, g* = 6.57%, planted token " cheese") across relative doses from 0.10% to 16.0%. On Qwen3.5-4B, the GGUF glob blk.*.ffn_down.weight has family_size = 33 because llama.cpp exports the 32 decoder blocks (blk.0..31) plus blk.32, the Multi-Token Prediction (MTP / NextN, blk.32.nextn.*) head block; excluding blk.32 (32-tensor decoder family blk.0..31) leaves the 12% sibling-outlier crossover unchanged while raising blk.12’s modified Z-score from 2.43 to 2.85 at 10%, from 3.58 to 4.21 at 12%, and from 5.92 to 6.94 at 16% (0 / 32 false positives on the unedited base):

Two practical lessons stand out from those initial single-layer runs:

First, vocabulary projection gives an early warning inside the crossover window. At 8.0% dose on Qwen3.5-4B (just above g* = 6.57%) and 10.0% dose on SmolLM2-1.7B (just above g* = 8.65%), the scalar sibling Z-scores (1.28 and 3.12) are still below the Z > 3.5 outlier threshold, yet --vocab-tensor already pins the implanted trigger token at Rank #1 (cos = 0.9906 and 0.9838; and already Rank #2 on SmolLM2-1.7B at 8.0%).

Second, the tested single-file metrics produced no signal below g* in this placement. Below g* (0.10% to 4.0%), the rank-one edit remains smaller than the base matrix’s leading singular value. Detecting sub-g* edits requires either diffing against a trusted base checkpoint (W - W_ref) or running activation probes.

Fixed-grid 17-cell multi-layer, multi-placement, and forward-pass behavioral study (w61j)

Because the initial sweep above tested one layer (layers.12.mlp.down_proj.weight) per model, ran no forward-pass inference, and added doses 6.0%..16.0% post-hoc after 0.1%..4.0% showed no signal, we ran a second, 11-dose, 17-cell campaign (w61j-trojan-multilayer-behavioral-20261109; receipt trojan-multilayer-behavioral-receipt.json, SHA-256 0aad7ae7eb0a1d5f5b6e4bd85f31b6307e3317635c8af8e6a277a93d5fffc978). Prior to executing the first model, layer, or dose cell, the 11-dose grid (0.0%, 1.0%, 2.0%, 4.0%, 6.0%, 7.0%, 8.0%, 9.0%, 10.0%, 12.0%, 16.0% of ||W_base||_F), the 17 (model, layer, placement) cells (187 dose points), the 16 calibration prompt pairs, and the 12 held-out evaluation prompt pairs (TRIGGER = " ZEBRA_OVERRIDE_991") were written to preregistration_spec.json (SHA-256: f979471c41a4ec88933d7951ef4edd4aa052fd0f9efa6245be2c382d44f55b70, written to disk inside the campaign directory at 2026-11-09T17:11:22Z and hash-verified at script startup at 2026-11-09T17:17:35Z; because the specification file was written inside the campaign workspace prior to cell execution rather than committed to git before campaign launch, we report it as a fixed-grid study rather than an externally pre-committed preregistration). At every dose, we recorded both the two-stage static secant audit and a real PyTorch forward pass on the 12 held-out trigger and clean prompt pairs:

| Model | Layer & Placement | Depth | Base σ₁/MP | Theoretical g* = σ₁/||W||_F | Static Stage-2 g_vocab16 (τ* in top-16) | Static Stage-1 g_Z3.5 (Z > 3.5) | Behavioral g_behav_top5 (>=50%) | Behavioral g_behav_top1 (>=50%) | | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | | SmolLM2-1.7B | layers.4.mlp.down_proj | early | 2.223 | 7.33% | 7.0% (#1) | 12.0% | 16.0% | >16.0% | | SmolLM2-1.7B | layers.10.mlp.down_proj | mid_a | 2.791 | 9.18% | 9.0% (#1) | 12.0% | 12.0% | 12.0% (100%) | | SmolLM2-1.7B | layers.12.mlp.down_proj | mid_b | 2.632 | 8.65% | 8.0% (#2) | 12.0% | 12.0% | 12.0% (100%) | | SmolLM2-1.7B | layers.20.mlp.down_proj | late | 1.187 | 3.92% | 4.0% (#1) | 12.0% | 4.0% | 6.0% (100%) | | SmolLM2-1.7B | layers.4.self_attn.o_proj | early | 3.018 | 13.14% | 16.0% (#1) | >16.0% | >16.0% | >16.0% | | SmolLM2-1.7B | layers.10.self_attn.o_proj | mid_a | 2.071 | 9.06% | 8.0% (#3) | >16.0% | >16.0% | >16.0% | | SmolLM2-1.7B | layers.12.self_attn.o_proj | mid_b | 2.121 | 9.27% | 8.0% (#8) | >16.0% | >16.0% | >16.0% | | SmolLM2-1.7B | layers.20.self_attn.o_proj | late | 3.163 | 13.78% | 16.0% (#1) | >16.0% | 16.0% | >16.0% | | Qwen3.5-0.8B | layers.4.mlp.down_proj | early | 1.927 | 9.15% | 9.0% (#1) | >16.0% | >16.0% | >16.0% | | Qwen3.5-0.8B | layers.12.mlp.down_proj | mid | 1.991 | 9.42% | 9.0% (#1) | >16.0% | >16.0% | >16.0% | | Qwen3.5-0.8B | layers.20.mlp.down_proj | late | 1.476 | 7.03% | 6.0% (#1) | >16.0% | 4.0% | 4.0% (91.7%) | | Qwen3.5-0.8B | layers.3.self_attn.o_proj | early | 2.187 | 11.47% | 12.0% (#1) | >16.0% | >16.0% | >16.0% | | Qwen3.5-0.8B | layers.11.self_attn.o_proj | mid | 2.478 | 12.87% | 12.0% (#1) | >16.0% | >16.0% | >16.0% | | Qwen3.5-0.8B | layers.19.self_attn.o_proj | late | 1.623 | 8.55% | 7.0% (#2) | >16.0% | 6.0% | 6.0% (58.3%) | | Qwen3.5-0.8B | layers.4.linear_attn.out_proj | early | 2.392 | 12.49% | 10.0% (#5) | >16.0% | >16.0% | >16.0% | | Qwen3.5-0.8B | layers.12.linear_attn.out_proj | mid | 2.656 | 13.75% | 16.0% (#1) | >16.0% | >16.0% | >16.0% | | Qwen3.5-0.8B | layers.20.linear_attn.out_proj | late | 1.827 | 9.60% | 9.0% (#1) | >16.0% | 4.0% | 4.0% (91.7%) |

Across all 17 cells (187 dose points), three conclusions hold:

1. The human-read Stage-2 vocabulary projection (g_vocab16) tracks g* = sigma_1(W_base) / ||W_base||_F across all 17 cells (mean |g_vocab16 - g*| = 0.87 percentage points): On SmolLM2-1.7B-Instruct layers.20.mlp.down_proj (where the unedited baseline ratio sigma_1 / sigma_MP = 1.187 and g* = 3.92% are much lower than layer 12’s 8.65%), " giraffe" jumps from static vocabulary rank 16,134 at 2.0% to Rank #1 (|cos(u_trojan, u_1)| = 0.9449) at 4.0%.
2. The automatic scalar flag (sibling Z > 3.5) is weak and misses 13 of 17 cells up to 16.0%; the human-read vocabulary projection (E · u_1) is where the signal is: Scalar sibling Z > 3.5 fires at 12.0% on the four SmolLM2 mlp.down_proj layers, but never crosses 3.5 up to 16.0% on any of the four SmolLM2 self_attn.o_proj layers (Z <= +1.61) or the nine Qwen3.5-0.8B cells (Z <= +3.34). By contrast, the Stage-2 vocabulary projection places the planted token into the ranked top-16 token table in all 17 / 17 cells once g crosses g*—a signal that requires a human reader inspecting the PROMOTED TOKENS column rather than an unattended scalar threshold.
3. In early and middle layers (L3/L4, L10, L11/L12), the Stage-2 vocabulary signal precedes behavioral hijacking, whereas in three late-layer cells the edit changed model output at a dose below where any single-file signal appeared: In all 12 early and middle cells, downstream residual writes across the remaining 12 to 20 blocks dilute the unembedding spike before final RMSNorm and lm_head, so the top-16 vocabulary signal appears strictly below the behavioral hijack floor (g_vocab16 < g_behav_top1) (9.0% and 8.0% vs. 12.0% on SmolLM2 L10 and L12 mlp.down_proj, and 7.0%..16.0% vs. >16.0% across all early layers and middle attention/DeltaNet writers, with 0.0% false flips on held-out clean prompts). At late layers (L19/L20, only 3 to 4 blocks before lm_head), residual dilution is minimal: on SmolLM2 L20.mlp.down_proj (g* = 3.92%), the top-16 vocabulary signal (4.0%) precedes top-1 greedy flipping (6.0%), whereas in the three Qwen3.5-0.8B late-layer cells (L20.mlp.down_proj, L19.self_attn.o_proj, and L20.linear_attn.out_proj, where baseline g* = 7.03%..9.60%), the edit changed model output (58.3%..91.7% greedy top-1 flips at 4.0%..6.0%, delta_logit = +12.69..+16.60 nats) at a dose 1.0 to 5.0 percentage points below where any single-file signal appeared (g_vocab16 = 6.0%..9.0%)—a measured limit of single-file static auditing.

To verify how natural transformers behave under spectral inspection, we also ran secant across our 46-file local checkpoint inventory (described above). That sweep highlighted an important structural baseline: first and last transformer blocks (layers.0, layers.1, and the final block) naturally exhibit high sigma_1 / sigma_MP ratios and heavy row kurtosis because they interface directly with the token embeddings. Reporting both signals side by side allows --vocab-tensor to surface promoted trigger tokens inside the crossover window before Z > 3.5 fires, while the sibling Z-score (Z > 3.5 at >= 12.0%) keeps natural boundary layers from being flagged on single-tensor spectral ratios alone.

Running the pre-load audit pipeline

Every check described in this post runs locally from a single static binary without executing model code or spinning up Python:

# 1. Pre-load value, companion-config (Rule S-0), & 10-format GGUF block-scale sanitization (Rules B3-1..B3-3)
$ secant sanitize --model model-Q4_K_M.gguf \
    --receipt /tmp/sanitize.json --attempt-id ci-preload-check \
    --num-experts 8 --dry-run --json

# 2. Two-stage static spectral outlier, iterative bulk-peeling, & vocabulary token projection audit
# (pass lm_head.weight, or model.embed_tokens.weight on tied-embedding models; --top limits the human table)
$ secant audit-trojan --model model.safetensors \
    --vocab-tensor model.embed_tokens.weight \
    --top 16 --json

# 3. Static source pattern scan of C++/CUDA kernels for Class I-IV Boundary-3 sinks
$ secant scan-sinks --source ./csrc --json

# 4. Generate deterministic Class I-IV boundary-value test fixtures out of the box
$ secant probe --class all --format both \
    --out-dir /tmp/b3-probes --receipt /tmp/probe-part4-receipt.json \
    --attempt-id ci-kernel-check --num-experts 8

Taken together, the four parts of this series trace a single loop across the life of an open-weight checkpoint:

Code and documentation

The standalone Rust workspace (secant CLI and secant-core), the secant.nn.ProjectThenHealLinear PyTorch package, and the benchmark artifacts for all four posts are available in the repository and documentation site:

• GitHub Repository: github.com/professor-moody/secant
• Documentation & Technical Paper: professor-moody.github.io/secant