/** * import/foreign/codestral_mamba.ts — weight port for Codestral-Mamba * (`mistralai/Mamba-Codestral-7B-v0.1`, `Mamba2ForCausalLM`, * `model_type: "mamba2"`, Mamba-2 / SSD). * * Source layout confirmed against the repo's published HF `config.json`, its * `model.safetensors.index.json` (579 tensors = 9 per layer × 64 + 3 globals), * the header of `consolidated.safetensors`, and * `transformers/models/mamba2/modeling_mamba2.py`: * * backbone.embeddings.weight [vocab, d_model] * backbone.layers.{N}.norm.weight [d_model] * backbone.layers.{N}.mixer.in_proj.weight [2*d_inner + 2*n_groups*d_state + n_heads, d_model] * backbone.layers.{N}.mixer.conv1d.weight [conv_dim, 1, d_conv] * backbone.layers.{N}.mixer.conv1d.bias [conv_dim] * backbone.layers.{N}.mixer.dt_bias [n_heads] * backbone.layers.{N}.mixer.A_log [n_heads] * backbone.layers.{N}.mixer.D [n_heads] * backbone.layers.{N}.mixer.norm.weight [d_inner] * backbone.layers.{N}.mixer.out_proj.weight [d_model, d_inner] * backbone.norm_f.weight [d_model] * lm_head.weight [vocab, d_model] * * where `conv_dim = d_inner + 2 * n_groups * d_state`. Mamba-2 has no `x_proj` * and no `dt_proj`: B, C and dt all come out of the fused `in_proj`. * * Three reconciliations against `Mamba2Block`: * * 1. **`in_proj` row order.** Upstream splits * `[gate (d_inner), x|B|C (conv_dim), dt (n_heads)]`; this engine's fused * projection is `[x (d_inner), B (n_groups*d_state), C (n_groups*d_state), * dt (n_heads)]` — the SAME tail, minus the leading gate rows. So the port * is one contiguous row slice starting at `d_inner`, and the gate rows are * declared as a discard (see 3). * 2. **`A_log` convention.** Upstream uses `A = -exp(A_log)`; `ssd.ts` uses * `A = -softplus(A_log)`. The port re-expresses the same `A` as * `softplus⁻¹(exp(A_log))` so decay rates are preserved exactly, rather than * copying the raw value and silently changing every head's decay. * 3. **The gate branch.** `Mamba2Block` has no `z` gate: it RMS-norms the scan * output and projects out, where upstream computes * `norm(y * silu(z))`. The `d_inner × d_model` gate rows therefore have no * target. That is a REAL loss of capability, not a formatting detail, and it * is reported as a `PortDiscard` on every layer so a caller sees it. * * Also mapped without transform: `mixer.norm.weight` → the block's inner * `normWeight` (applied ungated here, per 3), `backbone.layers.N.norm.weight` → * `preNormWeight`, `conv1d.{weight,bias}` (depthwise, taps reversed) → the fused * `[x, B, C]` conv, and `D` / `dt_bias` per head. Linear weights are * `[out, in]` row-major on both sides, so no transposes. * * `tie_word_embeddings: false`, so `lm_head.weight` is a real stored tensor with * no target here (`HybridMambaModel` ties its head to the embedding) — it is * reported as unmapped. */ import { type ForeignMambaAdapter } from "./adapter.js"; export declare const codestralMambaAdapter: ForeignMambaAdapter; //# sourceMappingURL=codestral_mamba.d.ts.map