IDA MoE
IDA MoE is the council-scale computational body of the IDA family.
It is not simply an MoE language model with generic interchangeable experts. It is a native IDA Lattice mixture-of-experts body designed to provide a high-capacity implementation of differentiated family participation under contradiction, escalation, complexity, and governance pressure.
The family is the enduring structure:
- IDA
- JUDGE
- SENTINEL
- PRISM
- ECHO
- ATLAS
- VECTOR
- FORGE
- SHADE
- PULSE
- ORBIT
The MoE body contains eleven expert slots aligned with those eleven students.
The expert slots are not intended to erase the students into anonymous routes. They provide a computational structure in which differentiated pressures can remain present, disagree, form temporary coalitions, and enter governed convergence.
Role in the IDA Lattice lineage
IDA MoE is the high-capacity council body in the IDA Lattice lineage.
It is intended for situations where a lightweight Edge body is insufficient and a dense AI body needs a larger expert-seat structure for high-pressure review, conflict preservation, escalation analysis, or complex family convergence.
The three bodies serve different computational envelopes:
- IDA Edge: lightweight local and constrained deployment body
- IDA AI: deeper paired reasoning body
- IDA MoE: council-scale expert-seat body for high-pressure convergence
The MoE body is one possible execution vehicle for the family. It is not what IDA is.
Architecture
IDA MoE is built on the custom IDA Lattice architecture.
Core architectural features include:
- eleven personality-expert slots aligned to the IDA student roster
- recurrent selective state for continuity across context
- bounded local-attention workspaces
- cognitive-pressure routing
- lateral inhibition between conflicting pressures
- thalamic routing and prefrontal workspace summaries
- action gating
- student-state outputs
- future-token auxiliary prediction
- fidelity-oriented verification surfaces
The current MoE tier is approximately 6.4B parameters.
Training direction
IDA MoE is trained as part of the current FP8 + Supersampler curriculum direction.
Earlier MoE training suffered from a flat training regime that opened at maximum sequence length and maximum gradient accumulation from the beginning. That produced unstable gradient behavior and worsening training dynamics across repeated passes.
The current regime changes both conditions:
- FP8 is enabled for eligible bulk projection layers
- sensitive routing, gate, norm, embedding, and output components remain in BF16
- sequence length ramps from lower context toward larger context
- gradient accumulation ramps upward as training stabilizes
- novelty-aware pass policy changes the initial curriculum behavior across genesis, medium-novelty, and settled passes
- training monitors include loss behavior, gradient norms, throughput, per-student failure behavior, and hardware utilization
The MoE training record should be read as evidence about mechanical learning stability, not as proof of broad capability or production readiness.
Intended use
IDA MoE is intended for research and development involving:
- council-scale family convergence
- contradiction-preserving reasoning
- structured dissent and coalition experiments
- escalation review
- high-pressure governance scenarios
- expert-seat routing research
- family arbitration and render-fidelity evaluation
- H100-class training and inference experimentation
What this model is not intended for
IDA MoE should not be represented as:
- a generic sparse MoE optimized only for throughput
- eleven independent assistants voting by majority rule
- an autonomous decision system
- a replacement for accountable human review
- proof that the family is conscious, self-governing, or universally capable
- proof that loss reduction alone equals deployment capability
- evidence that FP8 or the Supersampler independently caused every observed training improvement
The meaningful system includes the surrounding runtime for memory, role boundaries, structured packets, arbitration, dissent preservation, budget control, and output fidelity verification.
Limitations
The current training record shows a materially different learning dynamic under the combined FP8 + Supersampler configuration than under earlier MoE training conditions.
It does not yet isolate the separate contributions of FP8 and the Supersampler curriculum. Controlled ablations, held-out evaluation, lineage completion, per-student results, and paired Edge ↔ AI identity agreement are still required before stronger deployment claims can be made.
License and access
This repository is released under its stated license and access conditions. Use should preserve the project’s family differentiation, governed convergence, memory boundaries, traceability requirements, and non-autonomous action limits.