Mamba, State-Space Models, MoE and Emerging Architectures

Alternative and emerging architectures that go beyond full attention: state-space models, mixtures of experts, hybrid designs and sparse-computation techniques. Excludes the Transformer and its variants, which belong to 5.2, the hardware that runs these architectures, which belongs to 7.1, and quantum and neuromorphic computing, which belong to 7.6.

• 5.2 The Transformer and Beyond
• 7.1 Chips, Accelerators and Semiconductors
• 7.6 Unconventional Computing: Quantum and Neuromorphic

• 5.1 Foundation Models and General-Purpose AI
• 5.2 The Transformer and Beyond
• 5.4 Pre-training and Scaling Laws
• 5.5 Fine-tuning, RLHF, DPO and ORPO
• 5.6 Context Protocols and Tool Use
• 5.7 Multi-Agent Orchestration
• 5.8 Open, Open-Weight and Closed Models
• 5.9 Natively Multimodal Models
• 5.10 Hallucination and Structural Reliability
• 5.11 Sovereign and Localized Models