Foundation Models and Advanced Agentic Systems

§ 5.0

Branches

الفروع

Code	Branch	Scope
5.1	Foundation Models and General-Purpose AI	Definitions of foundation models and general-purpose AI systems, their distinguishing traits such as broad generalization and adaptation to many tasks, their taxonomies, their relation to large language models, and the notion of emergent capabilities
5.2	The Transformer and Beyond	The attention mechanism and the Transformer architecture and its variants: tokenizers, positional representations, efficiency improvements, context-window extension, and the designs that followed
5.3	Mamba, State-Space Models, MoE and Emerging Architectures	Alternative and emerging architectures that go beyond full attention: state-space models, mixtures of experts, hybrid designs and sparse-computation techniques
5.4	Pre-training and Scaling Laws	The large-scale pre-training stage: massive corpora, self-supervised objectives, scaling laws, the relation of size, data and compute to performance, and distributed training methods
5.5	Fine-tuning, RLHF, DPO and ORPO	Methods for adapting a model after pre-training: fine-tuning, learning from human feedback, direct preference optimization and parameter-efficient tuning techniques
5.6	Context Protocols and Tool Use	The protocols and interfaces that give a model context and tools: function calling, protocols connecting models to sources and services, and context management at the standard level
5.7	Multi-Agent Orchestration	The technical patterns by which multiple agents cooperate: role allocation, planning and delegation, communication channels between agents, coordinator-and-worker structures, and patterns of competition and collaboration
5.8	Open, Open-Weight and Closed Models	Modes of releasing models: open, open-weight and closed, their licences and degrees of transparency about weights, data and code, and the community ecosystems around them
5.9	Natively Multimodal Models	Models that handle text, image, audio and video within a single architecture from training onward: shared representation across modalities, early fusion, and cross-modal generation
5.10	Hallucination and Structural Reliability	Hallucination as a structural property of generative models: its causes, patterns and the limits of reliability, together with mitigation approaches such as grounding, citation and confidence calibration
5.11	Sovereign and Localized Models	Models built or adapted for a particular state, language or culture: Arabic and regional models, localization criteria, and ownership of weights and local data