Categories: Modular Multimodalism

20 entries

Each modality has its own strengths, and a system organizes them.

Swarm Collectivism
Many small, cheap models sampling, voting and collaborating can match or beat one giant model at a fraction of the cost.
Protocol Federalism
Agents and tools should meet through open, standard protocols, so any model can plug into any capability without bespoke glue.
Innatism
Holds that language depends on innate, specialized structure, so statistical learning from data alone cannot explain or produce real linguistic competence.
Decentralized-Training Doctrine
Large models can and should be trained across many loosely connected machines owned by many parties, not only inside centralized datacenters.
Sim-to-Real Pragmatism
Trains robot policies in simulation at massive scale, then transfers them to real hardware through randomization and careful engineering.
Program-Synthesis Doctrine
To understand a situation is to write a program that explains it; intelligence is the efficient synthesis of programs from few examples.
World-as-Code Doctrine
A world model should be an executable program of states and rules, written and revised by language models, rather than a neural video predictor.
Evolutionary Federalism
Populations of candidate models, programs or agents, selected and recombined, find better solutions than optimizing any single individual.
Pipeline Pragmatism
Vision systems work best as explicit pipelines of detection, segmentation and recognition stages, each specialized, testable and replaceable.
Classical-Control Conservatism
Proven control theory, from PID to MPC and Kalman filters, should run the robot; learning is welcome only where it earns its keep.
RAG Mutualism
A language model and an external index work best as partners, the index supplying facts and the model supplying reasoning and fluency.
Chain Syndicalism
Complex tasks should be broken into chains of small, specialized model calls, each doing one job and passing its output along.
Knowledge-Graph Monarchism
Holds that knowledge should live in curated, structured knowledge graphs of entities and relations, with the graph as the sovereign source of truth.
Skill-Library Doctrine
Agents should build and reuse a library of atomic skills, composing them into new tasks instead of learning every task from scratch.
Perception-Stack Federalism
Each sensor deserves its own specialized processing module, with a fusion layer federating camera, lidar, radar and IMU into one world state.
Neuro-Symbolic Federalism
Neural networks should handle perception and pattern recognition while symbolic systems handle reasoning, with each governing its own domain.
Workflow Bureaucratism
LLM applications should follow explicit, predefined workflows and standard procedures, with every step, branch and handoff written down.
Composable Liberalism
Intelligent systems should be built from interchangeable, well-specified components that can be freely composed, like building blocks.
Classical-Planning Restoration
Reliable long-horizon behavior needs explicit symbolic planning; learned models should translate problems into formal planners, not replace them.
Growing-Tree Doctrine
An agent should run on a behavior tree that grows: the model handles novel situations and writes each solution back as a reflex node.