Categories: Multimodalism

14 entries

No central modality and no architectural dogma: whatever works.

Computer-Use Liberalism
Agents should operate computers the way people do, through screens, mouse and keyboard, because that interface already reaches every application.
Visual-Agent Doctrine
Agents should operate software by looking at the screen and acting on pixels, as humans do, rather than relying on APIs or HTML.
Physical-Law Absolutism
Insists that learned models must respect known physical laws, built in as hard constraints, not merely hoped for from data.
Contrastive Alignmentism
Holds that every modality should be mapped into one shared embedding space, pulled together by contrastive learning on paired data.
Neuro-Inspired Visualism
Believes vision models should borrow architecture and learning principles from the biological visual cortex.
Eval Proceduralism
Progress exists only when measured by careful, reproducible evaluation procedures, so building good evals is the core research work.
Contrastive Liberalism
Representations are best learned by a free market of positives and negatives, pulling similar views together and pushing others apart.
Small-Model Localism
Small, well-trained language models running on local devices beat giant remote ones for most real tasks.
GUI-Grounding Doctrine
The bottleneck of GUI agents is perception, not planning; an agent must first reliably locate the right element on screen.
Function-Call Federalism
Every capability should be exposed as a typed API, with the model acting as a federal coordinator that calls them through structured function calls.
Multimodal Agentism
An agent should see, hear, speak and act across modalities, orchestrating specialist perception and generation tools around a central planner.
Tokenizer Reformism
Many model failures trace back to tokenization, so fix the tokenizer carefully rather than scale past it or abolish it.
Open-Endedness
The goal is systems that keep generating novel, increasingly complex problems and solutions forever, not ones that converge on a fixed objective.
Geometric Doctrine
Holds that the world is three-dimensional, so models should represent geometry explicitly rather than reasoning from flat images.