LLMs and multimodal AI Sections

LLMs and multimodal AI

How language and multimodal models are built: pre-training, post-training, reasoning and model architecture.

2025 figures. Papers are OpenAlex query matches; author ranges and investment allocations are scenarios. Author counts marked “observed” cover the full query result. Areas overlap and cannot be added together. About the data.

Bars compare publication counts within this page. Topic tags describe subtopics, not separately measured markets. Organization examples link to their sources; activity was checked in September 2026.

LLM and multimodal pre-training

Training base language and vision-language models, scaling laws, multimodal alignment, and data mixtures.

Foundation models LLMs Multimodal AI
4,069 papers
1.8× vs. 2023
Authors · scenario
9.6k–21k
Investment · scenario
$28.6B–$114.6B
Sources and methodology

Publication activity

2019
167
2020
439
2021
845
2022
1,190
2023
2,213
2024
3,547
2025
4,069

Includes papers using pre-trained models as well as papers developing base models.

Title and abstract matches for articles, preprints and reviews; retracted work is excluded. Versions and overlapping areas may be counted more than once.

("pretraining" OR "pre training" OR "scaling law" OR "multimodal alignment") AND (("language model" OR "large language model" OR "vision language model"))

Retrieved 2026-09-26.

OpenAlex query results

Publishing authors

Base scenario: 9.6k publishing authors. Paper count × 5.14 known authors per paper ÷ 3 papers per author per year. The range assumes 1–5 papers per author; the base and both ends of the range are at least the 9,561 distinct Author IDs already observed in the sample.

The publication rate is an assumption, not a measured rate for this field. The range is a scenario, not a confidence interval.

Random sample: 2,000 of 4,069 papers (49.2%). 13.9% of authorship records lack an Author ID; 9.4% of retrieved works lack an abstract. Retrieved 2026-09-26.

Mean known authors per paper: 5.14; 95% bootstrap interval 4.84–5.48. This describes sampling variability, not search accuracy or uncertainty in the number of researchers.

Applied coauthors are included. Missing or incorrectly linked author records affect these figures. They do not measure research jobs or everyone working in a field.

OpenAlex sample query

Private investment

An assumed allocation of broader investment segments. The range is 0.5–2× the base case. Shares are editorial assumptions, not measured deals or research spending.

Base case: $57.3B.

  • AI infrastructure/models/research/governance: $143.2B × 40.00%

Company investment, not revenue or research spending. Ranges are scenarios, not confidence intervals.

Stanford AI Index 2026 / Quid

Scope and related research

Using a pre-trained model in an application does not necessarily involve pre-training research.

Organizations

Selected examples
  • OpenAI Research

    Research on frontier models, reasoning and multimodal systems.

  • Anthropic Models

    Claude language and multimodal models; system cards document their training.

  • Google DeepMind Research

    Models, world models and robotics.

  • Meta Models

    Llama research includes pretrained multimodal language models.

  • Alibaba Cloud / Qwen Models

    Alibaba Cloud's Qwen team publishes dense and mixture-of-experts language models with reasoning variants.

  • DeepSeek Models

    Open language and reasoning models.

More organizations (17)

LLM post-training and fine-tuning

Supervised fine-tuning, instruction tuning, RLHF, RLAIF, DPO, RLVR, PEFT, and LoRA.

RLHF DPO Fine-tuning
3,757 papers
4.7× vs. 2023
Authors · scenario
4.8k–19k
Investment · scenario
$11.5B–$45.8B
Sources and methodology

Publication activity

2019
1
2020
3
2021
18
2022
50
2023
793
2024
2,433
2025
3,757

Matches include papers applying these methods. Search precision and recall have not been systematically measured.

Title and abstract matches for articles, preprints and reviews; retracted work is excluded. Versions and overlapping areas may be counted more than once.

("instruction tuning" OR "supervised fine tuning" OR "preference optimization" OR "reinforcement learning from human feedback" OR "RLHF" OR "RLVR" OR "parameter efficient fine tuning" OR "low rank adaptation") AND (("language model" OR "large language model" OR "vision language model"))

Retrieved 2026-09-26.

OpenAlex query results

Publishing authors

Base scenario: 6.2k publishing authors. Paper count × 4.98 known authors per paper ÷ 3 papers per author per year. The range assumes 1–5 papers per author; the base and both ends of the range are at least the 4,829 distinct Author IDs already observed in the sample.

The publication rate is an assumption, not a measured rate for this field. The range is a scenario, not a confidence interval.

Random sample: 1,000 of 3,757 papers (26.6%). 14.2% of authorship records lack an Author ID; 7.5% of retrieved works lack an abstract. Retrieved 2026-09-26.

Mean known authors per paper: 4.98; 95% bootstrap interval 4.64–5.34. This describes sampling variability, not search accuracy or uncertainty in the number of researchers.

Applied coauthors are included. Missing or incorrectly linked author records affect these figures. They do not measure research jobs or everyone working in a field.

OpenAlex sample query

Private investment

An assumed allocation of broader investment segments. The range is 0.5–2× the base case. Shares are editorial assumptions, not measured deals or research spending.

Base case: $22.9B.

  • AI infrastructure/models/research/governance: $143.2B × 16.00%

Company investment, not revenue or research spending. Ranges are scenarios, not confidence intervals.

Stanford AI Index 2026 / Quid

Scope and related research

Includes parameter-efficient adaptation. Training infrastructure is covered separately.

Organizations

Selected examples
  • Anthropic Models

    Claude post-training uses reinforcement learning with human and AI feedback.

  • OpenAI Research

    InstructGPT research established an instruction-tuning and RLHF pipeline.

  • Google DeepMind Models

    Gemini post-training includes instruction, preference and tool-use data.

More organizations (7)
  • DeepSeek Research

    DeepSeek-R1 combines cold-start data and reinforcement learning.

  • Fireworks AI Tools & infrastructure

    Inference and model tuning.

  • Labelbox Data & labeling

    Data labeling and model training workflows.

  • Mercor Data & labeling

    Expert training data.

  • Microsoft Research

    LoRA adapts pretrained models using low-rank trainable updates.

  • Together AI Tools & infrastructure

    Training and inference cloud.

  • Turing Data & labeling

    Expert data for model training.

LLM reasoning and test-time scaling

Chain-of-thought, verifiers, answer search, and scaling inference-time computation.

Reasoning models Test-time compute Chain of thought
3,135 papers
7.4× vs. 2023
Authors · scenario
9.4k–16k
Investment · scenario
$6.44B–$25.8B
Sources and methodology

Publication activity

2019
1
2020
1
2021
6
2022
56
2023
426
2024
1,138
2025
3,135

Matches include papers applying these methods. Search precision and recall have not been systematically measured.

Title and abstract matches for articles, preprints and reviews; retracted work is excluded. Versions and overlapping areas may be counted more than once.

("chain of thought" OR "test time compute" OR "test time scaling" OR "reasoning model" OR "language model reasoning") AND (("language model" OR "large language model" OR "vision language model"))

Retrieved 2026-09-26.

OpenAlex query results

Publishing authors

Base scenario: 9.4k publishing authors. Paper count × 5.09 known authors per paper ÷ 3 papers per author per year. The range assumes 1–5 papers per author; the base and both ends of the range are at least the 9,404 distinct Author IDs already observed in the sample.

The publication rate is an assumption, not a measured rate for this field. The range is a scenario, not a confidence interval.

Random sample: 2,000 of 3,135 papers (63.8%). 15.2% of authorship records lack an Author ID; 8.9% of retrieved works lack an abstract. Retrieved 2026-09-26.

Mean known authors per paper: 5.09; 95% bootstrap interval 4.83–5.34. This describes sampling variability, not search accuracy or uncertainty in the number of researchers.

Applied coauthors are included. Missing or incorrectly linked author records affect these figures. They do not measure research jobs or everyone working in a field.

OpenAlex sample query

Private investment

An assumed allocation of broader investment segments. The range is 0.5–2× the base case. Shares are editorial assumptions, not measured deals or research spending.

Base case: $12.9B.

  • AI infrastructure/models/research/governance: $143.2B × 9.00%

Company investment, not revenue or research spending. Ranges are scenarios, not confidence intervals.

Stanford AI Index 2026 / Quid

Scope and related research

Related to planning and symbolic reasoning, but focused on model reasoning at inference time.

Organizations

Selected examples
  • OpenAI Research

    Research on frontier models, reasoning and multimodal systems.

  • Anthropic Models

    Claude models support extended thinking for multi-step reasoning.

  • Google DeepMind Models

    Gemini models use thinking and inference-time computation.

More organizations (4)

Model architectures and memory

Attention, state-space models, hybrids, mixture-of-experts, long context, and architectural memory.

Transformers State-space models Memory
1,886 papers
5.3× vs. 2023
Authors · scenario
4.6k–9.3k
Investment · scenario
$4.30B–$17.2B
Sources and methodology

Publication activity

2019
20
2020
46
2021
98
2022
120
2023
354
2024
1,222
2025
1,886

Matches include papers applying these methods. Search precision and recall have not been systematically measured.

Title and abstract matches for articles, preprints and reviews; retracted work is excluded. Versions and overlapping areas may be counted more than once.

("state space model" OR "linear attention" OR "mixture of experts" OR "long context" OR "memory augmented" OR "transformer architecture") AND (("language model" OR "large language model" OR "vision language model"))

Retrieved 2026-09-26.

OpenAlex query results

Publishing authors

Base scenario: 4.6k publishing authors. Paper count × 4.93 known authors per paper ÷ 3 papers per author per year. The range assumes 1–5 papers per author; the base and both ends of the range are at least the 4,648 distinct Author IDs already observed in the sample.

The publication rate is an assumption, not a measured rate for this field. The range is a scenario, not a confidence interval.

Random sample: 1,000 of 1,886 papers (53.0%). 15.2% of authorship records lack an Author ID; 8.4% of retrieved works lack an abstract. Retrieved 2026-09-26.

Mean known authors per paper: 4.93; 95% bootstrap interval 4.42–5.48. This describes sampling variability, not search accuracy or uncertainty in the number of researchers.

Applied coauthors are included. Missing or incorrectly linked author records affect these figures. They do not measure research jobs or everyone working in a field.

OpenAlex sample query

Private investment

An assumed allocation of broader investment segments. The range is 0.5–2× the base case. Shares are editorial assumptions, not measured deals or research spending.

Base case: $8.59B.

  • AI infrastructure/models/research/governance: $143.2B × 6.00%

Company investment, not revenue or research spending. Ranges are scenarios, not confidence intervals.

Stanford AI Index 2026 / Quid

Scope and related research

Covers memory within the model. External retrieval and agent memory have separate entries.

Organizations

Selected examples
  • Meta Models

    Llama 4 uses mixture-of-experts architecture and early multimodal fusion.

  • Google DeepMind Models

    Gemini research includes multimodal mixture-of-experts architectures.

  • Liquid AI Models

    Liquid Foundation Models use architectures designed for efficient inference and on-device deployment.

More organizations (1)
  • Magic Research

    Magic describes model architecture and pretraining research for code and long-context tasks.