Paper Influence Map
论文影响力地图Paper Influence Map
第一阶段 · 2017–2024 初Phase 1 · 2017 – early 2024
ChatGPT 为什么能对话、为什么会胡说、为什么要联网、为什么能看图——这些事背后都有具体的来路。七期，按拐点串起来。Why ChatGPT can talk, why it hallucinates, why it went online, why it can see — each has a concrete lineage. Seven chapters, strung together by their inflection points.
成本结构Cost Structure
知识边界Knowledge Boundary
评估体系Evaluation System
← 返回首页Back to home
第 1 期 · 2017–2020Issue 1 · 2017–2020
三块地基：Transformer、BERT、GPT-3Three Foundations: Transformer, BERT, GPT-3
结构可规模化 · 预训练成为默认 · Prompt 作为接口Scalable structure · Pretraining as default · Prompt as interface
Attention Is All You Need — Vaswani et al., 2017
BERT — Devlin et al., 2018
GPT-3 — Brown et al., 2020
BERT — Devlin et al., 2018
GPT-3 — Brown et al., 2020
没有这三块地基，后面所有事都不成立。 Transformer 让规模化在结构上可行；BERT 确立了"预训练→微调"范式；GPT-3 用 prompt 把接口变成了产品形态。None of what came later stands without these three foundations. Transformer made scaling structurally possible; BERT set "pretrain → fine-tune" as the default; GPT-3 turned the interface into a product shape via the prompt.
阅读原文 ↗Read original ↗
第 2 期 · 2020–2022Issue 2 · 2020–2022
规模化为什么有效，边际在哪里Why Scaling Works — And Where the Margin Ends
Scaling Laws · Chinchilla · MoE
Scaling Laws for Neural Language Models — Kaplan et al., 2020
Training Compute-Optimal LLMs (Chinchilla) — Hoffmann et al., 2022
Switch Transformer — Fedus et al., 2021
Training Compute-Optimal LLMs (Chinchilla) — Hoffmann et al., 2022
Switch Transformer — Fedus et al., 2021
Kaplan 把"做大有用"写成了可量化的公式；Chinchilla 修正了配方，证明同等预算下数据比参数更值钱；MoE 给了一条只激活部分参数的工程路径。不是"更聪明"推动了行业，是"同样效果更便宜"。Kaplan turned "bigger helps" into a quantifiable formula; Chinchilla corrected the recipe, proving data beats parameters at equal budget; MoE gave an engineering path that activates only part of the parameters. It wasn't "smarter" that moved the industry — it was "the same result, cheaper."
阅读原文 ↗Read original ↗
成本Cost
第 3 期 · 2020–2023Issue 3 · 2020–2023
从"会生成"到"可交付"From "Can Generate" to "Can Deliver"
InstructGPT · Constitutional AI · 红队测试
Training language models to follow instructions (InstructGPT) — Ouyang et al., 2022
Constitutional AI — Bai et al., 2022
Red Teaming Language Models — Ganguli et al., 2022
Constitutional AI — Bai et al., 2022
Red Teaming Language Models — Ganguli et al., 2022
RLHF 让模型听话了，Anthropic 用原则清单给对齐一个可追溯的框架，红队把评估从"打分"推向"找漏洞"。ChatGPT 是这条线的产品化落点。RLHF made models obey; Anthropic gave alignment a traceable framework via a principles checklist; red-teaming pushed evaluation from "scoring" to "finding exploits." ChatGPT is the productized endpoint of this line.
阅读原文 ↗Read original ↗
成本Cost评估Evaluation
第 4 期 · 2020–2022Issue 4 · 2020–2022
知识的边界在哪里Where the Boundary of Knowledge Lies
把检索做成模型能力的一部分Making retrieval part of the model's capability
REALM — Guu et al., 2020
RAG — Lewis et al., 2020
Atlas — Izacard et al., 2022
RAG — Lewis et al., 2020
Atlas — Izacard et al., 2022
承认模型参数装不下所有知识之后，整个行业开始学习怎么把检索做成模型能力的一部分。"知识在哪存"这个问题的答案，从"全塞进参数"转向"模型做中枢、调度外部知识"。Once the field admitted that parameters can't hold all knowledge, it started learning how to make retrieval part of the model's capability. The answer to "where is knowledge stored" shifted from "shove it all into parameters" to "the model is the hub, orchestrating external knowledge."
阅读原文 ↗Read original ↗
知识Knowledge成本Cost
第 5 期 · 2022–2023Issue 5 · 2022–2023
从说到做：AI 不只是嘴上说说From Talking to Doing: AI Is More Than Words
Chain-of-Thought · Toolformer · ReAct
Chain-of-Thought Prompting — Wei et al., 2022
Toolformer — Schick et al., 2023
ReAct — Yao et al., 2022
Toolformer — Schick et al., 2023
ReAct — Yao et al., 2022
模型先学会把问题拆开想，再学会自己决定什么时候用工具，最后学会在推理和行动之间来回切换。Agent 这个词的学术源头就在这一期里。Models first learned to break a problem apart and think; then to decide on their own when to use a tool; finally to toggle between reasoning and action. The academic origin of the word "Agent" sits right in this issue.
阅读原文 ↗Read original ↗
知识Knowledge成本Cost
第 6 期 · 2022–2024 初Issue 6 · 2022 – early 2024
评估体系如何追上能力How Evaluation Keeps Up With Capability
从刷榜到打擂台From leaderboard-grinding to open arena
MMLU — Hendrycks et al., 2020
HELM — Liang et al., 2022
Chatbot Arena — Zheng et al., 2023
LLM-as-Judge — Zheng et al., 2023
HELM — Liang et al., 2022
Chatbot Arena — Zheng et al., 2023
LLM-as-Judge — Zheng et al., 2023
MMLU 在 GPT-4 面前饱和，旧基准集体失效。HELM 把评估展开成多维矩阵；Chatbot Arena 把评估搬到真实用户的持续投票里。评估这件事没有终点——每次能力跨过一个台阶，旧的尺子就要扔掉。MMLU saturated in front of GPT-4; the old benchmarks collectively broke. HELM unfolded evaluation into a multi-dimensional matrix; Chatbot Arena moved it into the continuous votes of real users. Evaluation has no finish line — each time capability clears a step, the old ruler is thrown away.
阅读原文 ↗Read original ↗
评估Evaluation
第 7 期 · 2021–2024 初Issue 7 · 2021 – early 2024
从看字到看懂：多模态的统一From Reading Words to Understanding: Multimodal Unification
CLIP · Flamingo · GPT-4V
CLIP — Radford et al., 2021
Flamingo — Alayrac et al., 2022
BLIP-2 — Li et al., 2023 LLaVA — Liu et al., 2023
Flamingo — Alayrac et al., 2022
BLIP-2 — Li et al., 2023 LLaVA — Liu et al., 2023
CLIP 证明了自然语言可以当视觉的监督信号；Flamingo 建立"冻结 LLM + 视觉适配层"的工程蓝图；GPT-4V 把多模态做成了对话框左下角那一个小图标。视觉、语言被放进同一个表征空间，第一阶段最后一块拼图。CLIP proved natural language could serve as a supervisory signal for vision; Flamingo set the engineering blueprint of "frozen LLM + vision adapter"; GPT-4V turned multimodality into that small icon in the bottom-left of a chat box. Vision and language entered one shared representation space — the final piece of Phase 1.
阅读原文 ↗Read original ↗
成本Cost
贯穿全阶段的三个视角Three Lenses Across All Phases
视角一 · 成本结构Lens 1 · Cost Structure
不是"更聪明"推动了行业，是"同样效果更便宜"It wasn't "smarter" that moved the industry — it was "the same result, cheaper"
从 Chinchilla 修正训练配方、到 Constitutional AI 降标注成本、到 RAG 省下重新训练的钱、到 Toolformer 用外部能力替代参数内能力——每一次主要切换，底下都压着一次成本结构的变化。From Chinchilla correcting the training recipe, to Constitutional AI cutting labeling cost, to RAG saving retraining money, to Toolformer using external capacity instead of in-parameter capacity — every major shift rests on a change in cost structure.
视角二 · 知识边界Lens 2 · Knowledge Boundary
知识从"全塞进参数"转向"模型做中枢、调度外部资源"Knowledge shifted from "shove it all into parameters" to "the model is the hub, orchestrating external resources"
GPT-3 那会儿默认把全世界塞进参数里。到 RAG，这个假设被推翻了。再到 Toolformer 和 ReAct，能力也被放到外面去了。模型从封闭生成器变成能调用外部检索、工具、代码的中枢。Back in the GPT-3 era, the default was to cram the whole world into parameters. RAG overturned that assumption. Then Toolformer and ReAct pushed capability outward too. The model went from a closed generator to a hub that calls external retrieval, tools, and code.
视角三 · 评估体系Lens 3 · Evaluation
评估一直在追能力，没有终点Evaluation has always chased capability — no finish line
BERT 时代用 GLUE，到 ChatGPT 已经失效；MMLU 在 GPT-4 面前趋于饱和。行业用活的排行榜、人类投票、模型互评重建尺子。但每次能力跨过一个台阶，旧的尺子就要扔掉。The BERT era used GLUE, already obsolete by ChatGPT; MMLU saturates in front of GPT-4. The industry rebuilds its rulers with living leaderboards, human votes, and model-on-model judging. But each time capability clears a step, the old ruler is thrown away.
第二阶段 · 方向预告Phase 2 · What's Next
推理范式改写Reasoning paradigm rewrite：o1/R1 之后，"多想一会儿"成为可独立分配算力的动作。: After o1/R1, "thinking a bit longer" became an action with its own allocatable compute.
对齐工程简化Alignment engineering simplified：DPO 系列证明不走 PPO 也能做到 RLHF 效果。: The DPO family proved you can reach RLHF-level results without PPO.
Agent 从能做到能卖钱Agents from capable to billable：Coding Agent、Browser Agent、数据分析 Agent。: Coding Agents, Browser Agents, data-analysis Agents.
开源生态重构Open-source ecosystem rebuilt：LLaMA、Mixtral、DeepSeek、Qwen——"开源追得上闭源"已经不是假设。: LLaMA, Mixtral, DeepSeek, Qwen — "open source catches closed source" is no longer a hypothesis.
数据天花板Data ceiling：高质量训练数据开始见顶，合成数据从辅助手段变成主要手段。: High-quality training data is plateauing; synthetic data has moved from auxiliary to primary.