As an engineer applying deep learning to autonomous driving, I have a solid grasp of deep learning fundamentals, but I always lacked imagination about the potential and extended applications of large models. New technologies keep stunning me — have we really come this far already?

Over the past month I had two experiences with coding agents. The first was using Trae to reorganize my personal website and support blog updates. That didn’t shock me much, because I knew exactly what I was doing — Trae was just a very handy tool. The second time, I used Claude Code to run some baseline experiments for my wife and organize the results. That was a genuine shock, possibly because:

1. I set up Claude Code and the Claude API, and could see token consumption measured in the millions. Previously, interacting with LLMs through chatbots, we never got close to that level of usage. It’s the difference between a kerosene lamp and a jet engine. No wonder LLM infrastructure needs hundreds of billions of dollars — otherwise you simply can’t reach that throughput.
2. My wife works in communications, a field I know nothing about. Following her instructions, I dumped her papers and rebuttal files into Claude Code; what experiments to run and how to compute the metrics were entirely up to the model. As long as the final metrics fell within a reasonable range, the experiments were basically sound. Along the way, the model even made targeted optimizations for methods whose time complexity was too high on large-scale data. This experience left me thoroughly impressed by the model’s breadth of knowledge — throughout the whole process I contributed nothing unique of my own, just launching scripts on servers and killing experiments that crashed.

Although an Agent is a system built on top of an LLM, it is completely different from the LLM itself: it amplifies the LLM’s intelligence so its influence is no longer confined to a text box, and it can accomplish almost any digital task.

How is an agent built? How does an LLM that can only output text sequentially become an agent capable of handling complex tasks?

At the foundation of agents, we cannot avoid the ReAct framework. ReAct is the combination of Reasoning and Acting. Given a task, an ordinary LLM’s logic is to answer directly, or to use CoT to improve answer correctness — but it cannot access external information or operate external systems. The ReAct framework instead loops through thought-action-observation, continuously acquiring new knowledge and completing the task step by step. Here an “action” is simply a pre-defined interface — Skills and MCP can both be seen as conventions for defining actions. When the LLM outputs an action conforming to the agreed format, the system parses it and calls the corresponding API to perform the operation. The feedback from the action is then added back into the LLM’s context, driving the next iteration.

Take debugging an error in coding as an example: the whole process might require reading the error message, reading the relevant code, writing corrected code back, executing the code, and so on — several actions to complete the task.

ReAct is still a fairly simple framework, and it’s not hard to derive a family of other architectures from it. For example, introduce a plan model dedicated to top-level planning: first decompose the problem, then hand subtasks to other agents (subagents). The plan model doesn’t need to know how things are done — only whether they succeeded. During planning, tree-of-thought structures can evaluate different plans to raise the success probability. Going further, the results of each action can feed into the system’s reflection process, supporting replanning.

These improvements are still at the level of system thinking. There is also work where multiple agents collaborate to complete a complex task — for example MetaGPT, which defines roles such as product manager, architect, engineer, and QA engineer, decomposing complex software development into a series of simply executable tasks. This kind of agent architecture borrows the division of labor from human society, letting a group of agents coordinate and operate autonomously.

In summary, the transformation from LLM to Agent involves at least:

1. An agreed tool-use protocol — through external tools and APIs, the LLM gains eyes and hands.
2. Advanced modes of thinking, such as planning and reflection. As Charlie Munger said, “You’ve got to have models in your head… you’ve got to array your experience — both direct and vicarious — on this latticework of models.”
3. Cooperative relationships defined through roles — like human division of labor, different agents specializing in different aspects is what makes complex tasks executable.

The LLM is the engine of the agent system, and modern LLM iteration increasingly emphasizes agent-oriented capabilities like tool use and planning. Concretely, that means synthesizing large amounts of tool-use, API-call, problem-decomposition, and reflection data, and doing large-scale rollouts in environments to support RLVR-based reinforcement learning.

Agents are currently developing faster than ordinary users can learn — often before you’ve figured out one framework, a better one is out. People with strong learning ability have already multiplied their output tens of times with agents. As the inventor of Claude Code said, he hasn’t written a single line of code for months, and his productivity has doubled. Agents are unquestionably the technology that will rapidly transform productivity and production relations in the next few years.

Other

• 马克的技术工作坊, 128. Manus’s Final Interview Before the Sale, 飞天闪客 — the above helped me understand some agent details.

• 写代码的人不写代码了 (The people who write code no longer write code)

• SpaceX has already shifted focus to building a self-growing city on the Moon — Moon-focused milestones are indeed more realistic than Mars. This time Elon is promising a sustainable lunar city within 10 years; no idea when it will materialize. Long-term, I’m optimistic about SpaceX’s engineering capability, and I hope there can be such a beacon pulling everyone to keep breaking through.

• cosmos-drive-dreams-toolkits — a condition editor for autonomous driving world models.

• DreamZero uses (fine-tunes) video generation models as foundation models for embodied AI. Video generation models really are powerful geometry-understanding and scene-prediction models. In 3DV, video generation models have already been fine-tuned into depth estimation, reconstruction, and novel-view synthesis systems. Compared with the VLM→VLA paradigm, video generation models have denser supervision and naturally accommodate various robot embodiments, since everything is image in/out. It’s quite reasonable as part of system-1.

• World-VLA-Loop — as I understand it, generative models like world models are used to supplement VLA training data. The WM generates both image sequences and rewards, densifying sparse VLA supervision data and thereby improving VLA capability.