中文

Wang Kaixuan / 3D Vision & Robotics

Wang Kaixuan Blog

Personal blog on 3D vision, robotics, embodied AI and weekly notes.

Mar 21, 2026

Letting Intelligence Step Out the Door

#周报

robot-navi

In recent interviews, I often lack the time to explain clearly to candidates what we mean by a mobile foundation model. So I wrote this piece to give people a relatively concrete idea.

When people talk about embodied AI, the mental picture is usually a dual-arm robot folding laundry, folding cardboard boxes, or pouring coffee — or a humanoid tightening screws on a production line, or tidying up a countertop at home. What the industry invests most in is indeed two things: making manipulation in a local space work, and making the robot’s motion control stable, so humanoids can walk and stand and dual arms can coordinate. By stacking these two capabilities, we can build a general-purpose robot for fixed locations, handling a variety of household and factory tasks.

These directions are all valuable, but they assume a default premise: the robot stays in one relatively fixed site with task boundaries roughly known in advance. In other words, intelligence still “lives” inside factories, labs, or single apartments. It is rarely asked to do what a person does: understand a natural-language sentence, figure out on its own how to get somewhere else, what to watch out for on the way, and then keep working after arriving.

Can a robot step out that door — into hallways, campuses, shopping malls, streets — and complete tasks equally described by language or images? Existing systems reveal their weaknesses: executability across larger spaces, understanding of semantic elements in open environments, and integration with task planning and external information (navigation, contact directories, ticketing systems) are all still in early stages.

The class of models behind this ability of “letting intelligence step out the door” is called the mobile foundation model. What it emphasizes is not the locomotion of staying balanced, but the layer of intelligence supporting “understanding, arriving, and sustained execution.”

On embodiment and motion, progress from traditional gait and balance to full-body coordination in recent years has been rapid. The direction represented by Figure Helix 02 — putting walking, manipulation, and balance into a unified framework, achieving long-horizon loco-manipulation at room scale — shows that “doing whole-body work within one room” is becoming a pursuable goal. Dual-arm manipulation is another main thread: pick-and-place, folding, and other rich manipulations keep improving in precision and reliability with the introduction of reinforcement learning.

These capabilities complement “stepping out the door.” However fluent the in-room whole-body planning gets, the default assumption is still “enclosed scene, limited scale.” If intelligence can really step out the door and work out of the box without complex setup by consumers, a batch of tasks unlocks naturally, for example:

  • Discovering a forgotten charger in a meeting room and telling the robot in natural language to fetch it from your desk;
  • A food delivery arriving downstairs, with the robot going to the pickup point, verifying the order, and delivering it to a specified floor or seat;
  • At a front desk or in an office-administration scenario, distributing packages to individuals according to a searchable directory;
  • Security patrol: patrolling a route, logging loitering or abnormal behavior, and performing initial interaction;
  • In a mall or complex, after a customer places an order, the robot bringing the product from near the store to the customer’s area.

The core behind these examples is not “can it walk” — it’s that tasks are defined by language and scene semantics, execution chains are long, and the system must make use of objects, rules, and other information sources in the external environment. The upper layer needs a highly capable agent: understanding intent, breaking down steps, retrieving or calling tools to gather data, then handing structured or semi-structured goals to the mobile foundation model that is closer to the physical world, to obtain mobility and interaction decisions that can be tracked in real time.

In embodied AI, the current mainstream choice is VLA (e.g., π0.5): adding an action module on top of a pre-trained VLM, extending language understanding and visual alignment down to motor control. But the VLM architecture was designed for image and text understanding, not optimized for mobile robots. For instance, when handling a robot’s multi-view images, a VLM cannot fuse them into one complete scene understanding; failing to tell front from back is the norm. With multi-stream video input, redundant information gets processed repeatedly, and inference latency directly conflicts with the demands of real-time control.

The other route is world models (WM), which are friendlier to modeling video and motion trajectories, but weaker at grounding language instructions — textual and semantic information like road signs, door plates, and business rules is hard to understand directly and turn into correct predictions and actions. Given the current state of both routes, VLAs and WMs in their present form cannot independently support long-chain, semantically-driven mobility tasks in open environments; what the mobile foundation model must solve is precisely the gap each of them leaves.

After building such a robot-system-friendly foundation model, we also need massive data support (internet data plus self-collected data) and reinforcement learning on long-horizon tasks, to lift the system to the point of being genuinely usable out of the box and trustworthy. That goal is the original intention of our investment in the mobile foundation model.