Report version of this blog post

Today I took my child to a physical-fitness class. The teacher built a race course out of a few arches and cones: first push a ball through the arches, then weave a figure-eight between the cones, and finally carry the ball back to the start. The teaching method was simple: the teacher first walked the course with the kids, explaining each step as they went, then let the kids start from the beginning on their own. Where they did something wrong, the teacher called out and pointed to the details to watch. Usually within one or two laps, the kids could run the full course.

The method looks ordinary, but thinking it over, it’s fascinating: the child wasn’t “trained.” He didn’t practice a hundred times — he watched once, listened once, understood the structure of the task, and could then do it. For a two- or three-year-old this is so natural that we don’t even consider it an “ability.” But swap in a robot, and it’s nearly impossible — most robots can only execute a few pre-programmed task types; you can’t teach it a new route on the spot and have it learn the route right then and there.

Last week I saw Generalist release Gen 1.5, which is about exactly this — they call it the GPT-3 moment of embodied AI. GPT-3 is considered a watershed not because it topped some benchmark, but because it first made few-shot in-context learning usable: give two or three examples in the prompt and it can do a new task it was never specifically trained on. The cost of teaching it a new task collapsed from collecting thousands or tens of thousands of training samples to a few seconds of typing. Gen 1.5 aims for the same thing to happen to robots: no retraining, just a few examples given on the spot, and the robot picks up a new task.

Gen 1.5’s core capabilities break into two blocks. The first is in-context learning — defining the task on the spot with context of various forms, rather than picking from a preset task library. This context can be language, images, or videos of human demonstration — record a video of how you wipe a table or plate food, show it, and it can follow along, no separate data collection or training for that motion. The second is improvisation: the model can flexibly complete the task with the tools at hand or the current conditions, rather than rigidly applying a fixed execution procedure. No dedicated brush at hand? A banana can wipe the counter clean too — this ability to figure things out on the spot in unexpected situations is what really makes you feel “this thing has some intelligence.”

Put together, these two capabilities say the same thing: tasks no longer need to be enumerated and trained in advance; the model can understand a task the moment it sees it and figure out how to complete it on the spot. This is isomorphic to the fitness-class teacher teaching kids the course — the teacher didn’t have the child practice that specific route a hundred times; the child watched once and got it, and when a cone was misplaced or the ball rolled off, he could adjust on the fly.

I previously wrote “Letting Intelligence Step Out the Door,” which was more about imagining what mobile intelligence could do after entering society: fetching a charger for a colleague who forgot theirs, delivering food to a specified floor, security patrol. That piece leaned toward application-level imagination. Today I want to push one step further: to reach “can be taught new tasks on the spot,” what levels must mobile intelligence pass through, and how is each achieved?

The Levels of Mobile Intelligence

I roughly divide mobile intelligence into three levels, judged by “how much does it cost to change the environment or the task.”

The first level is pre-mapped navigation dependent on manual mapping. Maps must be built in advance, semantic targets manually bound to coordinates, and changing venues requires redeployment. Today’s commercial food-delivery and hotel-delivery robots are basically here; the cost is days to weeks of on-site engineering for every deployment.

The second level is the full repertoire of conventional navigation: following people, finding objects, map-based navigation, roaming, executing clear instructions — the basic forms all work, and it can read door plates and road signs and plan routes including floor changes. But this level still only executes pre-defined task types; you can’t have it do something on the spot that it never saw in training.

The third level — the real watershed, I believe — is the emergence of in-context learning: the robot can take a concrete task defined on the spot by language, images, or demonstration, and execute it without additional training, even if that task was never enumerated during training. The biggest change at this level isn’t a longer capability list — it’s that the way of acquiring new capabilities is completely transformed: the cost of adapting to a new task drops from “time and data” to “almost nothing.” This matches the fitness-class analogy perfectly: the child doesn’t need to practice each new route separately — one look and he’s off. Level-one and level-two robots can’t do this; they always need separate investment for each new task.

Levels one and two are quantitative accumulation — doing known things more completely and more reliably. Level three is qualitative change — it determines whether mobile intelligence can truly enter ordinary people’s lives, rather than just circling inside a few carefully staged scenarios. Levels one and two determine whether it’s usable; level three determines whether it’s genuine intelligence.

How the Third Level Can Be Controlled

The key to the third level is that tasks can be defined on the spot with many different forms of context, not just one fixed instruction format. The forms I can think of today:

• Demonstration: walk the robot through the route once, or record a video of a human doing it and show it; it follows along. This carries the most information, containing both spatial path and temporal rhythm — the fitness teacher’s method is essentially this.
• Language instructions: e.g., “go left first, turn right when you see the elevator” — specifying a path sequence, not an endpoint coordinate. Taking the wrong path counts as failure even if you end up in the right place. Language instructions also have granularity: the above might be a 10-to-50-meter granularity; I could also have meter-level granularity, like “snake around these cones and turn back at the fire hydrant.”
• Image or image-plus-text paths: draw a red line on a photo to indicate the route. The model must recognize that this line is an instruction, not scene content, and map the 2D curve onto its own 3D path.
• Real-time multimodal interaction: saying “turn left,” “go around here” while walking. The difficulty isn’t semantic understanding — it’s temporal alignment and response speed.

This list doesn’t aim to be exhaustive; the ways humans give instructions in the real world are far richer. The purpose is to show that the input channels must be broad enough — if users have to learn “which instruction formats are supported” before they can use it, we haven’t reached the true third level yet.

How the Data Should Be Built

Following this logic, the direction for data collection becomes clear: pursue human, diverse, varied trajectory and task data. The causal chain is direct — only with sufficiently diverse data can diverse instructions be annotated, and only then can the model learn to output diverse behaviors. If the data itself isn’t diverse, the instructions can’t be annotated and the model naturally can’t learn them.

Here is a counter-intuitive but important insight: when data volume is large enough, large numbers of reasonable-but-rare trajectories naturally appear. These trajectories are not noise and don’t need to be cleaned out — they are themselves part of intelligence. The reason is simple: today we may not even be able to imagine certain situations arising, but when actually encountering them, that handling is exactly what’s reasonable. A person’s on-the-spot reaction to an unexpected situation in a real scene is precisely the most valuable data, because such reactions can’t be designed in advance. The model’s improvisational ability comes, in the end, from this batch of data.

So data collection cannot be limited to open lots where people walk pre-set routes, nor to rule-based synthetic videos in simulators — those collect “performed” trajectories that fail to produce corresponding actions in real scenarios. What’s truly valuable is the trajectory a person leaves in a real building when they really need to get somewhere — containing hesitation, looking up at signs, walking the wrong way and doubling back, waiting for elevators, dodging pedestrians. These details are precisely the source of rare trajectories and improvised reactions. Authenticity and diversity are, here, the same thing.

How the Business Model Should Be Built

The problem with real data is that it doesn’t appear out of thin air — only when robots actually run in real scenes do real trajectories flow back. This determines that the business model cannot be “perfect the technology first, then sell it,” but must be a closed loop: pick a real scenario the system can take over, enter at a low level with current capabilities (with human backstop), flow the real data back to train the model, and as the model grows stronger, expand the scope it can take over — or enter the next scenario.

The most critical step in this loop is choosing the scenario, and choosing a scenario is essentially choosing the data — choosing the future capability boundary of the model. A scenario is worth entering if it meets a few conditions: the abilities it demands resemble those of the eventual target scenarios (even with detours, it must not be a dead end); even completing just one sub-step of the task lets you start running; and the data can be brought back for training within privacy and compliance rules. In-office delivery within an office building is a good starting point — floors, elevators, corridors share core capabilities with most indoor mobility scenarios, and on-site-defined task needs like “it’s in the second cabinet to the left of my desk” or “use the back door, don’t pass the reception” arise naturally every day — exactly the kind of data needed to train level-three capabilities.

This model doesn’t require high success rates from the start. Even if the robot can only handle the most frequent, simplest 30% of needs, with humans backstopping the other 70%, the business still works — a hotel can go from ten cleaners to seven cleaners plus three robots, while that 30% of tasks continuously produces real data that pushes capability upward. Technical breakthrough and commercialization are not sequential; they are two sides of the same loop.

Closing

Back to the fitness class. The child can learn a new route in one pass not because he’s born good at routes, but because he has two more fundamental abilities: first, understanding a new task from a few examples given on the spot (in-context learning); second, when a cone is misplaced or the ball rolls off — unexpected minor situations — he can still adjust on the fly and finish the task (improvisation).

For mobile intelligence to step out of carefully staged scenarios into genuine daily life, the key is not how stable the robot’s movements are or how fine the maps are — it is the intelligence itself, especially these two capabilities. Few-shot learning frees robots from collecting data and training separately for every new task; improvisation lets it still find a way to finish when facing situations no one designed for. Together, these two are the true GPT-3 moment of mobile intelligence.

References

1. Generalist Team. GEN-1.5: Embodied Foundation Models are One-Shot Learners. Aug 2026. generalistai.com/blog/gen-1.5
2. Physical Intelligence. π0.7: a Steerable Model with Emergent Capabilities. Apr 2026. pi.website/blog/pi07
3. Physical Intelligence. π*0.6: a VLA that Learns from Experience. Nov 2025. pi.website/blog/pistar06
4. Huang Qingqiu (CTO, Moqi Intelligence). Embodiment Is a Marathon; Neither VLA Nor World Models Are Fundamental Enough. 漫谈 Light the Star #82, Aug 2026.
5. Wang Kaixuan. Letting Intelligence Step Out the Door. Mar 2026. wang-kx.github.io