Embodied AI | Xuhong Wang

Navimaster:The First Unified Navigation Model Across Digital and Physical Worlds, with Minecraft Support

Wed, 18 Mar 2026 00:00:00 +0000

Table of Contents

Abstract

Navigation now happens in two worlds at once: digital interfaces such as mobile, desktop, and web GUIs, and embodied environments that require movement, localization, and interaction in physical or simulated space. Although both are fundamentally navigation problems, they have long been studied through separate datasets, separate action spaces, and separate training pipelines.

That separation creates several practical bottlenecks:

separate models increase system cost
cross-domain generalization remains weak
sparse RL rewards slow training down
models often reason correctly but execute poorly

NaviMaster tackles this fragmentation by introducing a unified navigation agent that learns GUI navigation and embodied navigation under the same framework. Through a shared trajectory formulation, a unified RL pipeline, and distance-aware dense rewards, it substantially improves cross-task generalization, optimization efficiency, and grounding precision.

Task Demos

Spatial Grounding

The model predicts a target point directly from visual context and language constraints.

The agent interacts with interfaces through multi-step actions such as click, type, and wait.

Mixed GUI and Embodied Interaction

NaviMaster can also combine navigation and action execution in more complex settings such as Minecraft.

Existing systems usually treat GUI navigation and embodied navigation as different task families. GUI agents operate through clicks and scrolling, while embodied agents rely on movement, viewpoint shifts, and spatial control. Even if both can be abstracted as Markov decision processes, their data formats and training pipelines remain hard to merge.

NaviMaster starts from a different premise: once actions, goals, and trajectories are aligned, both domains can be learned together as one navigation problem. This turns GUI and embodied data into complementary supervision instead of isolated silos.

Three Core Innovations

1. A Unified Visual-Goal Trajectory Formulation

The first challenge is that GUI trajectories and embodied trajectories do not share a common language. NaviMaster resolves this by converting both into a unified visual-goal trajectory format.

The action space is systematically aligned:

task-specific actions are preserved and inserted into a shared action vocabulary
GUI [SCROLL] and embodied [TURN] actions are both discretized into directional changes
GUI [CLICK(x, y)] and embodied movement actions are unified through explicit target-point actions such as [MOVETO(x, y)]

GUI trajectories are built from existing datasets such as GUI-Odyssey, while embodied trajectories are derived from shortest-path keypoints extracted with A* search and then converted into visual-goal action sequences. NaviMaster also adds reasoning intents for each step, generated with GPT-4o, so history contains not only actions but also why those actions were taken.

2. A Unified Reinforcement Learning Framework

Once trajectories are aligned, NaviMaster trains directly with GRPO on mixed GUI and embodied data instead of using separate warm-start pipelines. Both domains are treated as the same decision process: given an observation, an instruction, and execution history, the model selects the next action from a language-defined action space.

This allows one policy to learn from 2D GUI data and 3D embodied data together, strengthening cross-domain generalization rather than overfitting to a single environment family.

3. Distance-Aware Dense Rewards

To address sparse supervision, NaviMaster decomposes success into three components:

whether the output is executable
whether the action type is correct
whether the predicted target is sufficiently close to the ground truth

Instead of binary success-or-failure signals, the model receives graded feedback based on how close it is to the target. This makes learning more stable, reduces useless exploration, and improves convergence speed.

Experimental Highlights

NaviMaster shows strong out-of-domain generalization on GUI tasks. Evaluation is performed entirely on OOD test sets isolated from the training distribution. Against strong baselines, NaviMaster consistently improves success rate across mobile, web, and desktop benchmarks.

More importantly, mixed GUI-plus-embodied training outperforms training on only one domain, showing that the two data sources provide complementary navigation signals.

Spatial Grounding

On four spatial grounding benchmarks, NaviMaster outperforms all baselines. The gains hold for both object-level reference and free-space pointing, showing that the model learns substantially stronger fine-grained visual-spatial alignment.

On embodied navigation tasks such as ObjectNav-unseen, NaviMaster also delivers clear improvements. Under the VLMNav framework, replacing only the base model is enough to reveal the contribution of the method. The results suggest that NaviMaster is the first agent model in this setting to demonstrate strong generalization.

Again, mixed training outperforms GUI-only or embodied-only variants, reinforcing the value of unified optimization.

Deeper Analysis

Mixing Ratio

Overall performance peaks when GUI and embodied data are mixed at roughly 5:5, indicating that balanced cross-domain supervision is especially effective. Even under imbalanced ratios, mixed training typically still beats single-domain training.

Gains Across Backbones

The method brings consistent improvements on multiple base models, including Qwen2.5VL-7B, Qwen2.5VL-3B, and Qwen2VL-7B, showing that the gains are not tied to one specific backbone.

Data Scale and Reward Design

The unified training strategy remains effective at both smaller and larger data scales. Meanwhile, dense rewards converge faster than sparse rewards and also reach stronger final performance.

NaviMaster is important not because it simply merges two benchmarks, but because it demonstrates a more natural route toward unified multimodal agents. A future agent should not need to be split into one that only clicks screens and another that only moves in physical space. It should be able to perceive, reason, and act across both.

From that perspective, NaviMaster is best seen as an early but meaningful step toward a broader class of unified agents.

Paper: NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Tasks
Project page:

SafeVerse: Building a Safe and Trustworthy Digital Twin Arena for Embodied AI

Tue, 10 Feb 2026 00:00:00 +0000

Table of Contents

Abstract

Safe and trustworthy embodied AI needs realistic testing grounds, but running attack-and-defense drills directly in the physical world is expensive, risky, and hard to reproduce. SafeVerse addresses this gap by first digitizing a specified real-world environment, then turning that twin world into an editable arena for safety evaluation, adversarial testing, and online reinforcement learning.

Unlike world models that mainly aim to generate plausible open-ended environments, SafeVerse focuses on reconstructing a particular real scene with low cost and high controllability. Its emphasis is not just realism, but operational realism: the reconstructed world should be interactive, editable, and useful for agent training and verification.

Why SafeVerse Matters

Existing embodied environments often fall into one of two extremes:

Traditional simulators rely heavily on manual asset construction, offer limited interactive objects, and struggle to reflect the diversity of real-world scenes.
Generative world models can imagine rich environments, but they are not faithful twins of a user-specified home, office, or factory floor, so they are hard to use for targeted security drills.

SafeVerse starts from a more practical premise: what safety testing needs is not an imagined world, but a controllable digital twin of a real one. It therefore builds a three-step loop:

Reconstruct a real environment from video.
Edit that environment according to attack or evaluation goals.
Let the agent evolve online under continual adversarial pressure.

Three Core Breakthroughs

The system is organized around three main capabilities:

Real-world Ctrl+C / Ctrl+V: it tries to preserve not only appearance, but also structure, semantics, and interaction logic.
Minute-level construction with operable objects: a short video can become an interactive 3D scene where doors open, lights switch, and furniture moves.
Unified evaluation, attack, and evolution: the same environment can support testing, adversarial scene mutation, and online RL-based agent improvement.

This makes SafeVerse less like a standalone simulator and more like a digital twin infrastructure for trustworthy embodied intelligence.

From Ordinary Video to an Interactive Twin World

The first step in SafeVerse is to make the system actually understand the source video. Instead of leaning on a purely geometric 3D optimization pipeline, it uses multimodal understanding to parse objects, layouts, and scene semantics, then maps them into operable 3D entities.

Built on top of Minecraft and its rich physical interaction rules, SafeVerse converts recognized scene elements into 3D objects with explicit interaction affordances. The result is not a static reconstructed set, but a sandbox where an embodied agent can enter, move, manipulate, and explore.

These four GIFs illustrate the pipeline from input video to interactive 3D environment. The key point of the original webpage is that SafeVerse makes video in, operable twin out practical at minute-level turnaround.

Editing the Scene with Attack Instructions

Reconstruction alone is not enough for safety validation. The environment also needs to change in response to specific attack goals.

SafeVerse emphasizes a combination of realism and editability. Once a twin scene has been built, it can be modified directly for attack-and-defense scenarios, including:

changing interaction properties, such as turning an ordinary door into one that must be unlocked first
altering semantic cues, such as changing an object’s appearance to mislead recognition
perturbing spatial layout, such as repositioning furniture or obstacles to break a navigation plan

This allows attack vectors to be injected into the environment itself, creating more realistic and better targeted embodied stress tests.

Online Evolution Against Discovered Vulnerabilities

SafeVerse does not stop at evaluation. It pushes the loop one step further toward online evolution.

Conventional embodied training often depends on fixed datasets and static scenes. When a new attack or environmental shift appears, agents can fail catastrophically. SafeVerse tries to solve this through a reconstruction-attack-defense loop: rebuild the scene, perturb it dynamically, and retrain the agent immediately after failure.

That means the agent is no longer tested in a frozen benchmark. It must adapt to changing layouts, newly inserted obstacles, altered object states, and other evolving threats in real time.

One example from the original page is especially clear: when a chair blocks the only path to the goal, the agent initially fails. After online training, it learns to recognize the obstacle, reroute, or even move the chair away. The point is not only that it encounters a failure, but that it can grow inside that failure.

Full SafeVerse Demo

This video restores the full-process demo referenced in the original webpage and shows the combined effect of reconstruction, attack-oriented editing, and online evolution more clearly.

Your browser does not support the video tag.

What This Work Shows

SafeVerse is not just another embodied simulator. Its main contribution is that it connects three capabilities into one loop: fast digitization of a specified real scene, attack-oriented scene editing, and online RL-based agent evolution.

Many embodied platforms are good at offering training spaces. SafeVerse is more specifically about offering safety drill spaces. It turns real-scene digitization into a working capability for safe and trustworthy embodied AI research.

GitHub:

Embodied AI | Xuhong Wang

Navimaster:The First Unified Navigation Model Across Digital and Physical Worlds, with Minecraft Support

Abstract

Task Demos

Spatial Grounding

GUI Navigation

Mixed GUI and Embodied Interaction

Why Unify GUI and Embodied Navigation

Three Core Innovations

1. A Unified Visual-Goal Trajectory Formulation

2. A Unified Reinforcement Learning Framework

3. Distance-Aware Dense Rewards

Experimental Highlights

GUI Navigation

Spatial Grounding

Embodied Navigation

Deeper Analysis

Mixing Ratio

Gains Across Backbones

Data Scale and Reward Design

NaviMaster as the Opening of Navigation Agents

SafeVerse: Building a Safe and Trustworthy Digital Twin Arena for Embodied AI

Abstract

Why SafeVerse Matters

Three Core Breakthroughs

From Ordinary Video to an Interactive Twin World

Editing the Scene with Attack Instructions

Online Evolution Against Discovered Vulnerabilities

Full SafeVerse Demo

What This Work Shows

Embodied AI | Xuhong Wang

Navimaster:The First Unified Navigation Model Across Digital and Physical Worlds, with Minecraft Support

Abstract

Task Demos

Spatial Grounding

GUI Navigation

Mixed GUI and Embodied Interaction

Why Unify GUI and Embodied Navigation

Three Core Innovations

1. A Unified Visual-Goal Trajectory Formulation

2. A Unified Reinforcement Learning Framework

3. Distance-Aware Dense Rewards

Experimental Highlights

GUI Navigation

Spatial Grounding

Embodied Navigation

Deeper Analysis

Mixing Ratio

Gains Across Backbones

Data Scale and Reward Design

NaviMaster as the Opening of Navigation Agents

Related Links

SafeVerse: Building a Safe and Trustworthy Digital Twin Arena for Embodied AI

Abstract

Why SafeVerse Matters

Three Core Breakthroughs

From Ordinary Video to an Interactive Twin World

Editing the Scene with Attack Instructions

Online Evolution Against Discovered Vulnerabilities

Full SafeVerse Demo

What This Work Shows

Related Links