A Practical Causal Framework for Autonomous Mobile Robots:
From Simulation to Real-World Human Environments

Luca Castri1, Gloria Beraldo2,1, Nicola Bellotto1
1University of Padua, Italy
2National Research Council of Italy
IEEE Robotics and Automation Letters
arXiv Paper causal-sim2real

Abstract

Causal inference has emerged as a powerful paradigm enabling robots to make safe, efficient decisions by reasoning about the ``why'' behind events. However, its real-world application is hindered by the difficulty of collecting causal data, especially in human-populated environments where training takes time, while users expect immediate operation. To this end, we tackle the challenge of transferring simulation-learnt causal relations to the real world with minimal effort. While retaining the invariant high-level causal structure, we empirically demonstrate the system adapts efficiently to real-world situations by updating only the causal model parameters with a limited set of new observations. Our results show that the parameter update enables rapid sim-to-real transfer. The framework runs efficiently onboard of a mobile robot and significantly outperforms a non-causal baselines navigating successfully and safely in complex human-shared scenarios.

Challenge ⚠️

Causal inference requires an extensive data collection process to identify underlying relationships, which is often safety-critical and time-prohibitive on physical hardware.

In human-populated settings like hospitals or warehouses, long-term data collection is impractical, yet users expect robots to be immediately functional. Conventional "black-box" sim-to-real methods often fail because they rely on statistical correlations that change between environments.

Idea 💡

Our framework exploits Causal Structural Invariance: the principle that while environment-specific parameters change, the high-level causal mechanisms remain the same.
By transferring the invariant causal structure discovered in simulation and performing a rapid parameter update with limited real-world observations, we allow the robot to adapt the causality-based decision-making framework to the real-world environment.
➡ Efficient Sim-to-Real transfer with limited real-world data.

Causal Structural Invariance Overview

Causal Robot Framework

This end-to-end framework was adapted from the original Icon Causality-based Decision-Making Framework. The core of the latter consists of three interconnected pipelines: data extraction, learning, and inference.

Data Extraction Pipeline: The data extraction pipeline continuously gathers information about the robot, humans, and contextual factors from the environment. The collected data is then preprocessed through subsampling to manage volume and discretisation to prepare it for the learning and inference components. For real-time inference, data is processed and used directly without being stored.

Learning Pipeline: The learning pipeline is responsible for discovering the underlying causal relationships from the processed data and then learning the parameters of the model. It uses a causal discovery method, such as J-PCMCI+ (from CausalFlow icon CausalFlow), to identify the causal structure from time-series data containing contextual variables and a Maximum Likelihood Estimation (MLE) to estimate the Conditional Probability Distributions (CPDs) for each node in the discovered causal graph.

Inference Pipeline: The inference pipeline enables the robot to make decisions by using the learned causal model to evaluate its actions. When assigned a task, the robot formulates a query as a causal intervention, such as, "What if I go to the delivery point now at velocity v?". The pipeline then estimates the expected outcomes, specifically the battery consumption and people density. These estimates are incorporated into a heuristic that guides the A* algorithm to find an optimal path and decide whether to execute or abort the task.

Simulation-Based Learning

To facilitate rapid deployment, our framework leverages the concept of causal structural invariance. The core assumption is that high-level causal mechanisms learnt in simulation remain valid in the real world, even though specific internal parameters, such as exact probabilities of congestion or battery discharge rates, are environment-dependent. To learn this invariant structure safely and efficiently, we utilise the PeopleFlow simulator to model context-driven human-robot spatial interactions alongside the robot's internal dynamics, such as battery consumption influenced by velocity and unexpected obstacles.

Real-World Adaptation via Data-Efficient Update

To transfer the simulation-trained causal model to the real world, we adopt a data-efficient fine-tuning strategy. We equip the physical robot with the invariant causal graph already learnt in simulation.

The robot then undergoes a brief observational phase to record real-world data. Because the causal dependencies are already fixed by the invariant graph, we skip the computationally expensive structure learning phase. Instead, we perform a rapid parameter update using MLE on this limited real-world data to recalibrate the CPDs. This allows the robot to immediately adapt to the specific crowd behaviours of its new physical venue with minimal effort.

Causal Decision-Making Framework

Experimental Setup

Objective: To evaluate the framework's navigation performance in simulation and reality, alongside its sim-to-real adaptability and real-time viability on a TIAGo robot.

Scenarios: Four social scenarios across two environments:

  • Conference: Poster session (S1), with queues forming along a main corridor; buffet event (S2), where groups gather around food stations.
  • Airport: Passenger crowds queuing at baggage reclaim belts upon flight arrivals (S3 and S4).

Across all scenarios, the robot must navigate through the context-dependent crowds to reach a designated goal.

Baselines: Standard ROS~1 move_base with shortest-path A* (base), and a socially-aware variant integrating human proxemic costs into path planning (social), both using TIAGo's default TEB local planner (max speed 0.5 m/s).

Hypotheses:

  • Effectiveness: The system yields safer paths and higher task success rates than the baselines across both simulated and physical environments.
  • Adaptability & Generalization: The invariant causal structure transfers effectively to new topological layouts with unseen human activities (sim-to-sim) and physical environments (sim-to-real) through lightweight parameter adaptation, avoiding structural discovery from scratch.
  • Efficiency: The system is efficient, utilising a light-data parameter update process for rapid real-world adaptation and performing causal inference fast enough for real-time onboard execution.

Metrics: Task Success (binary), Safety (human-robot proxemics compliance), Inference Time.

Learning the Causal Model

Before deploying the robot in the real world, we first needed to teach it how different environmental factors interact and influence one another. To do this safely and efficiently, we collected data using the PeopleFlow simulator across two distinct social scenarios: a structured poster session and a dynamic buffet event.

The Causal Hypothesis

We hypothesised an underlying "invariant causal structure" that governs these human-shared environments. In simple terms, we expected the following chain of cause-and-effect:

  • Context ➡ crowds: The specific Time slot (T) and Waypoint (W) determine the People Density (D).
  • Crowds and obstacles ➡ speed: The local density of people, along with unexpected Obstacles (O), directly influences the robot's Speed (S).
  • Speed and obstacles ➡ energy: The robot's speed and the avoidance maneuvers caused by obstacles ultimately determine its Battery Depletion (L).

From Raw Data to Causal Graph

To validate this structure, we let the robot navigate the simulated environments for an hour using a standard baseline planner during the poster and buffet events. During this time, we continuously recorded its internal state (speed and battery) alongside contextual data (time, location, and obstacles).

We then fed this time-series data into the J-PCMCI+ causal discovery algorithm via CausalFlow icon CausalFlow, which confirmed our hypothesised causal relationships.

Causal Graph

Simulation Results

This section presents the first stage of our experimental evaluation, focusing on simulation results. Specifically, we validate the navigation performance of our proposed causality-enhanced framework and demonstrate that the causal model outperforms a standard Bayesian network for inference tasks.

To assess the effectiveness of our path planning and task execution, we evaluated the robot's navigation behavior using PeopleFlow. A topological map was employed to facilitate both TIAGo's and the simulated humans' spatial navigation.

Conference Topological Map with Waypoints

Conference: Topological map with navigation waypoints (yellow nodes).

Conference Scenarios S1 and S2

(a) Conference

S1 (Poster Session): Queues forming along a main corridor.
S2 (Buffet Event): Groups gathering around food stations.
Shown: 3D snapshots, baseline shortest paths, and causal proactive navigation.

Airport Topological Map with Waypoints

Airport: Topological map with navigation waypoints (yellow nodes).

Airport Scenarios S3 and S4

(b) Airport

S3 & S4 (Baggage Reclaim): Passenger crowds queuing upon flight arrivals.
Shown: 3D snapshots, baseline shortest paths, and causal clear routes.

To test the sim-to-sim transfer capabilities, we evaluated whether the invariant causal graph structure learned from the conference scenarios (\(\text{S1}\) and \(\text{S2}\)) could be directly applied to a completely new, unseen simulated airport environment (\(\text{S3}\) and \(\text{S4}\)). The airport model was adapted using a 2-minute observational data window to recalibrate the tabular CPD parameters over the invariant structure, mirroring the real-world adaptation protocol.

Navigation Performance

We evaluated the framework against the base and social baselines across 200 simulated navigation trials per scenario.

  • Task Success: For both the Conference and Airport domains, the causal planner consistently outperformed the baselines across all scenarios (\(\text{S1}\), \(\text{S2}\), \(\text{S3}\), and \(\text{S4}\)), confirming parameter recalibration across distinct topologies and crowd dynamics via lightweight tabular CPD adaptation.
  • Proxemic Compliance: In both environments, while all methods operate mainly within the "Public" zone (\(> 3.6\text{ m}\)), the causal planner minimizes personal and social space intrusions by proactively taking detours.
Conference Success-Failure Rate

Success-Failure Rate (Conference)

Conference Human-Robot Proxemic Compliance

Human-Robot Proxemic Compliance (Conference)

(a) Conference

Airport Success-Failure Rate

Success-Failure Rate (Airport)

Airport Human-Robot Proxemic Compliance

Human-Robot Proxemic Compliance (Airport)

(b) Airport

Figure 5: Navigation performance across 200 trials per method in the simulated Conference (a) and Airport (b) environments. (Top) Success and failure distributions for base, social, and causal planners. (Bottom) Human-robot proxemic compliance distributions across Hall's proxemic zones: Intimate (red), Personal (orange), Social (blue), and Public (green)[cite: 7].

Causal vs. Bayesian Prediction

A core advantage of our framework is its ability to disentangle true causation from spurious correlation, which we demonstrate via a confounded inference task: predicting battery consumption (\(L\)). The selected speed (\(S\)) directly affects \(L\), but this relationship is confounded by unexpected obstacles (\(O\)).

  • Standard Bayesian Inference: Estimates depletion probability based on observed speed:
    $$P(L \mid S) = \sum P(L \mid S, O) P(O \mid S)$$
    This approach implicitly introduces bias due to confounding obstacles (\(O\)).
  • Causal Engine (do-operator): Uses the do-operator to predict the causal effect of a hypothetical speed on battery consumption, regardless of any obstacle:
    $$P(L \mid \text{do}(S)) = \sum P(L \mid S, O) P(O)$$
    This adjusts for the confounder \(O\) by marginalizing over its prior distribution rather than its conditional one.
Mean Absolute Error of Bayesian and causal battery predictions

Mean Absolute Error (MAE) of Bayesian and causal battery predictions for varying obstacle probabilities

To evaluate these approaches, we ran simulations where the robot performed a navigation task at a maximum speed \(S_{\text{max}} = 0.5\text{ m/s}\) while varying the probability of encountering unexpected obstacles \(P(O)\) from \(10\%\) to \(50\%\) in steps of \(10\%\).

As shown in evaluation results, the causal prediction consistently yielded a lower Mean Absolute Error (MAE) across all \(P(O)\) values, demonstrating that the causal model is significantly more robust within confounded environments.

Real-World Experiments

The framework was also evaluated in a real-world setting. Two scenarios were staged to evaluate our approach when deployed on a real TIAGo robot: a busy poster session in a corridor and a social event in a kitchen.

Twenty participants took part in the data collection for the learning pipeline of our causal framework.

INB-3floor-map

Scenario 1: Poster Session

Learning

S1-learning

By exploiting the causal model from the simulation experiments, data was collected from the poster session scenario. This was used to update the conditional probability distributions for each node in the causal graph.

S1-map

The map shows the waypoints in the corridor where the poster session took place. This area had a higher people density compared to the other waypoints in the environment.

Inference

Robot Task: Navigate from the starting pink waypoint to the target green waypoint.

S1-task

Non-causal approach (baseline)
Causal approach (Our)

The non-causal approach uses an A* algorithm with a shortest-path heuristic. This directs the robot straight into the crowded corridor where the poster session is taking place. As a result, the robot cannot find enough space to navigate and ultimately fails the task.

In contrast, our causal approach leverages its knowledge of the relationship between the contextual factor time of day (T) = "poster session" and the resulting people density (D). It correctly identifies the corridor as a congested area and chooses a longer but clearer path. This alternative route is ultimately more efficient, as the robot can complete its task, and safer for the people at the poster session.

Scenario 2: Social Event

Learning

S1-learning

Similarly, for the social event scenario, data was collected using the same causal model. The goal was to again update the model's parameters, this time with data from a different real-world interaction.

S2-map

The corresponding map displays the kitchen waypoint where the event was held. As expected, this location shows a higher concentration of people than the other areas in the environment.

Inference

Robot Task: Navigate from the starting pink waypoint to the target green waypoint.

S2-task

Non-causal approach (baseline)
Causal approach (Our)

Without causal reasoning, the baseline approach again fails. It attempts to take the shortest path directly through the kitchen, where the crowd from the social event obstructs its route and causes the task to fail.

The causal approach, however, correctly predicts that a "social event" time slot will lead to high people density (D) in the kitchen. By knowing this, it chooses an alternative route that bypasses the congestion. While physically longer, this path is ultimately safer for the people at the event and more efficient, ensuring the robot successfully completes its task.

Results and Discussions

The real-world experiments on the physical TIAGo robot successfully validated our initial hypotheses, proving that the causal framework is effective, adaptable, and highly efficient.

Rapid Sim-to-Real Adaptation

One of the biggest hurdles in robotics is bridging the sim-to-real gap safely. The standard pipeline requiring causal structure learning from scratch in the real world would have demanded roughly two hours of continuous data collection, plus 6 minutes of computation for the causal discovery.

By transferring the invariant causal structure learned in simulation, we bypassed this bottleneck. We reduced the real-world adaptation to a lightweight parameter update requiring just two minutes of observation per scenario. In practice, this strategy reduced deployment time by 98.4%.

Navigation Performance

During live deployment, our causality-enhanced framework drastically outperformed the standard baseline planner:

  • Completion Rate: Treating the planned paths as sequential subtasks, the baseline planner only managed a ~33% completion rate before getting stuck in the crowds. Our causal planner proactively generated detours, achieving a flawless 100% completion rate in both the poster session and the social event.
  • Proxemic Compliance: The baseline frequently intruded into the participants' "Social" space and occasionally breached "Intimate" distances. By reasoning about congestion causes, our robot consistently maintained a much safer, socially acceptable "Public" distance.

Real-World Proxemics

Data and Computational Efficiency

Beyond just working well, the system is lightweight enough to run practically on standard robotic hardware.

  • Data-Efficient Adaptation: To quantify the efficiency of the sim-to-real parameter update, a sensitivity analysis was performed on the robot's observational data. By subsampling the 3D LiDAR stream, results showed that adapting the model to real-world human dynamics doesn't require massive, high-frequency data. Because human crowd behavior changes relatively slowly, the accuracy of the parameter update stabilises rapidly, reaching satisfactory levels with just a few sensor readings per second.
  • Onboard Speed: Querying the causal engine on the physical robot took an average of just ~0.12 seconds across a 29-node map. This introduces negligible computational overhead, confirming the framework is perfectly suited for real-time, onboard decision-making.

Real-World Sensitivity