Warwick Collaboration: Decoupled Hierarchical Embodied AI and Agentic Robotics for Long-Horizon Tasks

Mon, 21 Sep 2026 00:00:00 +0000

In embodied robotic manipulation within open, unstructured environments, robots face profound physical uncertainty. Whether performing multi-step industrial assembly, automated laboratory specimen preparation, or parcel sorting in semi-structured warehouses, practical missions frequently encompass dozens of consecutive physical interactions. In such extended horizons, minor unmodelled errors introduced during any individual action compound exponentially, precipitating global task failure if left unmitigated.

This project is an ongoing research collaboration with the robotics and sensor systems group at the University of Warwick. We focus on developing a hierarchical embodied AI and closed-loop AI Agent architecture designed specifically for long-horizon physical manipulation. I am responsible for designing and deploying the full-stack software and hardware robotic system, bridging multimodal perception, high-level task decomposition, metric geometric motion synthesis, and low-level Cartesian impedance control to establish an autonomous operating pipeline with physical self-healing capabilities.

Project context

University of Warwick · Professor Sai Gu

In complex real-world environments, pure end-to-end models struggle to balance multi-minute logical consistency with high-frequency geometric control. We develop a decoupled hierarchical framework where high-level agents orchestrate subgoals while low-level controllers handle geometry and contact dynamics.

Projected Objectives & Research Focus: Closed-loop 8–15 step physical embodied manipulation; targeting >84% end-to-end success, >70% localized recovery, and <3 mm 3D localization error under multimodal fusion.

My contributions

As project lead, formulated the hierarchical agent manipulation architecture; for 8–15 step long-horizon tasks, established target objectives to elevate end-to-end task success rate from 35% to >84%, with >70% autonomous recovery from localized physical anomalies.

Physical Brittleness and the Challenge of Compound Error Propagation

In embodied manipulation, the fundamental barrier to scaling long-horizon tasks lies in compound error propagation. Consider a representative physical task composed of 10 sequential manipulation stages (e.g., identifying a chemical vial on a cluttered bench, uncapping the vessel, aspirating a reagent, re-threading and torquing the cap, transferring the vessel to an assay tray, and stowing it in an incubator). Even if every individual atomic skill possesses an isolated execution reliability of 90%, under open-loop execution the cumulative reliability decays exponentially to roughly 35% across 10 stages, and even further as steps increase. This compounding decay demonstrates that without explicit physical verification contracts and bounded local recovery mechanisms, open-loop robotic systems are virtually guaranteed to fail midway. Contemporary research exhibits two prevalent paradigms, both of which reveal distinct physical vulnerabilities when applied to real-world hardware:

Vulnerabilities of Pure End-to-End Visuomotor Policies (VLA Models): End-to-end policies excel at reactive short-horizon reflex behaviors within bounded training distributions. However, across operational timelines spanning several minutes and dozens of diverse actions, latent vector representations suffer from severe temporal drift. Critically, contemporary vision-language-action models operate at inference rates between 5 Hz and 20 Hz, which is fundamentally inadequate for the hundreds-of-hertz control loops required for stable contact dynamics. Furthermore, when unexpected mechanical binding or slippage occurs, end-to-end models lack explicit mechanisms for backtracking, fault localization, and structured error recovery.
Geometric Hallucinations in Direct LLM Waypoint Generation (LLM-as-Planner): Inquiring large language models to directly generate end-effector SE(3) waypoints or joint trajectories yields semantic commonsense but lacks metric spatial reasoning and inverse kinematics (IK) feasibility awareness. The resulting trajectories routinely violate joint velocity or torque limits, penetrate obstacle envelopes, or command configurations within singular regions. Crucially, direct text generation fosters an open-loop hallucination that issuing an action directive is physically equivalent to successfully completing the interaction.

To overcome these structural limitations, our architectural paradigm enforces hierarchical decoupling between task planning and motion synthesis, combined with strict closed-loop physical state verification. High-level symbolic reasoning is isolated within the AI Agent, high-frequency trajectory tracking and contact compliance are governed by deterministic real-time controllers, and multimodal sensing provides empirical ground truth across every execution transition.

Research architecture

3D perception → hierarchical decisions → robot execution

Task planning and motion control coordinate through observations and execution state.

Sensing and spatial understanding

01Vision and mmWaveComplementary observations · multimodal sensing

023D scene representationObjects · geometry · spatial relations

03Language and temporal groundingInstruction meaning · relevant events

AI Agent-based robot system

04High-level task decisionsInterpret instructions · organize subgoals

05Low-level executionRobot skills · motion control

06Feedback and updatesObserved outcomes · continued planning

My responsibility: build the robot system linking perception, planning, and executionIntegrate the collaboration's research components in a shared robot task loop.

Three-Tier Decoupled Embodied AI Architecture

To achieve semantic adaptability while preserving physical execution determinism, the system is organized into three strictly partitioned hierarchical layers:

1. Level 3: Semantic and Long-Horizon Task Planning Layer (AI Agent Brain, 0.5 Hz - 2 Hz)

Responsible for parsing open-ended natural language directives or complex procedural task objectives into a directed acyclic task graph (DAG) or a reactive behavior tree:

The agent operates via an onboard language model adapter, consuming symbolic state inputs comprising the robot’s current status, a spatial-semantic scene graph, and temporal execution history;
Candidate agent outputs are strictly restricted to a registered library of formally verified parameterized atomic skills (such as ApproachObject, AlignEndEffector, ExecutePinchGrasp, VerifyEngagement, LinearExtract);
Level 3 produces no direct motor current or joint torque commands; its decision manifold resides entirely within the domain of task sequencing, skill invocation, and verification scheduling.

2. Level 2: Metric-Geometric Skill Synthesis Layer (Skill Synthesis & Trajectory Generator, 10 Hz - 50 Hz)

Translates parameterized skill specifications from Level 3 into metric SE(3) Cartesian trajectories and joint space paths:

Ingesting a dynamic Euclidean Signed Distance Field (ESDF) maintained from onboard depth cameras and radar point clouds, the layer invokes OMPL and the Pinocchio rigid-body dynamics library to compute time-parameterized polynomial trajectories compliant with joint velocity and acceleration bounds;
For contact-intensive phases (such as drawer sliding, cap threading, or peg-in-hole insertion), the planner seamlessly switches to Cartesian linear interpolation and helical micro-search primitives, continuously auditing kinematic continuity and self-collision clearance.

3. Level 1: High-Frequency Compliant Execution Layer (Deterministic Real-Time Control, 500 Hz - 1000 Hz)

Serves as the low-level hard real-time foundation, driving joint actuators over deterministic buses while governing physical contact forces:

Executes Cartesian space impedance control: in free-space transit, stiffness is elevated to guarantee tracking accuracy; during physical contact, virtual stiffness is attenuated along contact axes to provide compliant deflection, absorbing positional misalignments;
Actuator currents and 6-axis wrist force-torque sensors are queried at 1 kHz against hardware-level safety thresholds. In the event of an anomalous collision wrench, the controller triggers a controlled stop within 2 milliseconds, preventing hardware damage.

Multimodal 3D Spatial Perception: Visual-mmWave Radar Fusion

In unstructured operational environments, relying exclusively on optical vision exposes the robotic system to catastrophic sensor degradation. In backlit conditions, structured-light and active stereo depth sensors suffer pattern saturation, inducing substantial depth dropouts. On transparent glassware or specular metallic tooling, specular reflections cause false point-cloud shifts of several centimeters. Furthermore, airborne dust or aerosol suspensions completely attenuate optical propagation.

Leveraging Warwick’s extensive expertise in advanced electromagnetic sensing and industrial digitalization, we developed a synchronized RGB-D and 60-77 GHz FMCW Millimeter-Wave (mmWave) Radar spatial perception architecture:

1. Complementary Physical Sensing and Microsecond Synchronization

Optical Sensors: Deliver dense, megapixel-resolution RGB imagery with fine contour boundaries and rich semantic segmentations, capturing micro-scale visual geometry, yet remain highly vulnerable to ambient illumination and optical occlusions;
Millimeter-Wave Radar: Exploits millimeter-range electromagnetic wavelengths that penetrate dust, smoke, and moisture, remaining impervious to illumination extremes; FMCW radar directly returns radial distance and high-precision Doppler velocities, though it exhibits coarse angular resolution and susceptibility to multipath ghost reflections;
Spatial-Temporal Calibration: Utilizing specialized retro-reflective geometric calibration targets, we solve for the rigid spatial extrinsic transformation matrix; hardware synchronization utilizes a Pulse-Per-Second (PPS) reference signal, constraining temporal projection jitter below 1 millisecond.

2. Kinematic Clutter Filtering and Log-Odds Volumetric Occupancy Mapping

To eliminate multipath clutter, the perceptual front-end applies Constant False Alarm Rate (CFAR) adaptive thresholding alongside Doppler velocity distribution analysis, isolating static structural landmarks from moving obstacles.

For volumetric mapping, we maintain a 3D log-odds temporal voxel occupancy grid, fusing optical depth projections with radar backscatter cross-sections via a recursive Bayesian framework. Through this design, when optical cameras experience blinding glare, radar range returns maintain high occupancy confidence, preventing the manipulator from colliding with undetected obstacles; conversely, when radar returns exhibit angular blooming near metallic edges, optical high-resolution masks delineate crisp object surfaces, establishing an all-weather geometric foundation.

Video Temporal Grounding and Physical State Verification

In long-horizon execution, verifying that a preceding manipulation stage has successfully attained its objective is the cornerstone of closed-loop robustness. If the system relies purely on elapsed trajectory time or encoder confirmations to assert success, undetected slippage or jamming will cascade through downstream tasks.

We incorporate a video-semantic grounding mechanism: throughout execution, the system maintains a circular video buffer $\mathcal{V}_{[t-\Delta t, t]}$ spanning window length $\Delta t$. The high-level agent establishes explicit condition descriptors for each sub-action (such as “vessel base lifted completely clear of the tabletop” or “slider rail pulled to terminal mechanical stop”). Utilizing cross-modal cross-attention layers, the model grounds semantic milestones along the temporal video timeline, extracting verifiable timestamped evidence of physical state transitions.

To guard against optical occlusions, every atomic skill is bound to rigorous multi-modal physical assertions:

Visual Assertions: The 3D bounding box centroid of the target object must exhibit a rigid vertical translation exceeding threshold $\Delta z > h_{\text{thresh}}$, accompanied by the visual reappearance of the supporting tabletop texture;
Force-Dynamic Assertions: The wrist 6-axis force-torque sensor must register a vertical gravitational wrench increase $\Delta F_z$ concordant with the object’s calibrated mass ($ \Delta F_z \approx m_{\text{object}} \cdot g \pm \epsilon_f $), while internal gripper finger encoders confirm a grasp aperture matching the item’s metric profile;
Deterministic Gating: Only when both visual geometric evidence and force metrics concurrently satisfy verification criteria does the skill node return a SUCCESS token to the behavior tree. If either assertion fails, execution is immediately intercepted.

3. Hierarchical Bounded Error Recovery

Upon verification failure, the system avoids indiscriminate aborts or blind task restarts. Instead, it triggers localized bounded recovery trees based on the failure classification:

Incipient Slip and Empty Grasps: If force sensing indicates an unweighted gripper, slippage is identified. The behavior tree executes a local re-observation and grasp re-estimation routine, adjusting gripper approach yaw and re-attempting acquisition;
Mechanical Binding and Over-Torque: During insertion or mating, if resistive wrenches surge prior to reaching target depth, alignment binding is diagnosed. The controller executes a structured sequence: retract 15 mm, apply micro-scale dither perturbations to find the slot axis, and resume compliant low-stiffness insertion;
Bounded Escalation and Safe Holding: If repeated recovery attempts reach a pre-defined threshold (e.g., 3 attempts) without resolution, the system gently places the object into a designated buffer dock, decrements object confidence within the scene graph, logs time-series diagnostic data, and switches to an alternate independent subtask, preserving overarching operational uptime.

System Integration & Projected Target Objectives

The system architecture combines a visual-mmWave multimodal perception rig, a hierarchical behavior tree engine, and a 6-DOF compliant manipulator, establishing clear target benchmarks across 8-to-15-step sequential physical manipulation tasks:

Long-Horizon End-to-End Target Completion: By enforcing physical verification contracts and bounded error recovery, the framework targets elevating end-to-end multi-step task success from an open-loop 35% baseline to >84%, with an objective of autonomously resolving >70% of localized execution anomalies without human intervention;
Perception Robustness Objectives under Degraded Conditions: In harsh illumination and airborne particulate environments, the visual-radar fusion pipeline aims to constrain 3D boundary and distance tracking RMSE within 3 millimeters;
Geometric and Kinematic Consistency: The Level 2 motion planning layer is architected to guarantee 100% inverse kinematics solvability and boundary clearance across all agent-synthesized directives, eliminating joint limit collisions.

Engineering Lessons and Technical Takeaways

Developing this hierarchical embodied architecture yielded several fundamental engineering insights into robotic deployment:

Embodied AI Is Fundamentally an Engineering Discipline in Managing Uncertainty: In purely virtual or simulation environments, action outcomes are conveniently abstracted into deterministic transitions. In the physical world, mechanical tolerances, surface friction, thermal drift, and lighting shifts are persistent realities. A robust embodied architecture succeeds not by naively assuming a perfect environment, but by adopting a defensive posture that rigorously audits physical states after every contact and provides deterministic recovery pathways when deviations occur.
Language Models Excel at Symbolic Reasoning; Continuous Mechanics Belongs to Physics-Based Control: Large language models possess unmatched breadth in semantic parsing and long-horizon task structuring. However, forcing them to output high-frequency continuous joint commands is structurally unsound. Positioning the language model as a high-level strategist while assigning low-level trajectory generation, collision checking, and impedance control to deterministic robotics libraries achieves the optimal balance between open-world intelligence and physical safety.
Sensor Diversity Requires Orthogonal Physical Modalities: Redundantly stacking identical optical sensors fails to resolve structural weaknesses such as ambient backlighting, optical glare, or particulate obscuration. Incorporating millimeter-wave radar introduces an orthogonal electromagnetic sensing modality that compensates for fundamental optical blind spots, raising the lower bound of perceptual reliability in harsh, real-world manipulation environments.

Multimodal Sensing | Chang Nie