Industrial Logistics Robotics: Autonomous Dual-Arm Mobile Manipulation for Heavy Tote Handling

Wed, 01 Jan 2025 00:00:00 +0000

In flexible manufacturing and warehouse environments, fixed manipulators have limited operational reach, while conventional AGVs can transport items across stations but lack manipulative capability. Mobile dual-arm robots combine mobility with dual-arm manipulation, enabling autonomous de-stacking and material transfer.

To address these tasks in industrial logistics, we developed an autonomous material handling system based on a wheeled mobile dual-arm manipulator, realizing 6D tote pose estimation, closed-chain coordinated lifting, inter-station docking, and vibration-damped transport.

Project context

CATL Collaboration · Mobile Dual-Arm Material Handling Robot

In flexible manufacturing, fixed arms have limited reach while conventional AGVs lack manipulation capabilities. We developed an autonomous dual-arm mobile platform to tackle black plastic visual degradation, millimeter gap insertions, closed-chain internal stress suppression, and vibration mitigation during transit.

Key Deliverables & Real-World Validation: Bimanual 10 kg payload handling, high-accuracy 3D tote pose estimation (98%+ unstacking success), closed-kinematic-chain impedance decoupling, and robust inter-station transfer

My contributions

Orchestrated the heavy industrial mobile manipulation system; combined VLM perception and 6D pose estimation for 10 kg bimanual payload handling, achieving 98%+ tote unstacking and docking success for high-frequency station transfer.

Key Engineering Bottlenecks in Production

In modern clean-energy battery production and precision electronic assembly plants, intra-logistics material flow relies heavily on standardized industrial totes (tote boxes) weighing between 15 and 25 kg. Traditional automated facilities commonly deploy heavy-duty six-axis industrial arms or gantry robots at fixed workcells. However, these installations occupy critical floor space, require extensive safety fencing, and cannot flexibly reallocate across disparate workstations. Conversely, conventional automated guided vehicles (AGVs) or autonomous mobile robots (AMRs) only provide planar transit, lacking the dexterity required to unstack, manipulate, and precision-seat nested totes. Mobile dual-arm collaborative robots—integrating an omnidirectional mobile platform with dual collaborative arms—bridge the crucial “last ten meters” of agile intra-logistics material handling.

Nevertheless, picking boxes in laboratory mockups differs fundamentally from continuous 24/7 industrial production. Deploying such systems into battery manufacturing plants exposes severe physical and perceptual constraints:

Optical Dropouts on Black Light-Absorbing Plastics & Heavy Surface Scuffing: Factory totes are fabricated from black electrostatic discharge polypropylene (ESD-PP), which strongly absorbs near-infrared (NIR) structured light. Furthermore, continuous friction against automated roller conveyors coats tote surfaces with specular wear tracks and molding blemishes. Commodity industrial 3D cameras projecting active patterns experience severe optical attenuation, resulting in over 40% missing or wildly floating depth returns around critical rim contours and handle pockets.
Dense Stacking with Tight Lateral Clearances (Tight Packing Clearance): Totes are arranged in dense 3-to-5-layer palletized stacks or logistic carts, where lateral gaps between adjacent totes often measure less than 20 mm. End-effector parallel grippers clearing into handle recesses operate under bilateral tolerances of only 3 to 5 mm. A positioning error of mere millimeters causes rigid impact with neighboring containers, inducing pallet tipping and stack collapse.
Closed-Chain Kinematic Internal Stresses & Overcurrent Protection: While single-arm manipulation represents an open kinematic chain, two robotic arms grasping a single rigid 10 kg container instantaneously form a closed kinematic loop. Slight tracking discrepancies between the two arms (even sub-millimeter trajectory drift or millisecond communication jitter) generate destructive internal forces exceeding 300 N across the rigid container. These parasitic forces instantly trip robot joint torque limiters, aborting the process or shearing container handles.
Transit Dynamic Rocking & Chassis Overturning Moments: During long-distance transit across factory epoxy floors at 1.0–1.5 m/s, floor expansion joints, ramp transitions, and emergency deceleration subject high-mounted suspended payloads to severe inertial shocks. If the dual arms maintain an extended cantilevered posture, dynamic rocking produces massive tipping moments that destabilize the mobile base.

System & Algorithmic Architecture

Target Recognition → Station Alignment → Dual-Arm Handling → Closed Loop

High-precision navigation aligns with workstations; 3D vision resolves 6D tote poses; coordinated dual arms execute adaptive grasp and stable transport.

Perception and task preparation

01Observe the sceneImages and depth · tote segmentation · neighbors

02Select the targetUnstacking order · handling mode · target pose

03Navigate and alignApproach · side view · fine pickup position

Tote transfer cycle

04Lift with both armsPose update · bilateral grasp · coordinated lift

05Transport the toteStable hold · travel to the delivery area

06Place and releasePosition adjustment · lower · release and retreat

Action complete → reset state → refresh target → next transferNavigation arrival and manipulation status connect the stages of the task.

Full-Stack Architecture & Algorithmic Design

To surmount these industrial hurdles, we engineered a full-stack robotics architecture uniting multi-modal geometric vision, closed-chain admittance decoupling, and hierarchical docking servoing.

1. Robust 6D Pose Estimation on Weak-Texture Totes & Stacking DAG Reasoning

Rather than relying purely on noisy surface point clouds, we developed a hierarchical pipeline combining 2D edge contour back-projection with surface normal-constrained CAD model alignment:

Multi-Modal Edge Extraction & Spatial Ray Back-Projection: Exploiting high-contrast luminance gradients in 2D RGB streams, sub-pixel edge detection isolates the continuous rectangular boundary of the tote rim and bilateral handle pockets. Ray back-projection lifts these 2D line primitives into 3D camera frustums, anchoring them against sparse non-degraded point clusters to construct an initial geometric wireframe;
CAD-Constrained Coarse-to-Fine Geometric Alignment: Point Pair Features paired with aspect ratio priors generate bounded pose candidates. Joint non-linear optimization over surface normal distances and 3D silhouette consistency bounds 6D pose estimation within millimeter-level precision;
Stacking Contact Graph & Safe Top-Down De-Stacking: From recovered multi-tote spatial poses, the system constructs a Directed Acyclic Graph encoding vertical physical support relationships. The planner strictly enforces top-down topological extraction, executing virtual load-bearing checks before gripper closure to eliminate structural collapse risks.

2. Virtual-Mechanism Leader-Follower Admittance Control & Internal Stress Decoupling

To eliminate internal mechanical stresses during dual-arm rigid payload manipulation, we formulated a virtual linkage leader-follower admittance control framework, upgrading dual-arm coordination from open-loop trajectory tracking to closed-loop force-position compliance:

Contact Wrench Space Decoupling: Wrist-mounted 6-axis force/torque sensors monitor interaction forces. In the virtual linkage frame established at the payload centroid, measured forces are orthogonally decomposed into net accelerating forces driving motion and parasitic internal stresses that pinch or twist the tote;
Asymmetric Leader-Follower Dynamic Admittance Regulation: The primary arm operates as the kinematic leader tracking Cartesian trajectories. The secondary arm executes dynamic admittance control in task space: maintaining pre-tension along the clamping axis to prevent slip while minimizing stiffness along shear and rotational axes, compliantly yielding to cancel destructive internal torques.

3. Hierarchical Base-to-Workstation Alignment & Hand-Eye Visual Servoing

Industrial floor conditions and wheel slip limit mobile base parking repeatability, exceeding tolerance margins for narrow-gap de-stacking.

We implemented a two-stage docking architecture:

Coarse AMR Ingress: The mobile base navigates to nominal docking waypoints via 2D LiDAR SLAM, orienting its heading to maximize dual-arm manipulability relative to the target stack, before locking chassis brakes;
Eye-in-Hand Fine Servoing: Wrist-mounted cameras observe tote handle features and workstation fiducials from close range, solving closed-loop relative transformation errors directly into the manipulator kinematics solver to cancel residual base offsets.

4. Transit Anti-Sway Dynamics & Jerk-Limited Trajectory Profiles

To ensure payload stability during inter-station transit across extended plant floors:

Low-Center-of-Gravity Transport Posture: After unstacking, the arms immediately retract the tote inward against the central chassis axis and descend to a low elevation, doubling chassis overturning stability margins during emergency braking;
S-Curve Jerk-Bounded Velocity Profiling: The mobile base limits jerk throughout motion execution, suppressing start-stop inertial jolts that could dislodge sensitive parts inside the totes;
Whole-Body Deformable Convex Hull Protection: The chassis, deployed arms, and grasped tote are continuously wrapped in dynamic bounding polyhedra, adapting LiDAR safety fields between navigation and manipulation modes.

5. Whole-Body Kinematic Redundancy & 1 kHz Synchronous Bus

The combined mobile base and dual collaborative manipulators form a redundant robotic system. In the control layer, task-priority projection enforces end-effector grasping poses and closed-chain distance constraints, while secondary null-space projection maximizes manipulability and avoids joint limits.

Running a real-time Linux kernel, the system synchronizes chassis drives, joint servomotors, and 6-axis force sensors over industrial bus communication at 1 kHz with microsecond-level cycle jitter, ensuring high-frequency dynamic response.

6. Fault Detection & Self-Healing Recovery State Machine

To mitigate unpredicted factory disturbances:

Grasp Slip Monitoring: During initial vertical lift, the vertical force profile is evaluated against nominal mass. A sudden drop triggers an immediate abort, gently reseating the crate before executing secondary visual re-alignment;
Compliant Contact Back-Off: If lateral contact forces indicate unexpected contact during narrow-gap insertion, the follower arm compliantly backs off along the contact normal, re-estimates orientation via eye-in-hand vision, and re-engages.

Factory Demonstration & Field Verification

Autonomous Dual-Arm Tote Transfer PipelineReal-world demonstration: robot autonomously identifies stacked totes, executes synchronized dual-arm lifting, navigates across workstations, and achieves millimeter-accurate placement.

The footage above captures the autonomous robot executing continuous closed-loop handling inside battery manufacturing facilities:

De-Stacking from Multi-Tier Stacks: Reliably separating dark-colored totes from 4-tier stacks with 15 mm lateral clearances under dynamic shop-floor illumination;
Synchronized Dual-Arm Heavy Transport: Transporting 10 kg payloads while wrist F/T sensors maintain parasitic torques below 2 N·m;
Precision Docking to Conveyor Lines: Traversing 50-meter plant corridors, circumnavigating parked forklifts, and seating totes onto automated roller lines within 2 mm repeatability.

Engineering Summary & Technical Takeaways

Deploying dual-arm mobile manipulators in heavy industrial logistics highlights key architectural principles:

Closed-Chain Admittance as a Fundamental Enabler: In contact-rich manipulation of rigid heavy objects, open-loop position control inevitably induces damaging stresses. Closed-loop force admittance provides the necessary compliance for reliable operation.
Multi-Modal Geometry Beyond Sensory Dropouts: Fusing 2D photometric contours with 3D structural normals overcomes optical failures on dark, specular materials where pure point-cloud registration breaks down.
Layered Decoupling Absorbs Global Tolerances: Macroscopic navigation drift must be isolated and compensated by local visual and force servoing at the end-effector, ensuring operational repeatability across disparate workstations.
Deterministic Bus Synchronization Prevents Limit-Cycle Instability: In high-gain admittance control ($K_d > 500\text{ N/m}$), communication jitter or phase delays exceeding 2 ms inevitably trigger contact chattering and system limit cycles. Enforcing 1 kHz deterministic bus timing with PREEMPT_RT kernel scheduling is as vital to stability as the algorithmic formulation itself.

Dual-Arm Manipulation | Chang Nie