A Robotic Chemist: Full-Stack Dual-Arm Mobile Manipulation for Biochemical Experiments

In the emerging paradigm of automated biochemistry and AI for Science, computational chemistry pipelines and large language models (LLMs) can autonomously predict retrosynthetic pathways and design high-throughput reaction conditions. However, the definitive bottleneck in scientific discovery remains closed-loop physical validation in wet laboratories. Traditional specialized automated liquid handling stations are bulky, inflexible, and mechanically plumbed to dedicated microplates, rendering them incapable of operating standard multi-purpose human-scale laboratory instruments (such as chemical fume hoods, rotary evaporators, benchtop centrifuges, analytical balances, and magnetic stirrers). Developing mobile dual-arm robotic chemists with human-scale dexterity and spatial mobility represents a pivotal breakthrough toward fully autonomous biochemical experimentation.

Nevertheless, transitioning collaborative dual-arm robots from structured industrial assembly lines into unconstrained wet-lab environments exposes severe physical and perceptual bottlenecks. The robotic platform must interact with completely transparent, specular glassware, execute precision mechanical assembly in narrow centrifuge slots, suppress dynamic liquid sloshing, and sustain multi-step execution over dozens of interconnected phases.

This project developed a full-stack mobile dual-arm robotic chemist system deployed on an omnidirectional mobile base paired with dual collaborative manipulators. Integrating multi-view geometric vision, kinematic demonstration retargeting, parameterized atomic skills, and explicit physical state contracts, the system accomplishes autonomous end-to-end wet-lab operations: handling fragile test tubes and beakers, microliter fluid transfers, and symmetrical dynamic counterbalancing in high-speed centrifuges.

Real Laboratory Challenges & Architectural Layers

Unlike structured factory assembly lines with rigid metal parts and dedicated fixturing, a biological or chemical laboratory imposes extreme demands on spatial perception, compliant bimanual dexterity, and execution fault tolerance:

Scroll horizontally to view the remaining columns.

Core Technical Solutions: From Physics Priors to Closed-Loop Self-Healing

To guarantee dependable unattended operation at the physical chemistry bench, we developed an integrated processing pipeline spanning multi-modal perception, ergonomic retargeting, smooth fluid control, and contract-governed self-healing.

1. Robust 6D Pose Estimation on Transparent & Specular Glassware

In wet laboratories, beakers, test tubes, and graduated cylinders are predominantly fabricated from transparent borosilicate glass or fused quartz. Structured-light active infrared patterns pass directly through the glass walls, generating complex internal refractive caustics that destroy active depth triangulation and yield massive depth dropouts.

To bypass active depth degradation, we constructed a multi-view boundary contour intersection and parametric CAD constraint pipeline:

• Contour Extraction & Spatial Cone Intersection: Multi-angle high-resolution cameras capture glassware from calibrated vantage points. By back-projecting extracted rim contours into space, the system builds intersecting visual cones whose common intersection resolves the 3D position and surface orientation of the opening;
• Parametric Model Alignment: Extending along the central symmetry axis, the system fits a cylindrical geometric model against valid local point clouds, bounding 6D pose errors within reliable tolerances for compliant grasping.

2. Kinematic Retargeting of Human Demonstration & Singularity Avoidance

Expert chemists handle volumetric pipettes, unscrew threaded caps, and pour liquids with innate smoothness and dexterity. To capture this expert intuition, we deployed an optical-inertial motion-capture demonstration rig. However, collaborative manipulators are strictly bounded by joint limits, link lengths, and kinematic singularities. Blind Cartesian waypoint replication frequently triggers joint velocity spikes or self-collisions against the workbench.

We engineered a task-space kinematic retargeting algorithm with manipulability optimization:

• Invariant Feature Extraction: Isolating inter-vessel relative poses, approach/retreat normal vectors, and characteristic velocities while filtering non-rigid human tremor;
• Manipulability Optimization: Resolving inverse kinematics by projecting manipulability criteria into the null space, ensuring the robot mirrors human bimanual coordination while operating within singularity-free, high-stiffness joint configurations.

3. Precision Alignment and Compliant Centrifuge Assembly

Benchtop centrifuges feature angled cylindrical rotor holes with tight assembly clearances. If the tube axis deviates upon entry, the tube rim binds against the rotor block, risking tube damage or joint overcurrent trips.

We designed a three-stage cascade assembly controller:

1. Coarse Staging: Following mobile base docking, the arms convey the tube to a staging plane directly above the designated rotor slot;
2. Eye-in-Hand Coaxial Servoing: High-resolution wrist cameras perform contour fitting on the rotor aperture, extracting relative lateral offsets and axial tilt, dynamically driving coaxial alignment errors toward rapid convergence;
3. Axial Compliant Insertion & Helical Micro-Search: The manipulator engages axial impedance control, translating vertically downward at low speed while monitoring contact force. If resistance indicates edge binding, the controller executes a localized micro-motion search, allowing the tube to seat cleanly and reliably into the rotor socket.

4. Fluid Motion Stabilization & Meniscus-Monitored Pouring

Transporting open beakers and vials introduces dynamic fluid sloshing, risking chemical spillage and center-of-mass perturbations:

• Smooth Trajectory Generation: Filtering acceleration and deceleration commands with jerk-bounded smooth profiles tailored to container dimensions and fluid fill levels, eliminating resonance during high-speed transit and rapid directional changes;
• Pouring Meniscus Vision Feedback: When pouring reagents between vessels, dual arms coordinate tilt angle while wrist cameras observe the fluid surface tension meniscus and droplet detachment, providing closed-loop control over volumetric transfer.

5. Physical State Contracts & Long-Horizon Self-Healing Execution

In long-horizon biochemical workflows (e.g., retrieve vial, unscrew cap, weigh on balance, pipette reagent, vortex mix, symmetric centrifuge load, spin, separate supernatant), open-loop execution inevitably fails when a single micro-slip occurs.

We formulated a hierarchical self-healing execution engine governed by physical state contracts:

• Hierarchical Task Orchestration: High-level experimental protocols are compiled into a topological graph of task milestones. A Vision-Language-Action agent handles multi-step sequence reasoning, mid-level coordinators instantiate parameterized skill primitives, and low-level controllers maintain high-frequency feedback loops;
• Pre- and Post-Condition Physical Assertions: Every atomic skill is bound to deterministic multi-modal verification contracts (e.g., gripper aperture checks and weight step detection upon grasping, and diametric symmetry verification before centrifuge loading);
• Localized Self-Healing Fallback Branches: If a post-condition verification fails, the agent does not abort the experiment. Instead, it activates targeted local recovery branches (open gripper, retract to safe observation pose, re-scan multi-view geometry, and re-attempt grasp with adapted parameters), substantially increasing overall protocol completion.

Real-Robot Experiments & Demos

The video recordings above show the robot in action at a real wet-lab bench:

• Transparent Labware Retrieval & Fluid Transfer: Autonomous 6D pose extraction of transparent tubes and open beakers under complex lab lighting, followed by smooth bimanual liquid aspiration, transfer, and dispensing without sloshing or surface spatter;
• Precision Centrifuge Workstation Manipulation: Sub-millimeter coaxial insertion of micro-tubes into tight, angled centrifuge rotor sockets, accompanied by symmetric counterbalance loading and smooth retraction out of the fume hood.

Engineering Summary & Technical Takeaways

Mobile dual-arm manipulation in biochemical laboratories represents the definitive proving ground for robotic dexterity bridging deterministic robotics and complex physical reality:

1. Overcoming Optical Degeneracy via Geometric Priors: Transparent and specular glassware defeats naive depth sensors. Fusing 2D sub-pixel boundary geometry, spatial cone intersections, and implicit 3D scene representations provides the prerequisite foundation for reliable physical contact.
2. Embedding Physical Dynamics into Trajectory Generation: Handling delicate glassware and fluid reagents requires multi-body dynamic constraints. Explicitly incorporating input shaping, slosh suppression, and force admittance into low-level controllers enables clean, professional-grade laboratory execution.
3. Explicit State Contracts as the Bedrock of Long Horizons: In open-ended, multi-step scientific workflows, open-loop assumptions are fatal. Arming every atomic manipulation skill with verifiable physical assertions and local recovery branches creates the physical closed loop essential for real-world laboratory automation.
4. Hardware-Software Co-Design for Unstructured Workspaces: In confined fume hood environments, decoupling mobile base global mobility from dual-arm local manipulability is essential for preventing joint limits and wrist singularities.