Ph.D. Researcher (Final Year)
I am Chang Nie (聂畅), a final-year Ph.D. candidate in Control Science and Engineering at the School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, advised by Prof. Hesheng Wang. I expect to defend my thesis in January 2027.
My research focuses on Embodied AI, VLM & LLM foundation models, and Robot Learning, specializing in end-to-end robot policies (VLA, WAM), VLM/LLM cognitive reasoning for robotics, AI Agent manipulation architectures, generative world models, and 3D perception, with physical robot deployment. Experienced in full-stack mobile dual-arm system integration and real-robot manipulation. Dedicated to driving closed-loop deployment of embodied foundation models (VLM/LLM/VLA), AI agents, and generative world models in physical environments.
Specializing in end-to-end embodied policies and generative world models, including the training-free skill agent architecture in Teach and Grow, the sound-centric Vision-Sound-Language-Action foundation model in HEAR with causal audio memory and flow matching, and the 4D multi-view generative world model in ERMV for trajectory demonstration editing.
Coupling foundation models and geometric reasoning for robust estimation and scene understanding, including self-supervised multimodal iterative denoising (MID), embedded compact Gaussian SLAM (VCGS-SLAM), multi-camera rig SfM and 4D map aggregation (MRASfM), dynamic object segmentation via deep thinking (MovSAM), and robust consensus estimators (RLSAC, DiffSAC, LNR, DTFI).
Experienced in full-stack mobile dual-arm system integration and real-robot manipulation. Led or orchestrated system frameworks for a mobile dual-arm robotic chemist, heavy industrial material handling and tote unstacking at CATL, production-grade 4D auto-annotation with NETA Auto, on-vehicle Autonomous Valet Parking under GNSS denial, commercial robotic mowers, and hierarchical agent manipulation.
Hierarchical AI Agent architecture for general robot manipulation, compiling sparse demonstrations into reusable Skill Blocks with frozen foundation backbones and physical self-healing.
First to extend Vision-Language-Action (VLA) models to Vision-Sound-Language-Action (VSLA) for manipulation, coupling causal audio memory and flow matching for sound-centric control.
A 4D multi-view generative world model for robot demonstration augmentation, editing spatiotemporal appearances while strictly preserving recorded physical actions for downstream VLA policies.
Introduces a voxelized 3D Gaussian representation for dense visual SLAM on embedded vision platforms, combining redundancy pruning and attribute quantization for compact mapping and tracking.
Adapts multi-camera Structure-from-Motion to autonomous driving scenes, combining rigid multi-camera geometry, road-surface filtering, and multi-run spatiotemporal map aggregation.
Coupling VLM cognitive reasoning with foundation models for single-image moving object segmentation without optical flow.
Presents a self-supervised multimodal iterative denoising framework that learns corruption-stage estimation and residual reversal from noisy data alone without clean ground truth.
Presents LNR, a coarse-to-fine robust multi-model fitting framework that screens minimal sample sets and refines hypothesis neighborhoods to overcome combinatorial explosion under multi-structure overlap.
Proposes a geometry-conditioned diffusion paradigm for sample consensus, iteratively refining candidate minimal sets to achieve robust estimation under extreme outlier ratios.
Reinforcement-learning-guided hypothesis sampling for consensus-based robust estimation under extreme outlier ratios.
DTFI combines local LiDAR-camera feature fusion with IMM-UKF multiple-model filtering for 3D vehicle detection and tracking in highway driving.
Experienced in full-stack mobile dual-arm system integration and real-robot manipulation. Led or architected systems across biochemical mobile dual-arm robots, CATL heavy industrial logistics, NETA Auto 4D auto-annotation, on-vehicle Autonomous Valet Parking (AVP), and commercial robotic mowers.
Dual-Arm Manipulation · Embodied PolicyA mobile dual-arm robotics system combines perception, skill learning, and task planning to automate multistep chemistry laboratory operations.
Skill Planning · Mobile ManipulationThis project investigates mobile dual-arm manipulation with object perception and task planning for autonomous book retrieval, placement, and shelf organization.
Hierarchical Planning · Multimodal PerceptionThis collaboration explores multimodal 3D perception and hierarchical planning to connect high-level task instructions with closed-loop robot control.
6D Pose Estimation · Dual-Arm ControlA mobile dual-arm robot combines 3D vision, navigation, and coordinated manipulation to unstack and transfer material totes between factory stations.
Multi-Camera SfM · Scene AggregationA 4D vision auto-annotation system combines multi-camera 3D reconstruction and multi-run scene aggregation to generate road-element annotations from driving data.
Sensor Fusion · Motion Planning & ControlAn automated valet parking system combines surround-view perception, localization, planning, and vehicle control for autonomous driving and parking in garages.
Terrain Modeling · Multimodal Obstacle DetectionAn outdoor mowing robot combines terrain and obstacle perception, multisensor localization, and coverage planning for autonomous lawn maintenance.
Drive-by-Wire Control · Sensor IntegrationPassenger-vehicle and four-wheel independent-steering AGV platforms integrate perception, localization, planning, and control for low-speed autonomous driving research.
