Autonomous Valet Parking: Production-Grade Full-Stack System for Underground Garages

Project context

My contributions

As project lead, architected the full-stack AVP system framework; directed planning and control under GNSS denial, completing 500+ in-vehicle road tests with 95%+ parking success.

Automated Valet Parking (AVP) encompasses autonomous stall search, tight-space bay parking, and designated-point summoning. In contrast to infrastructure-heavy vehicle-to-infrastructure (V2I) approaches—which require deploying dense ultra-wideband (UWB) anchors, roadside LiDARs, or dedicated camera networks throughout parking structures—vehicle-centric AVP requires zero structural modifications to parking facilities. By relying entirely on onboard sensors and compute, it delivers superior cross-facility generalization and commercial scalability.

In partnership with Tier-1 smart driving supplier Voyager Tech, we developed and deployed an onboard AVP system utilizing automotive-grade mass-production sensors: 4 surround-view ultra-wide fisheye cameras and 12 ultrasonic sensors. As Project Tech Lead, I directed the top-level system architecture and led algorithm development across the integrated driving-parking planning and control stack, validating GNSS-denied multi-sensor SLAM, corridor dynamic obstacle avoidance, Hybrid A* tight-space maneuvering, and low-speed chassis control on real production test vehicles.

Key Engineering Bottlenecks in Underground Garages

Underground garages represent semi-enclosed, highly constrained operational design domains (ODDs) that present formidable physical and geometric challenges to autonomous systems:

1. Total Satellite Denial & Severe Perceptual Degeneracy (GNSS-Denied & Perceptual Aliasing): Upon entering a basement structure, satellite positioning is immediately lost. Simultaneously, multi-story garages present uniform, highly repetitive architectural features: mirrored structural pillars, polished epoxy resin flooring with severe specular reflections, and featureless concrete walls. Under dim, flickering fluorescent illumination, standard point-feature extractors (such as ORB or SIFT) suffer from catastrophic false associations, precipitating visual odometry drift and scale collapse.

2. Severe Geometric Confinement & Non-Holonomic Kinematic Limits: Primary basement aisle widths typically measure between 2.8 and 3.2 meters, with clearance between concrete pillars occasionally narrowing below 2.5 meters. Standard passenger vehicles feature wheelbases approaching 3 meters and overall lengths near 5 meters. Constrained by non-holonomic Ackermann steering kinematics, the vehicle cannot translate laterally. When executing reverse maneuvers into a standard 2.4m $\times$ 5.2m perpendicular bay, the vehicle must execute multi-point forward-reverse (D/R gear shift) maneuvers under safety margins of less than 30 cm, where minor tracking deviations cause structural collisions.

3. Dynamical & Control Dichotomy Between Cruising and Parking: During corridor cruising, the system requires continuous, smooth motion at 10–15 km/h, with lateral acceleration strictly constrained for passenger comfort. Conversely, during stall insertion, vehicle velocity drops below 3 km/h, while steering actuators must rapidly sweep toward maximum lock within minimal linear displacement. This induces massive static tire scrub torque against the floor. Conventional trajectory-tracking controllers fail under this dual-regime operation, triggering actuator oscillation, severe deadband lag, or tracking overshoot.

4. Multi-Story Spiral Ramps & Dynamic Grade Distortions: Underground garages incorporate steep connecting spiral ramps with grades ranging between 10% and 15% (5.7°–8.5°). During ramp traversal, dynamic suspension compression induces pronounced chassis pitch and roll angles, introducing geometric skew into inverse perspective mapping (IPM). Furthermore, the longitudinal gravitational component severely perturbs velocity feedback loops, risking roll-back during stop-and-go maneuvers on inclines.

To systematically address diverse user requirements, the system supports four operational workflows:

Scroll horizontally to view the remaining columns.

Full-Stack Architecture & Algorithmic Design

To overcome these constraints under onboard embedded compute limits, we developed a tightly coupled, modular processing pipeline encompassing bird’s-eye-view multi-task perception, factor-graph fusion localization, hierarchical planning, and precision chassis actuation.

1. Surround-View Fisheye BEV Multi-Task Perception & Near-Field Shielding

The sensing suite consists of 4 automotive-grade ultra-wide fisheye cameras (each offering $>190^\circ$ horizontal FOV) positioned at the front, rear, and wing mirrors, paired with 12 ultrasonic transceivers.

• Dynamic Pitch-Compensated Inverse Perspective Mapping: To eliminate image warping caused by vehicle acceleration, braking, and ramp slope variations, high-frequency IMU pitch and roll estimates are fed directly into the online homography transformation. This dynamic transformation stabilizes the bird’s-eye view, ensuring projected ground markings and stall boundaries maintain rigorous metric scale.

• Lightweight Shared Backbone with Multi-Task Branched Heads: Operating within embedded thermal envelopes, the perception network adopts a single feature-extractor backbone paired with four decoupled task heads: parking stall head, ground marking head, obstacle detection head, and free space boundary head.

• Ultrasonic Near-Field Time-of-Flight Protection: When approaching obstacles within close range, vehicle blind zones fall outside camera sightlines. Short-range and long-range ultrasonic sensors establish a continuous proximity barrier, using multi-transceiver time-of-flight triangulation to filter multi-path false echoes and provide deterministic hardware-level collision prevention.

2. Factor-Graph Multi-Sensor Fusion Odometry & Hierarchical Mapping

To guarantee drift-free localization across extended GNSS-denied trajectories, we constructed a tightly coupled factor-graph optimization backend:

• Slip-Adaptive Wheel Odometry & IMU Preintegration: On polished epoxy basement surfaces, tire micro-slip frequently occurs during cornering and braking. The fusion filter continuously evaluates consistency between non-driven wheel speeds and IMU yaw rates, adaptively tuning observation covariances to bridge disturbances without pose divergence;
• Visual-Semantic Landmark Constraints & Global Loop Closure: In addition to IMU and odometry factors, the factor graph integrates geometric constraints derived from detected parking slot corners and lane centerlines, triggering full pose-graph relaxation upon loop detection to eliminate accumulated drift;
• Hierarchical Lightweight Map Representation: A local rolling occupancy grid caches dynamic obstacles within a sliding window, while a global topological semantic graph stores corridor centerlines and stall attributes with minimal memory footprints.

3. Hierarchical Cruising & Parking Motion Planning

Recognizing the distinct kinematic scales of aisle navigation versus stall maneuvering, the planning pipeline is decoupled into two specialized stages:

Corridor Cruising: Reference Spline Smoothing & Dynamic Evasion

In aisle transit, the global planner fits continuous-curvature reference corridors using spline curves. The local planner samples polynomial trajectory candidates in the Frenet frame at high frequency, evaluating safety distance, comfort, and lateral acceleration to smoothly evade obstacles or yield to traffic.

Bay Parking: Modified Hybrid A* & Non-Linear Optimal Control

Upon reaching the designated stall entry position, the system transitions to precision parking mode using a two-stage coarse-to-fine framework:

1. Coarse Geometric Search (Modified Hybrid A)*: Search progresses in continuous configuration space under non-holonomic turning radius constraints and distance field heuristics, generating kinematically feasible initial paths within milliseconds;
2. Continuous Trajectory Optimization: Taking the Hybrid A* path as an initial guess, constrained non-linear optimal control enforces kinematic differential equations, steering rate limits, and multi-circle collision avoidance, producing continuous, curvature-smooth trajectories that prevent driveline shudder.

4. Precision Chassis Tracking & Delay Compensation

At low crawl speeds with aggressive steering angles, drive-by-wire latency constitutes the primary source of path deviation:

• Kinematic Preview with Curvature Feedforward: Lateral control employs delay-adaptive preview lookahead, combining curvature feedforward with tracking error feedback to compensate for steering actuator delay;
• Grade-Adaptive Gravitational Feedforward: Longitudinal tracking couples speed feedback with real-time IMU grade estimation, directly compensating for gravity components on steep ramps to prevent roll-back;
• Multi-Tier Hardware Safety Watchdog: High-frequency watchdogs monitor system communications, issuing direct emergency brake requests if ultrasonic sensors detect unexpected close-proximity incursions.

Real-Vehicle Field Verification

Field trials were conducted across diverse commercial underground parking facilities, spanning modern multi-story garages, unlit legacy basements, and complex spiral structures:

• Perpendicular, Parallel, and Slanted Bay Ingress: In tight stalls flanked by parked vehicles with less than 30 cm of lateral margin, the vehicle consistently parked within two to three forward-reverse maneuvers, achieving alignment within 1° of stall markings and symmetric wheel clearance;
• Multi-Floor Spiral Navigation & Dynamic Clearance: Across hundreds of meters of subterranean transit, the vehicle smoothly negotiated consecutive blind corners, executing automatic lane yields when encountering oncoming delivery vans or wandering pedestrians;
• End-to-End Remote Summoning: Production vehicles operated completely unmanned in response to mobile app summon commands, autonomously exiting bays and arriving at ground-level passenger staging areas.

Engineering Takeaways & Architectural Lessons

Automated Valet Parking represents one of the most demanding low-speed autonomous driving domains due to tight geometric tolerances and extreme perceptual degradation. Key takeaways include:

1. Multi-Modal Synergy in Featureless Environments: When individual sensing modalities degrade, combining vehicle kinematics, IMU preintegration, and topological semantic landmarks provides the only viable defense against long-term localization drift.
2. Explicit Physics Constraints in Trajectory Formulation: Kinematic paths must not be treated as purely geometric splines. Embedding actuator rate limits, acceleration limits, and gear-shift costs directly into the optimization formulation is essential for smooth, production-grade tracking.
3. Decoupled Architecture with Deterministic Safety Shields: Decoupling open corridor cruising from tight stall optimization preserves high real-time control frequencies, while dedicated hardware-level ultrasonic emergency braking guarantees operational safety under real-world uncertainties.