From Semantic Decisions to Feasible Trajectories: Self-Evolving LLM-Guided Optimal Control for Narrow-Space Parking

SE-LLM-OCP: A Unified Framework Combining LLM Semantic Reasoning with Optimal Control for Robust Autonomous Parking

Zhengbao Yao; Yuanfu Luo; Kehan Xue· 2026· DOI 10.48550/arXiv.2609.24631

The core problem

Autonomous parking in nonconvex and narrow environments remains a challenging problem in robotics and control. Optimal-control methods can explicitly enforce vehicle dynamics and collision constraints, but nonconvexity compromises solver robustness and often leads to failures. Large language models (LLMs) exhibit strong semantic reasoning capabilities, yet directly generating dense trajectories makes it difficult to guarantee physical feasibility. This paper introduces SE-LLM-OCP, a unified framework that leverages LLMs for high-level discrete maneuver decisions while an optimal-control module enforces low-level vehicle dynamics and collision constraints. The key idea is to decompose the parking task into a sequence of short-horizon trajectory-optimization problems, where the LLM proposes sparse maneuver plans and a low-level solver sequentially solves optimal-control problems. If the solver fails, the LLM aggregates failure evidence from the solver and validation stages to guide replanning. Offline, SE-LLM-OCP automatically evolves a structured decision-making knowledge base from scratch, driven by accumulated online failures. The framework is validated in simulation on a car-like

Experimental validation is conducted in simulation on two platforms: a car-like vehicle model and a differential-drive robot. The results show that SE-LLM-OCP enables safer autonomous parking in narrow scenarios compared to baseline methods. The framework successfully transfers the same maneuver representation to a different kinematic platform, indicating generality. Quantitative metrics such as success rate, collision avoidance, and solver robustness are evaluated. The self-evolving knowledge base improves performance over time by learning from online failures. Specific numerical results are not provided in the abstract, but the authors report that the proposed framework outperforms existing approaches in terms of safety and feasibility in nonconvex, narrow environments.

Autonomous parking in nonconvex and narrow environments remains a challenging problem in robotics and control. Optimal-control methods can explicitly enforce vehicle dynamics and collision constraints, but nonconvexity compromises solver robustness and often leads to failures. Large language models (LLMs) exhibit strong semantic reasoning capabilities, yet directly generating dense trajectories makes it difficult to guarantee physical feasibility. This paper introduces SE-LLM-OCP, a unified framework that leverages LLMs for high-level discrete maneuver decisions while an optimal-control module enforces low-level vehicle dynamics and collision constraints. The key idea is to decompose the parking task into a sequence of short-horizon trajectory-optimization problems, where the LLM proposes sparse maneuver plans and a low-level solver sequentially solves optimal-control problems. If the solver fails, the LLM aggregates failure evidence from the solver and validation stages to guide replanning. Offline, SE-LLM-OCP automatically evolves a structured decision-making knowledge base from scratch, driven by accumulated online failures. The framework is validated in simulation on a car-like vehicle model and on a differential-drive robot, demonstrating safer autonomous parking in narrow scenarios and transfer of the same maneuver representation to a different kinematic platform.

SE-LLM-OCP operates in two phases: online and offline. Online, the LLM acts as a high-level planner, proposing sparse maneuver plans that decompose the parking task into a sequence of short-horizon trajectory-optimization problems. These plans are then passed to a low-level optimal-control solver, which sequentially solves optimal-control problems to generate feasible trajectories that respect vehicle dynamics and collision constraints. The optimal-control problem can be formulated as:

The integration of LLMs with optimal control addresses the limitations of each approach individually. LLMs provide semantic reasoning to decompose complex parking tasks into manageable subproblems, while optimal control ensures physical feasibility and constraint satisfaction. The self-evolving knowledge base allows the system to learn from failures without manual intervention, enhancing robustness. The transfer of maneuver representation across platforms suggests that the high-level decision-making is largely platform-agnostic, which is a significant step toward generalizable autonomous parking systems. However, the reliance on simulation and the lack of real-world experiments are limitations. Future work could involve deploying the framework on physical vehicles and extending it to dynamic environments. The taxonomy candidates (Architecture, Cybersecurity, Network, Cryptography) are not directly addressed in this work, but the framework's architecture and learning components may have implications for secure and reliable autonomous systems.

CS practitioners and researchers

Opening member content…