Natural-Language Task Planning and Execution Validation in Simulation
01
Project Background
Tasks involving mobile robots and inspection drones are well suited to initial simulation checks of whether an agent understands the goal and invokes the correct actions. We also retain failure cases involving ambiguous instructions, unit errors, and unauthorized actions, without presenting code generation as production-grade reliability.
02
Technical Approach
Scene objects, permitted actions, and task constraints are provided to the planning layer, while a controlled execution layer checks parameters and permissions. Natural-language goals are first converted into evaluable conditions; generated code is not sent directly to physical equipment.
03
Implementation Process
Start with fixed objects and a limited task scope, including normal instructions, ambiguous instructions, and unexecutable tasks. Check each plan for nonexistent objects, incorrect units, and unauthorized actions. Retain execution traces and tool responses for replay-based analysis instead of masking failures by only changing prompts.
04
Deliverables
Deliverables include task and tool contracts, simulation execution interfaces, task evaluation examples, and planning and execution logs. Each identifies its version, inputs, operating environment, and corresponding acceptance steps.
05
Acceptance Approach
Acceptance focuses on goal completion, error handling, and permission boundaries. Model-generated plans still require verification. External model services, invocation costs, and the scope of data processing are specified at handover.
06
Implementation Outcomes
Make task-planning errors traceable.
The path from task intent to a testable prototype is shortened, and failed steps can be located through replay instead of seeing only a completion message.
Evaluation compares first-pass results, the number of human revisions, and troubleshooting time on a fixed task set, with safety-constraint violations recorded separately.