NVIDIA Jetson 上的边缘 LLM：为机器人构建 AI 助手

Edge LLMs on NVIDIA Jetson: Building an AI Assistant for Robots Build a robot assistant architecture in which an edge language model interprets natural-language requests but does not directly control actuators. The LLM produces structured intents; a deterministic command layer validates permissions, parameters, robot state, and safety before publishing ROS 2 commands. What You Will Build By the end of this tutorial, you will have: A clear Jetson/ROS 2 architecture. A working development workspace. A small, testable robotics pipeline. A path for connecting the system to Flutter where applicable. Basic logging, testing, and troubleshooting practices. Prerequisites You should have: An NVIDIA Jetson developer kit or compatible NVIDIA edge platform. A stable Linux/Jetson software environment. Basic Linux terminal knowledge. Basic Python or C++ knowledge. Familiarity with ROS 2 concepts such as nodes, topics, services, and actions. A network connection between the robot computer and development machine. Version note: NVIDIA Jetson, JetPack, CUDA, TensorRT, Isaac ROS, and ROS 2 compatibility changes over time. Check the current NVIDIA support matrix and the documentation for your exact board before installing packages. Do not blindly mix commands from different JetPack/ROS 2 releases. Step 1: Prepare the Jetson Start by confirming the device and installed software: uname -a cat /etc/os-release Enter fullscreen mode Exit fullscreen mode Then update package metadata: sudo apt update Enter fullscreen mode Exit fullscreen mode Keep the base system consistent with the JetPack release supported by your target robotics stack. Step 2: Install and Verify ROS 2 Install the ROS 2 distribution supported by your Jetson/Isaac ROS combination. After installation, source ROS 2: source /opt/ros//setup.bash Enter fullscreen mode Exit fullscreen mode Verify that ROS 2 is available: ros2 --help Enter fullscreen mode Exit fullscreen mode Add the source command to your shell configuration if appropriate: echo "source /opt/ros//setup.bash" >> ~/.bashrc source ~/.bashrc Enter fullscreen mode Exit fullscreen mode Step 3: Create a ROS 2 Workspace mkdir -p ~/robot_ws/src cd ~/robot_ws colcon build source install/setup.bash Enter fullscreen mode Exit fullscreen mode A typical workspace becomes: robot_ws/ ├── src/ ├── build/ ├── install/ └── log/ Enter fullscreen mode Exit fullscreen mode Step 4: Create a Package For Python: cd ~/robot_ws/src ros2 pkg create --build-type ament_python robot_ai_demo Enter fullscreen mode Exit fullscreen mode For C++: ros2 pkg create --build-type ament_cmake robot_ai_demo_cpp Enter fullscreen mode Exit fullscreen mode Choose the language that best matches the latency and integration requirements of your application. Step 5: Understand the Data Flow A production robot should separate responsibilities. Sensors | v ROS 2 Drivers | v Perception / Localization | v Decision / Mission Logic | v Safety Layer | v Motor Controller Enter fullscreen mode Exit fullscreen mode For a Flutter operator application: Flutter | HTTPS / WebSocket | Robot Gateway | ROS 2 | Jetson | Robot Enter fullscreen mode Exit fullscreen mode The Flutter application should normally communicate with a controlled gateway instead of directly exposing the ROS graph to the public internet. Step 6: Publish a Simple ROS 2 Message Create a small publisher and subscriber, then build the workspace: cd ~/robot_ws colcon build --symlink-install source install/setup.bash Enter fullscreen mode Exit fullscreen mode Run the publisher: ros2 run robot_ai_demo publisher Enter fullscreen mode Exit fullscreen mode In another terminal: source ~/robot_ws/install/setup.bash ros2 topic list ros2 topic echo /robot_status Enter fullscreen mode Exit fullscreen mode This simple test proves that your ROS 2 environment is functioning before you add cameras, AI models, or motor controllers. Step 7: Add the Main AI/Robot Component For this tutorial, the main component is conceptually one of: Camera and object detector LiDAR and navigation stack TensorRT inference node Isaac ROS perception node Robot telemetry collector Fleet gateway Voice/LLM intent service Keep this component independent from the UI. Publish structured ROS 2 messages instead of UI-specific data. Example: camera/image | v object_detector | v /objects | +----> decision_node | +----> telemetry_gateway Enter fullscreen mode Exit fullscreen mode Step 8: Add Logging and Diagnostics At minimum, log: Node startup/shutdown. Sensor connection failures. Inference errors. Network disconnects. Safety-state changes. Command acknowledgements. Processing latency. Useful ROS 2 commands include: ros2 node list ros2 topic list ros2 topic info /robot_status ros2 topic hz /robot_status Enter fullscreen mode Exit fullscreen mode Step 9: Add a Safety Layer Never allow an AI model or remote UI to directly bypass safety logic. A simple command path should be: User/AI Intent | v Command Validation | v Robot State Check | v Safety Rules | v ROS 2 Command Enter fullscreen mode Exit fullscreen mode Examples of safety rules: Stop if communication heartbeat expires. Stop if a critical sensor fails. Reject invalid velocity ranges. Reject commands while the robot is in an unsafe state. Give emergency stop the highest priority. Step 10: Connect Flutter When Applicable For Flutter projects, expose a small API such as: GET /api/robot/status GET /api/robot/telemetry POST /api/robot/command WS /ws/robot Enter fullscreen mode Exit fullscreen mode Example WebSocket payload: { "type": "command", "command": "stop", "sequence": 1024 } Enter fullscreen mode Exit fullscreen mode Flutter can then maintain: ConnectionState RobotState TelemetryState MissionState AlertState Enter fullscreen mode Exit fullscreen mode Use BLoC, Riverpod, or another state-management approach to keep network events separate from presentation. Step 11: Test the System Test one layer at a time. ROS 2 ros2 topic list ros2 topic echo /robot_status Enter fullscreen mode Exit fullscreen mode AI Measure: Model load time. Preprocessing time. Inference latency. Postprocessing time. End-to-end latency. Network Test: Normal connection. Temporary disconnect. Reconnect. Duplicate messages. Delayed messages. Safety Verify: Emergency stop. Heartbeat timeout. Sensor failure. Invalid command. Jetson restart. Step 12: Optimize for Jetson Do not optimize before measuring. Record a baseline and then investigate: CPU utilization. GPU utilization. Memory consumption. Temperature. Power mode. Camera pipeline latency. AI inference latency. ROS 2 message latency. For NVIDIA-accelerated applications, investigate TensorRT, DeepStream, and Isaac ROS where they match the workload. Step 13: Make the Deployment Reproducible Record: Jetson model: JetPack: CUDA: TensorRT: ROS 2: Isaac ROS: Python: Model: Camera: LiDAR: Enter fullscreen mode Exit fullscreen mode For serious deployments, containerize the application and keep configuration separate from application code. Step 14: Troubleshooting ROS 2 command not found source /opt/ros//setup.bash Enter fullscreen mode Exit fullscreen mode Package not found source ~/robot_ws/install/setup.bash ros2 pkg list | grep robot Enter fullscreen mode Exit fullscreen mode Topic has no data Check: ros2 topic list ros2 topic info /your_topic ros2 topic hz /your_topic Enter fullscreen mode Exit fullscreen mode Then verify that the sensor publisher is actually running. AI inference is too slow Profile the complete pipeline. Do not assume the neural network is the only bottleneck. Camera conversion, memory copies, preprocessing, ROS serialization, and postprocessing can all contribute significant latency. Flutter is disconnected Implement: reconnect with backoff, heartbeat messages, connection state, command acknowledgement, timeout handling. Step 15: Production Checklist Before deploying a robot, verify: [ ] Hardware/software versions are documented. [ ] ROS 2 nodes restart safely. [ ] Sensor failures are detected. [ ] Commands are validated. [ ] Emergency stop works independently. [ ] Network loss causes a safe state. [ ] AI inference is monitored. [ ] Logs are retained. [ ] Telemetry is available. [ ] The deployment can be reproduced. Conclusion NVIDIA Jetson is most useful when it is treated as an edge-computing platform inside a larger robotics architecture rather than simply as a small Linux computer. ROS 2 provides the communication and modularity layer, while NVIDIA acceleration can handle demanding perception workloads. For Flutter-based robotics applications, a gateway between Flutter and ROS 2 creates a clean separation: the mobile application focuses on user experience, while Jetson and ROS 2 remain responsible for robot-side computation. Useful Links NVIDIA Jetson Developer Resources: https://developer.nvidia.com/embedded/learn/getting-started-jetson NVIDIA JetPack: https://developer.nvidia.com/embedded/jetpack NVIDIA Isaac ROS: https://developer.nvidia.com/isaac/ros ROS 2 Documentation: https://docs.ros.org/ NVIDIA Developer Forums: https://forums.developer.nvidia.com/c/robotics-edge-computing/jetson-systems/jetson-projects/78 V-Modal Website: www.v-modal.com V-Modal Flutter SDK: https://github.com/v-modal/vmodal_sdk_flutter V-Modal Android SDK: https://github.com/v-modal/vmodal_sdk_android V-Modal Discord: https://discord.gg/K72z28KUx V-Modal Reddit: https://www.reddit.com/r/v_modal/