Skip to main content

Capstone Humanoid

Capstone Project: Full Humanoid Robotics System Integration​

This capstone project is the culmination of your learning, challenging you to integrate knowledge and skills from across the textbook to develop a functional humanoid robotics system. Your task is to design, implement, and demonstrate a humanoid robot capable of performing a complex, multi-step task that involves perception, navigation, manipulation, and human-robot interaction. This project will require you to combine ROS 2, simulation (Gazebo or Isaac Sim), AI models (VLA, LLM planning), and potentially physical hardware if available.

You will define a high-level task (e.g., "fetch a specific object from another room and bring it to a human"), then break it down into sub-tasks. For each sub-task, you will implement the necessary robotic capabilities: using VSLAM for localization, Nav2 for path planning, Isaac ROS for object perception, and a custom controller for manipulation. The project should culminate in a demonstration where the humanoid robot autonomously or semi-autonomously executes the task, ideally responding to natural language commands. This project emphasizes system integration, problem-solving, and robust error handling.

Project Requirements​

  • Task Definition: Clearly define a complex, multi-step task for the humanoid robot.
  • System Architecture: Design a system architecture diagram outlining all ROS 2 nodes, communication topics/services, and AI components.
  • Perception: Implement robust object detection and/or environmental mapping (e.g., using Isaac ROS or VSLAM).
  • Navigation: Enable the robot to navigate to target locations and avoid obstacles (e.g., using Nav2).
  • Manipulation: Implement grasping and placement of objects.
  • Human-Robot Interaction: Allow for natural language commands (e.g., via GPT/Whisper integration).
  • Simulation & Testing: Develop and test your system in a simulation environment (Gazebo or Isaac Sim).
  • Demonstration: Provide a video demonstration of the robot successfully performing the task.
  • Documentation: Submit a comprehensive report including:
    • Task description and motivation.
    • System design and architecture.
    • Implementation details for each component.
    • Challenges faced and solutions implemented.
    • Results, analysis, and future work.

Diagram: Capstone System Overview​

+-----------+    Voice/Text Cmds    +-----------+
| Human |----------------------->| LLM/ |
| Operator | | VLA Planner|
+-----------+ +-----^-----+
|
v
+---------------------------------+
| High-Level Task Orchestration |
| (Sub-task Sequence, Logic) |
+---------------------------------+
|
v
+-----------+ ROS 2 / Isaac ROS / Nav2 +-----------+
| Simulation|-------------------------------->| Humanoid |
| (Gazebo/ | (Sensor Data, Control Cmds) | Robot |
| Isaac Sim)|<--------------------------------| (Perception,|
+-----------+ | Navigation,|
| Manipulation)|
+-----------+

Summary​

This capstone project requires integrating a full humanoid robotics system to perform a complex, multi-step task. It assesses your ability to combine perception, navigation, manipulation, and HRI, culminating in a functional and well-documented robot demonstration.