In place of a final exam, you will work on a group project. The purpose of this project is to allow you to explore robotics in an independent, free-form fashion. You will design the project tasks and implement algorithms to get a robot of your choice to complete a pick and place task, with minimal guidance and help from the course staff.
Your goal is to come up with your own Pick and Place Challenge / Mock Competition. “Pick and Place” is a canonical task for robot manipulators: tasks often consist of objects in the world that must be detected, located, grasped / picked, moved to a desired position, and placed. Such tasks are common place in assembly lines, but remains an open challenge for more general use cases like warehouse management and automatic recycling sorting (see Amazon’s Pick and Place Challenge).
You will be tasked with coming up with a fun problem setting (e.g., what objects will you be working with? why is it an important (or fun) application?) and creating a dynamic simulation (i.e., a simulation of real physics like Gazebo), in which at least one robot effectively executes an interesting pick and place task. You may use your preferred simulator and language, and you are encouraged to use existing robot and object models. All of the code that controls the robot (e.g., forward kinematics, inverse kinematics, path planning, etc.) must be your own; however, you are encouraged to explore the open-source robotics community for ideas and to build on previous work, as long as you give credit to code bases you use or take inspiration from.
You are free to explore simulation options (e.g., Gazebo, Webots, CoppeliaSim), but we encourage you to use Gazebo, an open-source simulator that integrates with ROS. We will provide a simple Gazebo simulation that mirrors the labs may be used as a starting point. Note that the course staff will help you conceptually design your system and help you with the fundamentals, but they will not provide debugging support. This assignment is meant to encourage exploration and help you practice unguided project development, and you will be assessed and graded with that in mind.
Example Projects from Previous Semesters
Note that expectations may change between semesters.
Your project grade will be computed as follows:
– 20% project updates — detailed below. Each update is equally weighed.
– 20% final project video / presentation
– 50% final report (due during finals)
– 10% peer evaluation on perceived difficulty and teamwork
Rubrics for the presentation and report will be provided closer to the deadline.
For each project update, you’ll be asked to submit (1) your current codebase for the TAs to run, (2) a well-written readme, (3) a short description of your progress, and (4) a video uploaded to youtube demonstrating the deliverable.
You’ll be graded on the following (if applicable): (1) Was the update submitted on time? (2) Is your codebase and readme in good shape? (3) Does your video show the deliverable?
The deadlines for submission will be kept up to date on the Important Dates page.
Project Update 0
Form a team and tell us about a task you’d like your robot to perform.
Deliverable: Submit a 1pg pdf on Gradescope with your team name, list of team members, and one paragraph about what pick and place challenge you’d like to implement.
Project Update 1
Show us that you can interface with your simulator, that you can move the robot, and that you can access some sensor measurements. Check to make sure that your original project plan seems testable in the simulator environment you’ve chosen.
Deliverable: Submit a max 2pg pdf on Gradescope with your team name, list of team members, and an update to your project based on what you think is feasible. Provide a link to (1) your current codebase for the TAs to review, including a well-written readme, and (2) a video uploaded to youtube demonstrating that you can use your simulator, move the robots, and access the sensors.
Project Update 2 (recently combined with update 3!)
Demonstrate that you can detect the objects of interest and understand the forward kinematics of your robot. You should be close to integrating inverse kinematics, decision making, and planning. Try to be able to pick and place at least one item. Update us on the status of your project and let us know what the major roadblocks are for you achieving your goal.
Deliverable: Submit a max 2pg pdf on Gradescope with your team name, list of team members, and an update to your project based on what progress you have made for detecting the objects you will be picking up as well as a description of the model of your robot (i.e., can you describe the kinematics of your robot?). You should be able to move your robot to the object of interest and start working on grasping. Provide a link to (1) your current codebase for the TAs to review, including a well-written readme, and (2) a video uploaded to youtube demonstrating that you have made progress towards your goal.
Project Update 3 (see new update 2 deliverables) Demonstrate robot motion with integration of some inverse kinematics, decision making, and planning. Try to be able to pick and place at least one item. Update us on the status of your project and let us know what the major roadblocks are for you achieving your goal.
Deliverable: Submit a max 2pg pdf on Gradescope with your team name, list of team members, and an update to your project based on what progress you have made. You should be able to move your robot to the object of interest and start working on grasping. Provide a link to (1) your current codebase for the TAs to review, including a well-written readme, and (2) a video uploaded to youtube demonstrating that you have made progress towards your goal.
In class presentations
During the last full week of class, you and your team will show a three-minute video to the class of your project along with a short elevator pitch for your proposed challenge.
In the video, you must (1) describe/motivate your problem, (2) show a quick demo of what you have thus far, (3) give a short description of challenges / what you plan to finish (as applicable), and (4) a blooper (if time)
You will be graded by your peers on the following criteria:
- Difficulty rating for the group size
- Video quality
- Project creativity
Submission link is on the Important Dates tab.
Final report *due 12/17 at 8pm*
Summarize what you accomplished, the technical details that power your robot, and what you learned.
Your report will be submitted via Gradescope. You will be asked to submit typed responses and/or files for the following questions:
- Team and Links: Provide a list of your team members and NetIDs; your team name; link to your Github; and link to your final video
- Introduction: Provide brief motivation for your pick and place application, conveying why this is an important (or fun) task. Also provide a brief summary of the approaches you used.
- Task Description: Give a detailed description of your task and robot pipeline. Include specifications and design choices you made to make the task easier. Provide a block diagram that shows the different components of your system (e.g., perception/sensing, planning, grasping/picking). Describe how you implemented all of the different components in your system (e.g., how did you model your system? how did you implement inverse kinematics? what planner did you use?). If you built off of existing codebases, please give credit to the original authors.
- Experimental Setup: Carefully describe your simulator setup, both in terms of the environment design and the robot. Summarize how you generated scenarios to test and gather results (e.g., did you randomly generate blocks to be sorted? how did you control for difficulty).
- Data and Results: Describe how you measured the success of your robot (i.e., what metrics did you use?). Analyze the data you collected, providing quantitative metrics (e.g., success rates, error analysis, etc), and qualitative examples of success and/or failures you encountered (e.g., describe the behavior and show a few examples). Provide at least one plot illustrating your robot’s performance. Include an error analysis and discussion of sources of error. Characterize under what conditions your system performs well, and under what conditions your robot fails. Summarize what you varied when testing your robot. In particular: What parameters did you vary to get things to work? What are the tradeoffs made in your design?
- Summary and Challenges: Briefly summarize your project and what you found. Discuss what you learned, as well as some challenges you encountered (if any). Tell us what you would do differently if you were to attempt this project again and/or had more time.
There is no page limit to this report, but please try to keep it as concise as possible.