Make the Mustang a research car
Research cars carry calibrated cameras, an RTK antenna, and software that knows where every sensor sits. We will build that on our Mach-E.
- Two synchronized global shutter cameras on the roof bar
- The RTK mosaic-X5 antenna from Lab 2
- Calibration files and every transform
- One ROS 2 launch file that starts it all
2
What you get
Quiz retake credits
Earn up to 2 credits. Each credit lets you retake one quiz. How to earn them is on the Credits slide.Real research skills
3D design for a vehicle, camera calibration to research-grade targets, and a ROS 2 sensor stack: the same work done on autonomous driving cars.Your work stays on the car
Future labs and research will record data with the rig you build. Your name goes in the README and the course repo.It runs alongside Lectures 11 to 13: homography, then features and stereo, then calibration and mosaicing. Each lecture gives your team the tool it needs next.
3
The hardware
| Part | What it is | Why it matters |
|---|---|---|
| 2 cameras | Teledyne FLIR Blackfly S BFS-U3-19S4C-C: Sony IMX430, 1/1.7 inch, 1616 × 1240 px, 4.5 µm pixels, color, up to 131 fps, USB3 | Global shutter: the whole image is exposed at once, so a moving car does not bend the picture. A GPIO connector lets one camera trigger the other. |
| 2 lenses | Edmund Optics 5 mm wide angle, low distortion: f/2.8 fixed iris, C-mount, distortion under 0.5% | No refocusing from 100 mm to infinity, so the focus cannot drift after calibration. |
| RTK antenna | The SparkFun RTK mosaic-X5 kit from Lab 2 | Centimeter position on the map for every image. |
| Roof bar | Our custom bar, with mounting points near the top front corners of the roof | Sets where the cameras and antenna can go, and how wide the baseline can be. |
| Laptop | Ubuntu 24.04, ROS 2 Jazzy, two free USB3 ports | Runs the drivers, stereo, GNSS, and recording inside the car. |
The camera is rated for 0 to 50 °C. Keep it shaded on sunny days.
4
What one camera sees
| Focal length in pixels | f = 5 mm / 4.5 µm = 1111 px |
| Horizontal field of view | 2 atan(7.27 / 2 / 5) = 72° |
| Vertical field of view | 2 atan(5.58 / 2 / 5) = 58° |
| One pixel at 10 m | 10 m × 4.5 µm / 5 mm = 9 mm |
| One pixel at 30 m | 27 mm |
These are the numbers on paper. Lecture 13's calibration measures the real ones, and Lecture 11 explains every symbol.
The starting intrinsic matrix
K ≈
| 1111 | 0 | 808 |
| 0 | 1111 | 620 |
| 0 | 0 | 1 |
808 and 620 are the image center. A real lens puts the center a few pixels off, and calibration finds it.
Motion blur: at 2 m/s with a 5 ms exposure the car moves 1 cm, about one pixel at 10 m. In the dim garage, raise the gain before you lengthen the exposure.
5
Sensor size 7.27 × 5.58 mm from the Blackfly S specifications
Picking the baseline
Depth error for f = 1111 px and a 0.25 px matching error: δZ = Z² × 0.25 / (f B)
| Baseline B | 3 m | 5 m | 10 m | 15 m | 20 m | Closest (256 px search) |
|---|---|---|---|---|---|---|
| 0.30 m | 0.7 cm | 1.9 cm | 7.5 cm | 17 cm | 30 cm | 1.3 m |
| 0.50 m | 0.4 cm | 1.1 cm | 4.5 cm | 10 cm | 18 cm | 2.2 m |
| 0.60 m | 0.3 cm | 0.9 cm | 3.8 cm | 8.4 cm | 15 cm | 2.6 m |
| 0.80 m | 0.3 cm | 0.7 cm | 2.8 cm | 6.3 cm | 11 cm | 3.5 m |
| 1.00 m | 0.2 cm | 0.6 cm | 2.2 cm | 5.1 cm | 9.0 cm | 4.3 m |
Wider is better far away, but it cannot see objects close to the car, and the two images look less alike.
Start at 0.5 m: under 5 cm at 10 m, and it sees from about 2 m. Team A decides at the design review (M2).
Lecture 12 explains this table.
6
Three teams, ten students
Team A: Mount (3)
Design and 3D print the brackets that hold both cameras and the antenna on the roof bar, plus the cable route into the car.Team B: Calibration (4)
Intrinsics of each camera, stereo extrinsics, and the transforms from the cameras and antenna to the car.Team C: ROS 2 (3)
Drivers with hardware sync, stereo processing, TF, the GNSS driver, and recording, all from one launch file.The teams depend on each other: B calibrates on A's mount, and C loads B's files. Agree on the shared names early.
Pick a first and second choice when you sign up. Each team has one role per person, listed on the next three slides.
7
Team A: Mount
What you build
- Camera brackets on a rigid bar at the chosen baseline, with a pitch you set once and lock
- An antenna holder above the cameras, with a clear view of the sky
- Attachment to the roof bar at the front corner mounting points
- Cable routing and strain relief into the car, with the doors still closing
Roles
- CAD and bracket design
- Printing and assembly, including the antenna holder
- Roof fitting, cables, and the mounting guide
Deliverables
- CAD source, STEP and STL files, print settings, parts list
- Nominal transforms from the CAD model for Team B
- Photos and a one-page mounting guide
Done when
The rig mounts in under 15 minutes, tapping the bar does not move the image, and the stereo check still passes after one remount (target on the Targets slide).Print in PETG or ASA, not PLA: a dark roof in the sun softens PLA. Plastic holds parts to the bar; the bar, not the plastic, sets the baseline.
8
Team B: Calibration
What you build
- Intrinsics (K and distortion) of each camera with camera_calibration
- Stereo extrinsics: R and t from the left to the right camera
- Camera to base_link, and the camera to antenna lever arm: CAD first, then measured, then checked
- Depth tests at known distances in the garage
Roles
- Intrinsics for both cameras
- Stereo calibration and the rectification check
- Vehicle and antenna transforms
- Validation tests and the calibration report
Deliverables
- Camera info YAML files for both cameras
- The filled transforms table
- A short calibration report as a notebook
Done when
Every number on the Targets slide is met, and the report shows how it was measured.9
Team C: ROS 2
What you build
- Both cameras with spinnaker_camera_driver, hardware synced with spinnaker_synchronized_camera_driver
- stereo_image_proc for disparity and the point cloud
- Static TF from the transforms table, and the GNSS driver from Labs 1 and 2
- Recording launch file and a health check node
Roles
- Drivers and hardware sync
- Stereo pipeline, TF, and GNSS in one launch file
- Recording, health check, and verify script
Deliverables
- A ROS 2 Jazzy package with README and launch files
- A short bag from the classroom test
- A verify script like the ones in Labs 1 and 2
Done when
A fresh Ubuntu 24.04 laptop goes from clone to live disparity in RViz with the README alone, and left and right images carry the same timestamp.Plan the data rate: two cameras at 30 fps send about 120 MB/s, over 7 GB per minute of bag. Record at a lower rate when you do not need every frame.
10
Shared names
| Frame | Where |
|---|---|
| base_link | Center of the rear axle |
| roof_bar | Center of the bar, top face |
| stereo_left_optical_frame | Left camera optical center |
| stereo_right_optical_frame | Right camera optical center |
| gnss_antenna | Antenna phase center |
base_link: x forward, y left, z up. Optical frames: z forward, x right, y down.
| Topic | From |
|---|---|
| /stereo/left/image_raw | camera driver |
| /stereo/left/camera_info | driver, Team B's YAML |
| /stereo/right/... | same as left |
| /stereo/disparity | stereo_image_proc |
| /stereo/points2 | stereo_image_proc |
| [[ GNSS FIX TOPIC ]] | GNSS driver |
| /tf_static | transforms table |
Workspace: ~/EECE5554/Bonus/ros2_ws. Package: mustang_stereo
11
Targets
| Check | Target | Must pass | Why this number |
|---|---|---|---|
| Intrinsic reprojection error (RMS) | ≤ 0.3 px | ≤ 0.5 px | Half a pixel is the usual line for a good calibration; careful research rigs get well under it. |
| Stereo calibration error (RMS) | ≤ 0.5 px | ≤ 0.8 px | Both cameras and the transform between them, so a little higher than one camera. |
| Rectified rows line up | ≤ 0.3 px | ≤ 0.5 px | Stereo matching searches one row; a row error becomes a depth error. |
| Depth at 3, 5, 10, 15 m (garage) | ≤ theory | ≤ 2 × theory | The baseline table predicts the error; with B = 0.5 m that is 4.5 cm at 10 m (1% of the distance at 2 × theory). |
| Camera to base_link | 1 cm, 0.2° | 2 cm, 0.5° | 0.2° of pitch moves a point 20 m ahead by 7 cm on the ground. |
| Antenna lever arm | ≤ 1 cm | ≤ 2 cm | RTK fixed is good to about 2 cm; the lever arm should not be the weak link. |
| Left to right sync | same pulse | ≤ 1 ms | Hardware trigger gives identical timestamps; software sync is the fallback. |
| After a remount | ≤ 0.5 px | recalibrate | The rectified row check after remounting tells you if the old files still hold. |
12
How credits work
1 credit (one quiz retake)
- Own one role on your team
- Its deliverable is reviewed and merged into the team repo
- Your contribution log and commits show the work
About 8 to 10 hours of your own work.
2 credits (the most)
- Everything for 1 credit
- Your team's "done when" test passes at the Mustang classroom test (M4)
- You present your part at the showcase (M5)
About 16 to 20 hours in total.
Credits are per student. A team that passes together earns together, but no one earns a credit without their own merged work.
13
Milestones
Kickoff
Sign up. Measure the roof bar and mounting points. Write the requirements.
Sign up. Measure the roof bar and mounting points. Write the requirements.
Design review
Baseline chosen. CAD version 1. Node graph and shared names agreed.
Baseline chosen. CAD version 1. Node graph and shared names agreed.
Bench
Printed parts fit. Intrinsics done in the classroom. Launch file runs both cameras.
Printed parts fit. Intrinsics done in the classroom. Launch file runs both cameras.
Classroom test
Mount on the car, stereo and lever arm calibration, depth tests, bags.
Mount on the car, stereo and lever arm calibration, depth tests, bags.
Showcase
Live stereo depth, an RTK ground mosaic, and files in the course repo.
Live stereo depth, an RTK ground mosaic, and files in the course repo.
Dates are in the course schedule and on Canvas.
14
Where we work
The garage of the 225 building
- Mounting, calibration, and the depth tests at 3, 5, 10, and 15 m
- Out of the rain and the sun, with even light
- The car stays parked for all of it
Open sky for RTK
- A garage roof blocks the satellites, so RTK will not fix inside
- The lever arm check and the mosaic drive need open sky: [[ OPEN-SKY SPOT ]]
- Same NTRIP setup as Lab 2
Bring a phone hotspot for NTRIP, a tape measure, and the large calibration board. The instructor brings the car.
15
Safety rules
- All mounting and calibration happen with the car parked. Only the instructor drives, at walking speed.
- Nothing metal or sharp touches the paint or glass. Use the roof bar's mounting points only.
- Check every bolt before the car moves, and pull on the rig by hand.
- Keep cables out of the door seals and away from the pedals.
- Keep the cameras shaded on sunny days: they are rated to 50 °C.
- After any bump or remount, run the rectified row check before you trust new data.
16
Sign up
Sign up here: [[ SIGN-UP LINK ]]
- Give a first and second team choice, and the role you want.
- Teams are set by the next class: A has 3, B has 4, C has 3.
- M1 starts right away: Team A measures the roof bar, Team B prints a small checkerboard, Team C installs the Spinnaker SDK and the driver.
Questions? Ask after class or on the course discussion board.
17