Experience Lab: Eye Tracking & Object Detection
Combining traffic object detection in cycling footage with gaze and fixation data from Pupil Labs glasses to support behavioural research.
Role
Data Scientist · Team project
Focus
Object detection & gaze analysis
Context
BUas Experience Lab
The problem
Experience Lab uses Pupil Labs eye tracking glasses during cycling studies. The glasses record video footage together with gaze and fixation data. Researchers then need to determine which traffic objects a participant was looking at. Reviewing the recordings manually takes a lot of time.
Our goal was to build a workflow that detects traffic objects in the video, automatically annotates them, and connects those detections with gaze and fixation data so the results can be analysed further.
What I worked on
I worked as one of the data scientists on the project. My work included checking data quality, preparing datasets, developing and evaluating object detection models, and connecting the detections with data from Pupil Labs.
- Built a workflow for checking and reporting data quality.
- Developed and compared YOLO models for detecting several types of traffic objects.
- Processed complete cycling recordings and automatically annotated detected traffic objects over time.
- Connected detections with gaze and fixation data and produced structured results for further analysis.
- Used explainable AI methods to inspect how the models made their detections.
Project workflow
1. Data preparation
Check annotations, timestamps and data sources before training the model.
2. Object detection
Train and evaluate a YOLO model to detect different types of traffic objects.
3. Tracking and eye tracking data
Run the model on cycling footage, track detected objects over time and match gaze and fixation data to those objects.
4. Results
Export the combined data so researchers can analyse which traffic objects appeared and which objects participants looked at.