experience · University of Waterloo

Undergraduate Research Assistant

timeline: Sept 2026 — present

tech stack: Python, OpenCV, YOLOv8-pose, AprilTags, MuJoCo, NumPy

Overview

Research in the Department of Mechatronics Engineering on the biomechanics of judo throws. The existing lab pipeline lifted each athlete to 3D from a single camera and root-centred them independently, so the distance between thrower and opponent — exactly what kuzushi (off-balancing) analysis needs — was meaningless. I built a 2-camera motion-capture pipeline in Python and OpenCV that reconstructs 3D pose for both athletes at once from two consumer phones, with no hardware sync and no marker suits.

Synchronization

Two phones press record at different instants, and at 30 fps a single frame of error becomes several centimetres of triangulation error on a moving hip. Rather than wiring the cameras together, the pipeline syncs them in software: it runs 2D pose estimation on both videos, reduces each to a one-dimensional trunk-velocity signal, and cross-correlates the two anchored on the opening bow that every match begins with. That brings the videos to within 33 ms (one frame) of each other, validated independently against the audio of the throw landing.

Tracking Through Occlusion

Judo is a worst case for pose estimation — two bodies in near-identical gis, gripping and wrapping around each other, so baseline trackers constantly swap identities mid-throw. I fused coloured armbands and AprilTags into the detection stage so each athlete carries a persistent identity through contact, which kept joint tracking at 90% accuracy through severe occlusion and eliminated the identity-switch failures that broke downstream analysis.

Biomechanical Simulation

Synchronized 3D joint trajectories are mapped onto a 20-joint MuJoCo human model (a pelvis free joint plus 14 hinges) so we can simulate what the cameras can't directly measure — center of mass, lean angle, and support-base geometry — through each phase of a throw: kuzushi, tsukuri (entry), and kake (execution).