Sports analytics — computer vision
RallyLens
- Status
- Research / prototype
- Role
- Creator — solo project
- Organisation
- Independent
- Year
- 2026
The problem
Most tennis computer-vision systems assume a fixed camera, a regulation court, labelled training data and a GPU. Real coaching footage — a handheld camera, a driveway wall, no dataset — has none of those, so the tooling that exists does not run on it.
Outcome
- Ball speed range
- 22.8–76.1Ball speed rangekm/h, measured on a 21s CPU-only test clip.
- Ball track coverage
- 36.3%Ball track coverage229/630 frames — recovered by tiling, motion residual and Kalman prediction where YOLO alone misses it.
- Pipeline runtime
- ~13 minPipeline runtimeFull pipeline, CPU-only, for the 21s clip.
Live overlay: players tracked by role (coach / student), ball speed, rally and shot counts, wall-target distance.
Approach
- 01
Runs zero-shot on real footage: YOLO11x and YOLO11x-pose (COCO-pretrained) for player detection and pose, with no tennis-specific fine-tuning.
- 02
Tracks players with BoT-SORT and ReID embeddings, and recovers the small, fast ball through ROI-gated SAHI tiling, camera-motion-compensated residual detection and Kalman filtering — engineered around the ball-detection problem rather than trained around it.
- 03
Calibrates a moving, non-regulation camera with OpenCV homography (ORB/SIFT + RANSAC) and a one-time manual court/wall annotation, so pans and zooms do not break tracking.
- 04
Calls bounces, impacts and wall-target hits with physics-based rules instead of a black-box classifier, so every event is explainable, and scores coaching accuracy against painted wall targets by dual-plane geometry.
- 05
Delivers an annotated H.264 video through FastAPI and Streamlit, run entirely on CPU.
Stack
- YOLO11x
- YOLO11x-pose
- Ultralytics
- BoT-SORT
- ReID
- OpenCV
- SAHI
- Kalman filtering
- FastAPI
- Streamlit
Limitations
- Tested on a single 21-second clip — no multi-clip generalisation evidence yet.
- Ball track coverage is 36.3%: for most of the clip the ball position is inferred (tiling / motion residual / Kalman), not directly detected by YOLO.
- Court and wall-target geometry is still annotated once, by hand — automatic court-line detection is on the roadmap, not built yet.