What AI tennis video analysis can (and can't) tell you
We build an AI tennis coach. Here is what it's genuinely good at, what it still gets wrong, and what we do about each.
From ordinary phone video, a current AI model is good at repeated, visible patterns: late preparation, contact too close to the body, a missing split-step, poor recovery. It is unreliable at details a phone barely captures, especially whether a fast shot was a forehand or a backhand and what spin it had. Use it to find one pattern to work on, not to rebuild your technique.
What it reads well
- Patterns across many shots. A model watching every second of a clip doesn't get tired or miss the third rep. Baseline only reports a fault that appears on at least two separate shots, because a one-off mistake is noise.
- Movement and positioning. Whether you recover to the middle, where you stand to return, whether you're still when the ball is struck. These are large, slow movements a phone captures well.
- Timing relative to the ball. Late take-backs and split-steps that land after the opponent's contact show up clearly at 30 to 60 fps.
- Counting. How many shots, which strokes, which ones were struck cleanly. Each shot gets a timestamp you can click to watch it.
Where it struggles
Forehand or backhand, on a fast overhead
This sounds trivial, and it's the clearest limit we've measured. On real footage, our side calls on difficult shots weren't reliably better than a coin flip, across several different prompting strategies, frame-by-frame pose measurement, and tightly cropped full-resolution stills. The model also reported high confidence on its wrong answers, so a confidence threshold doesn't fix it. Baseline therefore doesn't tell you which side a shot was on. You can mark it with one tap, and your corrections are used as examples for that player's future analyses.
Spin
Spin depends on the racket face at the instant of contact, which at phone frame rates is usually a blur. Models also default to calling everything "topspin". Baseline doesn't assert spin for the same reason it doesn't assert side: a guess presented as a fact is worse than leaving it blank.
Small or distant players
On a phone video of a full court, the far player's shoulders can be just a few pixels apart. Detail at that size isn't recoverable. Film the player you want analyzed on the near side, and see the filming guide.
Ball speed and exact placement
A coaching report from video is not a radar or a line-calling system. If you want a speed number, the serve speed calculator measures it from frame timing instead.
How we check the AI's work
Some of the model's readings are checked by measurement rather than by asking it again:
- Every second gets looked at. Short clips are split into overlapping windows of about six and a half seconds, and each window is analyzed on its own, twice, at two different frame rates. Early versions trusted one pass over the whole clip and missed most of the shots in some videos.
- Disagreement becomes "unclear". If the two readings of a shot disagree, the report says so instead of picking one.
- Contact height is measured. When the model thinks a ball was struck above the head (which would make it a serve or overhead), a separate pose-estimation model compares the wrist to the head in the actual frame. If the wrist is lower, the reading is corrected.
- Narrative claims are checked against the shot log. A timestamp in the written feedback must match a logged shot, or it's removed.
How to use AI feedback well
- Pick the one pattern the report says repeats most, and work on only that for a week.
- Correct any wrong shot labels. It takes a second and improves that player's next reports.
- Film the same drill again after a week and compare the two sessions.
- If you have a coach, show them the clip and the report. It's a starting point for a lesson, not a replacement for one.
See what it makes of your game. Upload a clip and get a report in a few minutes; the first one is free.
Related
- How to film your tennis: Where to put your phone, which angle to use, what frame rate and clip length work best, and the mistakes that make tennis video useless for analysis. A practical setup guide.
- SwingVision alternative: SwingVision is Apple-only. If you're on Android, or want a written coaching report from clips you already have, here's how Baseline compares, including where SwingVision is the better pick.