How to Improve Pose-Detection Accuracy in Your App

Bad pose-detection accuracy is almost never one problem — it is a stack of small ones, from how the camera is framed to which model tier you loaded to whether you smoothed the landmarks. This guide is an ordered accuracy checklist: work top to bottom, because the capture-side fixes are free and undo the most damage, while the code-side fixes (thresholds, smoothing, normalization) only pay off once the pixels feeding the model are good. It assumes you are building on a free, on-device pose primitive (MediaPipe Pose Landmarker or MoveNet); if you use a fitness SDK, most of these knobs are tuned for you, but the capture-side rules still apply to your end users.

How to diagnose where accuracy is going wrong

Before you change anything, figure out which layer is failing. Draw the skeleton on screen (an overlay) and watch it:

• Landmarks in the wrong place, or missing entirely → a capture problem (framing, lighting, occlusion) or the wrong model tier. Steps 1 to 3.
• Landmarks roughly right but jumping/vibrating frame to frame → a jitter problem. Steps 4 and 5.
• Landmarks fine but your angles or rep counts are wrong at different distances → a normalization problem. Step 6.
• Everything correct but the app is slow, hot, or laggy → an on-device performance problem. Step 7.

Most single-person models (BlazePose behind MediaPipe, MoveNet) assume one whole person, reasonably centered, in frame. Break that assumption and no amount of code will save the output — so start at the camera.

What you'll need

• A working pose pipeline on a free primitive: @mediapipe/tasks-vision (MediaPipe Pose Landmarker) or @tensorflow-models/pose-detection (MoveNet). As of 2026, check the current package versions before you ship.
• An on-screen skeleton overlay so you can see what the model sees.
• Access to each landmark's confidence: MediaPipe exposes per-landmark visibility (0.0 to 1.0); MoveNet exposes score per keypoint.

Step 1: Fix framing, distance, and lighting

This is the single biggest win and it costs nothing. Cropped joints produce missing or garbage landmarks, and backlit or shadowed bodies wreck landmark localization.

• Full body in frame. Frame head-to-ankles with a small margin. MoveNet in particular re-crops around the previous detection, so the subject must start fully in view.
• Distance. Too far and there are too few pixels on the body (especially for MoveNet Lightning's small input); too close and joints clip out of frame. As a rule of thumb, keep the body around 70 to 90 percent of the frame height.
• Lighting. Use even, front-ish lighting. Backlight and silhouettes are the most common cause of bad landmarks. Avoid harsh single-side shadows.
• Clothing. Baggy clothes hide the true joint center — the model tracks the fabric edge and biases every angle you compute. Fitted clothing tends to improve joint localization.

If your end user is exercising in front of the camera, give them one setup line: clear the space, stand back so the whole body is visible head to ankle, and face a window or light rather than having it behind you.

Step 2: Match the camera angle to the plane of motion

A 2D landmark angle only equals the real joint angle when the limb moves roughly parallel to the image plane. Point the camera at the wrong plane and your numbers are wrong even when the skeleton looks fine.

• Side view for sagittal exercises — squat, curl, lunge, deadlift — so knee, elbow, and hip angles sit in the image plane.
• Front view for symmetry and knee-valgus checks, which compare left/right x-positions.
• Keep the camera level and stable. A tripod beats a hand or a couch. A tilted camera biases every rule that compares a body segment to vertical.
• Prefer the rear camera when someone can prop the phone. It has higher resolution and better optics, so it yields cleaner landmarks. The front camera is for self-guided mirror UX and gives noisier output — expect it and gate harder (Step 4).

Step 3: Pick the right model tier and input resolution

Model tiers trade latency for accuracy. If landmarks are consistently imprecise on a capable device, move up a tier before you write more code.

• MediaPipe Pose Landmarker: lite → full → heavy. Use lite for latency-critical UI; step up to full or heavy for form grading when the device can hold the frame rate.
• MoveNet: SINGLEPOSE_LIGHTNING (192x192 input) versus SINGLEPOSE_THUNDER (256x256 input). Thunder's higher input resolution is a large part of its accuracy edge — reach for it when accuracy matters, Lightning when speed does.

Switching tiers is a one-line change. For MoveNet:

import * as poseDetection from "@tensorflow-models/pose-detection";

// Thunder = higher input resolution (256x256), more accurate, slower.
const detector = await poseDetection.createDetector(
  poseDetection.SupportedModels.MoveNet,
  { modelType: poseDetection.movenet.modelType.SINGLEPOSE_THUNDER }
);

For MediaPipe, point modelAssetPath at the pose_landmarker_full.task or pose_landmarker_heavy.task bundle instead of lite. Feeding a larger, cleaner crop of the person also helps, since the model resizes to that small square internally regardless.

Step 4: Gate on confidence and visibility

Every landmark carries a confidence score, and occluded or low-confidence landmarks jump to garbage positions that trip thresholds and throw your angles. Do not measure a joint you do not trust.

Before you use a joint, check its confidence and skip the frame (or hold the last good value) when it is too low. A common gate is around 0.5 to 0.7 — tune it per camera, and gate harder on the noisier front camera.

const MIN_VIS = 0.6;

function trusted(...points) {
  // MediaPipe: point.visibility. MoveNet: point.score.
  return points.every(p => (p.visibility ?? p.score) >= MIN_VIS);
}

// Only compute an angle when all three joints are reliable this frame.
if (trusted(hip, knee, ankle)) {
  updateAngle(hip, knee, ankle);
} else {
  // Skip the frame; do not let a visibility-near-zero landmark drive a threshold.
}

Step 5: Smooth the landmarks and signals

Even correct landmarks vibrate frame to frame. That jitter is what makes overlays shaky and makes a single-frame spike defeat your rep logic. There are two different smoothers, and using the right one matters.

• One-Euro filter for the landmark coordinates. The One-Euro (1€) filter is an adaptive low-pass filter that uses position and velocity: it lowers the cutoff when the body is still (killing jitter) and raises it during fast motion (avoiding lag). It is MediaPipe Pose's built-in landmark smoother, applied as the last stage of the graph with min_cutoff around 0.1 plus a velocity term. Do not confuse it with a plain EMA — for raw coordinates One-Euro is better because it adapts to speed. MediaPipe applies it for you in LIVE_STREAM mode; MoveNet exposes enableSmoothing (on by default).
• EMA for the scalar signals you compute downstream (a joint angle, a class probability). An exponential moving average is the cheap, correct choice here, and it is what Google's own fitness samples apply to the classification stream before thresholding.

An EMA is one line of state per signal. Keep alpha modest (roughly 0.2 to 0.4) so you kill jitter without lagging real movement enough to miss a fast rep:

let angleSmoothed = null;              // per-signal state
const ALPHA = 0.3;                     // 0.2..0.4: lower = smoother but laggier

function smooth(raw) {
  angleSmoothed = angleSmoothed === null
    ? raw                              // seed on the first frame
    : ALPHA * raw + (1 - ALPHA) * angleSmoothed;
  return angleSmoothed;
}

If you need a One-Euro filter on a coordinate stream your library does not smooth for you, its core is the same adaptive idea: an EMA whose alpha is derived per frame from the point's speed. Reach for a small published One-Euro implementation rather than hand-rolling it.

Step 6: Normalize coordinates and handle missing joints

If your rules work at one camera distance but drift at another, you are thresholding raw pixels. Fix it by making everything distance- and resolution-invariant.

• Work in normalized coordinates. MediaPipe landmark x/y are already normalized to 0.0 to 1.0 of the frame; MoveNet returns pixels, so divide by frame width/height yourself.
• Normalize distances by a body reference. Divide any pixel distance (a knee-to-ankle offset, a valgus tolerance) by a stable segment like shoulder width, hip width, or torso length before you threshold it. This mirrors how MediaPipe's pose classifier normalizes landmarks by torso size. Angles are already scale-invariant, so this mainly matters for distance-based rules.
• Handle missing or occluded joints deliberately. When a required joint drops below the visibility gate, do one of three things, in order of preference: fall back to the contralateral side (use the right knee when the left is hidden), hold the last good value for a few frames, or pause counting rather than emit a wrong angle. Never let a near-zero-visibility landmark drive a decision.

Step 7: Tune on-device performance

Correct landmarks are worthless if the pipeline is slow, hot, or laggy — dropped and stale frames read as accuracy loss. These are the highest-leverage knobs.

• Use the real-time running mode. For live camera input use MediaPipe's runningMode = LIVE_STREAM (async, one callback per frame). It is built for streaming and internally applies the temporal filtering and tracking; IMAGE/VIDEO modes do not give you the same live smoothing behavior.
• Enable the GPU delegate. Turning on the GPU delegate (or NNAPI / Core ML where available) offloads inference and frees the CPU for your app — usually the single biggest on-device speedup.
• Downscale frames before inference. The model resizes to 192 or 256 internally anyway, so do not feed it 4K. A 640x480 capture is plenty. Convert once and reuse buffers instead of allocating per frame.
• Cap the frame rate deliberately and process the newest frame. Target a stable 24 to 30 FPS. Higher FPS gives finer temporal resolution so you do not miss fast reps, but costs compute and heat. If the queue backs up, skip to the newest frame rather than accumulating latency. When you must cut something for form use-cases, drop resolution before you drop the model tier.
• Keep the detector warm. Both MoveNet and BlazePose track the person frame to frame; recreating the detector every frame throws that tracking away. Create it once.

The accuracy checklist, in order

1. Full body in frame, good distance, even front lighting, fitted clothing.
2. Camera angle matched to the plane of motion, level and stable, rear camera when possible.
3. Model tier and input resolution matched to the accuracy you need.
4. Confidence/visibility gating so untrusted joints never drive a measurement.
5. One-Euro on coordinates, EMA on the scalars you compute.
6. Normalized coordinates and a deliberate plan for missing joints.
7. LIVE_STREAM mode, GPU delegate, downscaled frames, a warm detector.

Work them in order. The first two are free and fix most "the model is bad" complaints; the rest turn a working demo into something that holds up across body types, distances, and low-end devices.