How to Add AI Workout Tracking to an Android App (Kotlin)

AI workout tracking on Android means pointing the camera at someone exercising, estimating their body pose on-device every frame, and turning those keypoints into reps and form feedback. This guide builds that pipeline in Kotlin with CameraX plus Google's MediaPipe Tasks PoseLandmarker, then counts a squat rep from a single joint angle.

There are two paths. Build on a free primitive: wire CameraX to an on-device pose model (MediaPipe Tasks PoseLandmarker, or the more turnkey ML Kit PoseDetector) and write your own rep and form logic — that's what most of this guide does. Buy an SDK: a commercial fitness SDK bundles camera, pose, rep-counting and form feedback as a drop-in, so you skip the plumbing below (covered at the end).

What you'll need

• An Android app targeting a reasonably modern device, with Kotlin and Jetpack Compose or Views.
• CameraX for the camera feed and per-frame analysis (androidx.camera:camera-core, camera-camera2, camera-lifecycle, camera-view).
• A pose model. Primary here: MediaPipe Tasks Vision — com.google.mediapipe:tasks-vision — plus a .task model bundle. Turnkey alternative: ML Kit Pose Detection — com.google.mlkit:pose-detection.
• Camera permission (android.permission.CAMERA) requested at runtime.
• Verify the exact current version strings for each dependency as of 2026, since these packages release often.

Both MediaPipe PoseLandmarker and ML Kit derive from Google's BlazePose model and emit the same 33 body landmarks. MediaPipe gives you more control and cross-platform reuse; ML Kit is a one-line drop-in for Android and iOS only.

Step 1: Set up CameraX with an ImageAnalysis use case

The camera feed reaches your code through CameraX's ImageAnalysis use case, which hands you one ImageProxy per frame. Use STRATEGY_KEEP_ONLY_LATEST so the analyzer always processes the newest frame instead of building up latency.

import androidx.camera.core.CameraSelector
import androidx.camera.core.ImageAnalysis
import androidx.camera.core.ImageProxy
import androidx.camera.lifecycle.ProcessCameraProvider
import androidx.core.content.ContextCompat
import java.util.concurrent.Executors

val cameraExecutor = Executors.newSingleThreadExecutor()

val imageAnalysis = ImageAnalysis.Builder()
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .build()

imageAnalysis.setAnalyzer(cameraExecutor) { imageProxy: ImageProxy ->
    // We hand this frame to the pose model in Step 3.
    analyzeFrame(imageProxy)
}

// Bind the use case to the lifecycle (typical CameraX setup):
val cameraProvider = ProcessCameraProvider.getInstance(context).get()
cameraProvider.bindToLifecycle(
    lifecycleOwner,
    CameraSelector.DEFAULT_BACK_CAMERA,   // rear camera = better landmarks
    imageAnalysis
)

Step 2: Create the PoseLandmarker in LIVE_STREAM mode

For a live camera you want RunningMode.LIVE_STREAM: it runs asynchronously, delivers each result to a listener, and applies MediaPipe's built-in temporal smoothing (a One-Euro filter on the landmarks). Point BaseOptions at your .task model asset and register a result listener.

import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.poselandmarker.PoseLandmarker
import com.google.mediapipe.tasks.vision.poselandmarker.PoseLandmarkerResult

val baseOptions = BaseOptions.builder()
    .setModelAssetPath("pose_landmarker_lite.task")   // ship a .task bundle in assets
    .build()

val options = PoseLandmarker.PoseLandmarkerOptions.builder()
    .setBaseOptions(baseOptions)
    .setRunningMode(RunningMode.LIVE_STREAM)
    .setResultListener { result: PoseLandmarkerResult, _ ->
        onPoseResult(result)   // handled in Steps 4-6
    }
    .setErrorListener { error -> /* log it */ }
    .build()

val poseLandmarker = PoseLandmarker.createFromOptions(context, options)

MediaPipe ships three model tiers — pose_landmarker_lite (fastest, ~5.5 MB), full, and heavy (most accurate, ~29 MB). Start with lite for real-time; move up if you need cleaner form grading and the device can sustain the frame rate. Verify the exact builder method names against the current MediaPipe Tasks docs before shipping.

Step 3: Feed each CameraX frame to the model

In LIVE_STREAM mode you call detectAsync(mpImage, timestampMs) with a MediaPipe image and a monotonically increasing timestamp in milliseconds; the result comes back on the listener from Step 2. Convert the ImageProxy to an MPImage first, then close the proxy so the next frame can arrive.

import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.framework.image.MPImage
import android.os.SystemClock

fun analyzeFrame(imageProxy: ImageProxy) {
    val bitmap = imageProxy.toBitmap()                  // handle rotation as needed
    val mpImage: MPImage = BitmapImageBuilder(bitmap).build()
    // LIVE_STREAM requires a monotonically increasing timestamp. Use the
    // monotonic uptime clock, NOT System.currentTimeMillis() (wall-clock, can
    // jump backward and make detectAsync throw).
    val timestampMs = SystemClock.uptimeMillis()

    poseLandmarker.detectAsync(mpImage, timestampMs)
    imageProxy.close()                                  // required to get the next frame
}

The exact ImageProxy-to-MPImage conversion (rotation, YUV handling, buffer reuse) is boilerplate that varies by device — verify it against Google's PoseLandmarkerHelper.kt in the official mediapipe-samples repo rather than assuming toBitmap() covers every case.

Step 4: Read the 33 keypoints from the result

PoseLandmarkerResult.landmarks() returns a list of poses, each a list of 33 normalized landmarks with x, y, z (all in 0.0 to 1.0 for x/y) and a visibility score. BlazePose indices are fixed: hip is 23, knee 25, ankle 27 (left side). Pull out the joints your metric needs.

fun onPoseResult(result: PoseLandmarkerResult) {
    val poses = result.landmarks()
    if (poses.isEmpty()) return
    val lm = poses[0]                 // first (and usually only) person

    val hip   = lm[23]                // left_hip
    val knee  = lm[25]                // left_knee
    val ankle = lm[27]                // left_ankle

    // Confidence gate: skip frames where the driving joints aren't trustworthy.
    if (hip.visibility().orElse(0f) < 0.6f ||
        knee.visibility().orElse(0f) < 0.6f ||
        ankle.visibility().orElse(0f) < 0.6f) return

    updateRep(hip, knee, ankle)       // Steps 5-6
}

The accessor shapes (x(), y(), visibility()) follow MediaPipe's NormalizedLandmark — verify the exact method names against the current Tasks Vision API for your version.

Step 5: Compute the joint angle for a rep

A squat is driven by the knee angle: the interior angle at the knee between the hip-to-knee and ankle-to-knee segments. Use the atan2 of the cross and dot products — it is numerically robust and returns 0 to 180 degrees, unlike bare acos, which can lose precision or return NaN near the extremes.

import kotlin.math.atan2
import kotlin.math.abs
import kotlin.math.PI

// Angle at vertex B for points A-B-C.
fun angle(ax: Float, ay: Float, bx: Float, by: Float, cx: Float, cy: Float): Double {
    val baX = ax - bx; val baY = ay - by      // B -> A
    val bcX = cx - bx; val bcY = cy - by      // B -> C
    val dot   = baX * bcX + baY * bcY
    val cross = baX * bcY - baY * bcX
    return atan2(abs(cross).toDouble(), dot.toDouble()) * 180.0 / PI  // 0..180
}

// Knee angle: A = hip, B = knee, C = ankle.
val kneeAngle = angle(
    hip.x(),  hip.y(),
    knee.x(), knee.y(),
    ankle.x(), ankle.y()
)
// Standing ~170-180 degrees; parallel squat ~90; deep ~70.

The angle is scale-invariant, so normalized coordinates are fine as long as x and y are in the same units. It is also viewpoint-dependent — a 2D angle only matches the real joint angle when the limb is roughly parallel to the image plane, so a side view works best for squats, curls and lunges.

Step 6: Count reps with a hysteresis state machine

Rep counting is not machine learning — it is a two-state toggle (down or up) over the angle, with two separated thresholds so noise around a single line cannot fire phantom reps. Enter "down" when the angle drops below the low threshold, and count only when it climbs back above the high threshold. Smooth the angle with a light EMA first so a single-frame spike doesn't defeat the thresholds.

private var state = "up"     // start standing
private var reps = 0
private var angleS = Double.NaN   // EMA-smoothed angle

private val ENTER_DOWN = 100.0    // must go below this to be "down" (deep)
private val EXIT_UP    = 160.0    // must go above this to be "up" (standing)
private val ALPHA      = 0.3      // EMA factor (0.2..0.4)

fun updateRep(hip: NormalizedLandmark, knee: NormalizedLandmark, ankle: NormalizedLandmark) {
    val raw = angle(hip.x(), hip.y(), knee.x(), knee.y(), ankle.x(), ankle.y())

    // Temporal smoothing (EMA on the scalar angle).
    angleS = if (angleS.isNaN()) raw else ALPHA * raw + (1 - ALPHA) * angleS

    // Hysteresis: count on the down -> up completion only.
    if (state == "up" && angleS < ENTER_DOWN) {
        state = "down"
    } else if (state == "down" && angleS > EXIT_UP) {
        state = "up"
        reps += 1
        onRepComplete(reps)   // update UI, evaluate form for this rep
    }
}

Surface reps in your UI on the main thread (Compose state or a LiveData), and use onRepComplete as the hook to grade form — for example, tag the rep by the minimum knee angle you saw during the down phase and cue "go deeper" if it never passed 100 degrees.

The turnkey alternative: ML Kit Pose Detection

If you don't need MediaPipe's cross-platform control, ML Kit is a shorter drop-in for Android. You build a PoseDetector in stream mode, wrap each frame in an InputImage, and read the same 33 PoseLandmark points — then apply the exact same angle and rep logic from Steps 5 and 6.

import com.google.mlkit.vision.common.InputImage
import com.google.mlkit.vision.pose.PoseDetection
import com.google.mlkit.vision.pose.PoseLandmark
import com.google.mlkit.vision.pose.defaults.PoseDetectorOptions

val detector = PoseDetection.getClient(
    PoseDetectorOptions.Builder()
        .setDetectorMode(PoseDetectorOptions.STREAM_MODE)
        .build()
)

// Inside your ImageAnalysis analyzer:
val mediaImage = imageProxy.image
if (mediaImage != null) {
    val image = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
    detector.process(image)
        .addOnSuccessListener { pose ->
            val knee  = pose.getPoseLandmark(PoseLandmark.LEFT_KNEE)?.position
            // ...compute angle, run the rep state machine...
        }
        .addOnCompleteListener { imageProxy.close() }
} else {
    imageProxy.close() // always close, or the analyzer stalls on the next frame
}

The buy path: a commercial fitness SDK

Vendors such as KinesteX, Kemtai and QuickPose package the whole camera → pose → rep-counting → form-feedback pipeline as a drop-in SDK, plus a prebuilt overlay UI. You skip everything above: the SDK handles the camera lifecycle, pose inference and smoothing, exercise/rep classification and form cues internally. Integration surfaces and licensing vary by vendor — a fitness SDK wraps these steps, so refer to its docs for the exact classes and initializer signatures rather than reimplementing the pipeline.

Make it reliable

• Gate on confidence. Skip frames where the driving joints fall below a visibility of about 0.5 to 0.7, and hold state rather than counting on garbage landmarks.
• Smooth before you threshold. The EMA on the angle (alpha 0.2 to 0.4) plus the hysteresis band is what stops one real rep from registering as several.
• Count on one edge only. Incrementing on both the down and up transitions double-counts every rep.
• Match the camera to the movement. Use a side view for squats, curls and lunges so the knee and elbow angles sit in the image plane; a front view is for symmetry checks.
• Prefer the rear camera and the GPU delegate where available — better optics mean cleaner landmarks, and offloading inference frees the CPU for your app.

If your users will exercise in front of the camera, give them one setup line: clear a few feet of space, keep the whole body head-to-ankles in frame, and use even front-facing lighting. This is fitness tracking, not medical assessment — avoid clinical or efficacy claims.