Deduplicating Health Data From Multiple Sources

A user wears an Apple Watch, carries an iPhone, and has Strava and a third-party step tracker installed. On Saturday morning they walk for an hour. Four things write to the health store about that hour, and none of them are lying. If your pipeline sums what it reads, that user's Saturday shows twenty thousand steps and eleven hundred calories, and their weekly average is now permanently wrong. Nobody files a bug for this. They just stop trusting the number, and then they stop opening the app.

The design decision this page is about is where in your system that arbitration happens, and it has a counterintuitive answer. On the platforms, you should do less than most teams do — ask HealthKit for the merged figure instead of computing your own. Everywhere else, you have to do considerably more, because there is no arbitration at all and there never was. Getting that boundary in the right place is most of the work.

Three different problems that all get called "duplicates"

They have different causes, different fixes, and different failure modes. Conflating them is why so many dedupe implementations are simultaneously too aggressive and not aggressive enough.

Only the first is solved by anything the platforms hand you for free, and only for data you wrote. HealthKit's HKMetadataKeySyncIdentifier plus HKMetadataKeySyncVersion and Health Connect's clientRecordId plus clientRecordVersion are both versioned-upsert mechanisms: Apple documents that a save with a matching sync identifier and a greater sync version replaces the existing object, and Google documents the same version gate on insertRecords. Those are excellent, and they are write-idempotency tools. They do nothing about the iPhone step samples you did not author. Reading them as "the platform deduplicates" is the single most common misunderstanding in this area.

The third problem — overlap — is the one this page is really about, because it is the only one where the naive fix is worse than doing nothing.

What each platform actually does

Here is where the popular version of this story goes wrong.

Apple documents that statistics queries "automatically merge the data from all of your data sources before performing the calculations," and that you can turn that off with the separateBySource option. An Apple Frameworks Engineer put the consequence for everyone else in about as plain a form as it gets, on the developer forums in 2022: if you are using HKSampleQuery or one of the other non-statistics queries, "you will have to select between overlapping samples within your app, and it is unlikely that you will be able to match HealthKit's merge algorithm correctly." That is Apple's own position on a build-your-own step merge for iOS, and you should take it seriously.

Two traps hide inside that. First, separateBySource disables the merge for the per-source figures it returns — so summing the per-source values re-introduces exactly the double count you were trying to remove. Use the unseparated aggregate for totals and the per-source breakdown only for attribution, never both from one calculation. Second, the merge is not "pick the best device." Apple's own WWDC20 walkthrough describes a statistics query ignoring 900 iPhone steps as duplicated and adding 1,100 iPhone steps from a day the user left the Watch at home. That is interval-wise selection, and any rule you have heard stated as "Watch always wins" is folklore that Apple's own example contradicts. Apple has not published the arbitration rule.

Health Connect's situation is different in kind, not degree. Google documents flatly that "only the Activity and Sleep data types are deduped by Health Connect, and the data totals shown are the values after the dedupe has been performed by the Aggregate API," and that "end users can set priority for the Sleep and Activity apps ... Only end users can alter these priority lists." We found no documented API on HealthConnectClient for reading the current ordering. The pipeline consequence is severe and underappreciated: an Android aggregate for steps or sleep is a function of a user-mutable setting your backend cannot observe, and reordering two apps in system settings changes the historical total for every past day with no change event to tell you. If your product needs a number that is reproducible — anything you display twice, chart, bill on, or defend in support — you cannot use the bare aggregate. Google explicitly blesses the alternative: aggregate per DataOrigin using dataOriginFilter and do your own merge.

Identifying the writer is the hard prerequisite

You cannot rank sources you cannot name, and naming them is more fragile than it looks.

On iOS, HKObject carries source, sourceRevision, device, uuid and metadata. Use HKSource.bundleIdentifier as the machine key — name is a display string. HKSourceRevision is assigned by HealthKit at save time, not by the writer, which makes it the more trustworthy provenance signal; device is writer-supplied, optional, and a third-party app may leave it nil. HKSourceQuery will enumerate the sources that have actually written a given type for this user, which is how you populate a per-user priority table rather than guessing from a static list.

On Android the key is metadata.dataOrigin.packageName, and there is a recent, specific trap that will silently split your source table. Google documents that starting with the June 2026 update, steps counted natively by Health Connect are attributed to a Synthetic Package Name such as com.android.healthconnect.phone.jd5bdd37e1a8d3667a05d0abebfc4a89e, where previously built-in steps carried the package name android. Historical data keeps android forever. SPNs are device-specific and scoped per application — different apps on the same device see different SPNs — and Google states you must not hardcode them, but resolve them at runtime with getCurrentDeviceDataSource().

Three consequences follow, and all of them are invisible in testing. Any server-side allow-list or source-mapping table keyed on step package names is now wrong, because the value cannot be enumerated server-side at all. Source identity for on-device steps has to be resolved on the device and shipped up as a resolved label. And one physical phone legitimately appears under two DataOrigin values across the June 2026 boundary, so your priority table needs an alias concept that collapses them — otherwise the same device competes with itself in your resolution and a user's history splits into two "sources" mid-year. (If you do not filter by DataOrigin at all and just read the plain aggregate, Google documents that on-device steps are included automatically and nothing changes for you. That is a real reason to prefer the aggregate where you can live with its non-reproducibility.)

Both platforms also give you the provenance flag that should gate everything downstream: Health Connect's mandatory recordingMethod — RECORDING_METHOD_AUTOMATICALLY_RECORDED, RECORDING_METHOD_ACTIVELY_RECORDED, RECORDING_METHOD_MANUAL_ENTRY — and HealthKit's HKMetadataKeyWasUserEntered. Carry it all the way into storage. A manually typed 40-minute run and a watch-recorded one are not the same kind of fact and must not compete in the same priority ordering. Note the asymmetry, though: wasUserEntered is writer-set on iOS, so its absence proves nothing.

Building the priority order

Priority is per (user, metric). Never global, never per user alone. A user's chest strap is the best heart-rate source they own and a nonexistent step source; their phone is a decent step source and a useless sleep source. One ranking per user collapses those into a wrong answer for at least one metric.

A reasonable default ordering, which is our judgement and not documented behaviour anywhere:

1. Exclude, do not rank, manual entries for any metric that feeds scoring, streaks or coaching — but keep the rows. They are the user's data and belong in export and in the UI; they just should not compete with a sensor for the same interval. Excluding them from the ranking is only half the job: a typed set or a logged meal still needs a resolution strategy, and it is a merge rather than a priority order, which is the split argued in dedupe for measurements, merge for intentions.
2. Rank by measurement proximity for the metric, not by brand loyalty. For heart rate, a chest strap over a wrist device over an optical phone estimate. For steps, a wrist device over a pocket device. For sleep, whatever the user actually wears to bed.
3. Let the user override, and store the override per metric. This is what Health Connect does, and the reason is sound: the user is the only party who knows which device they actually wore on Tuesday.
4. Tie-break deterministically — shorter sample duration first (finer granularity smears less), then a stable key. Non-deterministic tie-breaks mean a re-run of the same resolution produces a different number, which turns a recompute into a support ticket.

Persist this table. It is an input to the resolution, and if you want to answer "why did this total change?" six months from now you need to know which ordering produced it. See metric versioning and recompute for the machinery around changing it safely.

Resolving overlaps: cut the day at every boundary

The naive approaches both fail on real data. Picking one winning source per day throws away every interval the winner did not cover — the user showered, the watch charged, the phone kept counting. Summing everything double-counts. What works is a sweep over interval boundaries:

1. Scope to one (user, metric, civil date). The civil date matters and is not a UTC range question; see timezones and day boundaries.
2. Collect every start and end instant from every candidate sample into a sorted set. Consecutive pairs define atomic sub-intervals over which the set of covering samples is constant.
3. For each sub-interval, find the covering samples and take the one from the highest-priority source.
4. Attribute that sample's value pro rata: value * (sub-interval duration / sample duration).
5. Sum the sub-intervals.

Step 4 is an assumption, and you should know you are making it: it assumes the rate is uniform within a sample. For a 10-minute step sample that is nearly harmless. For a 90-minute sample covering a sprint and a rest it is not. Apple's writer-side guidance — avoid samples 24 hours or longer, prefer samples of an hour or less for daily step counts — bounds the damage without eliminating it, and it is guidance to writers, not a guarantee to you.

Instantaneous samples must not enter this path at all. A weight reading has start == end, which makes the pro-rata denominator zero. Two scales reporting 81.4 kg three seconds apart is a coincidence-window problem, not a coverage problem, and there is no documented right window for it — pick one, write down why, and treat it as a tuning parameter you will revisit.

A worked example

Priority for this user and metric: Watch (1), Phone (2). Samples for 2026-07-26:

Naive sum: 7,500. Naive "highest-priority device wins the day": 2,500, which silently deletes an entire afternoon walk the user took with the watch on the charger.

Boundaries: 09:00, 09:15, 09:30, 10:00, 10:15, 14:00, 15:00.

Resolved total: 6,000. The 1,500 removed is exactly the phone's contribution over the 45 minutes the watch already covered. The 3,000 from the afternoon survives in full, and so does the phone's tail from 10:00 to 10:15 when the watch had stopped recording. That is what "interval-wise, not device-wise" buys you, and it is the same behaviour Apple describes for its own merge — duplicated data excluded, non-overlapping data from the second device added.

A predicate sketch

Postgres, one (user, metric, civil date) at a time. This is a sketch to think against, not production code.

WITH s AS (
  SELECT sample_id, source_key, value, start_ts, end_ts
  FROM   sample
  WHERE  user_id = $1 AND metric = $2 AND local_date = $3
    AND  end_ts > start_ts        -- instantaneous samples take another path
    AND  recording_method <> 'MANUAL_ENTRY'
),
bounds AS (
  SELECT start_ts AS t FROM s UNION SELECT end_ts FROM s
),
seg AS (
  SELECT t AS seg_start, LEAD(t) OVER (ORDER BY t) AS seg_end FROM bounds
),
covering AS (
  SELECT seg.seg_start, seg.seg_end,
         s.source_key, s.value, s.start_ts, s.end_ts, p.rank
  FROM   seg
  JOIN   s ON s.start_ts < seg.seg_end     -- strict on both sides:
          AND s.end_ts   > seg.seg_start   -- half-open intervals
  JOIN   source_priority p
         ON p.user_id = $1 AND p.metric = $2
        AND p.source_key = s.source_key
  WHERE  seg.seg_end IS NOT NULL
),
winner AS (
  SELECT DISTINCT ON (seg_start) *
  FROM   covering
  ORDER  BY seg_start, rank, (end_ts - start_ts), source_key
)
SELECT SUM( value
            * EXTRACT(EPOCH FROM (seg_end - seg_start))
            / EXTRACT(EPOCH FROM (end_ts  - start_ts)) ) AS resolved_total
FROM   winner;

The strict inequalities in the covering join are load-bearing. Using <= and >= admits samples that merely touch a boundary, which re-introduces adjacency as overlap and gets you a zero-width segment with a real winner. The DISTINCT ON ordering is the priority rule from the previous section, made explicit so the same inputs always give the same output.

The iPhone-plus-Watch case, specifically

This is the case everybody asks about, and the right answer is not to run the algorithm above.

On iOS, for quantity types, ask HealthKit. Run the anchored query to learn which days changed, then run an HKStatisticsCollectionQuery over those days to learn what those days now total, and send the recomputed daily figures upstream. That is Apple's own recommended shape from WWDC20, and it is the pattern that makes late-arriving and retro-edited samples survivable — see incremental sync for the cursor mechanics. Never sum anchored-query samples into a running total; the anchor tells you what changed, not what the day is worth.

Your resolution algorithm's job starts precisely where the platform's stops:

• Workouts. Statistics queries are documented as quantity-only. Two apps recording the same run produce two HKWorkout records and get no framework help at all. This is the case with the highest user-visible cost, because a duplicated workout duplicates its calories, its distance and its entry in a streak.
• Cross-provider totals on your server. When a HealthKit daily figure, a Fitbit daily figure and a Strava activity all describe the same Saturday, no platform is arbitrating between them. Nothing merges across vendors. That merge is yours, and it is where the algorithm above earns its keep.
• Everything on Android outside Activity and Sleep, plus Activity and Sleep whenever you need a number you can reproduce.

Say this out loud on your team, because it inverts most people's instinct: writing a step-merge algorithm for iOS is not a feature, it is a regression against a better algorithm you already have access to for free.

What we'd actually build

A raw sample table that is append-plus-tombstone and never edited in place, carrying source_key, source_revision, recording_method, the instant, the zone offset, and the civil date as a written column. A source_priority table keyed on (user_id, metric, source_key) with an alias column so one physical device that changed package names stays one source. A resolved_daily rollup keyed on (user_id, metric, local_date) that is recomputed by the query above rather than incremented, with the priority-table version that produced it stamped on the row. And a dirty_days queue that any late or edited write pushes into.

The property that matters is that the resolved number is a pure function of the raw data plus the priority table for one (user, day). "Is this number right?" becomes a question you answer by recomputing one cell and diffing, rather than by auditing the history of a counter. Incremental counters and health data are a bad pairing, because a counter that has drifted is indistinguishable from a counter that has not.

One invariant is worth monitoring from day one, catches real ingestion bugs, and needs no physiological knowledge: the resolved daily total must never exceed the naive sum. If it does, you are counting some interval twice and your coverage arithmetic is broken.

Resist the tempting second invariant — that the resolved total should never fall below the largest single source's total. It is false, and it will page you forever on correct data. If a watch at priority 1 covers an hour with 500 steps and a phone at priority 2 reports 3,000 for that same hour, the watch wins the overlap and the resolved total is legitimately far below the phone's. Resolution picks a winner per interval; it does not pick the largest. Data-quality monitoring covers the surrounding alerting.

The honest limits, and when to buy instead

You will not match HealthKit. Apple's engineer said so plainly, and the algorithm is unpublished. If your app has to explain to a user why your number differs from the Health app, the statistics query is your only route to agreement, and even then you cannot explain why.

Health Connect's aggregate is not reproducible and cannot be made so. The priority list is user-owned, there is no documented API to read it, and no change event fires when it moves. You choose between a number that matches what the user sees in system UI and a number you can defend. You cannot have both.

Pro-rata attribution is an assumption, and it is the weakest link in the algorithm above. It is defensible for short samples and increasingly wrong for long ones.

Resolution destroys information if you let it. Keep the raw rows. The resolved total is a derived view, and it will change — HealthKit condenses months-old workout data and deletes the originals, users edit history in the Health app, and providers retro-edit. A daily total is never final. Design for recompute, not for finality.

This is the single strongest argument for buying an aggregator rather than building. Not because the sweep is hard — it is thirty lines of SQL — but because the surrounding work is per-provider, per-metric, and permanent: knowing that this vendor's "active calories" excludes basal and that one's does not, that this vendor re-sends a whole day when one sample changes, that this device's step samples are hourly and that one's are per-minute. That table has to be maintained forever by someone, and when it is wrong you find out on real users' real bodies. If merged multi-source data is an input to your product rather than the product itself, buy it. Health data aggregator APIs covers the options, and what is a health data aggregator covers the vocabulary.

What would make us build anyway, honestly: if the resolved number is the product — you are selling the figure to an insurer, a clinician or a research protocol, and you need to state and defend the resolution rule. If you need a reproducible audit trail showing which source won which interval and why, which a vendor's opaque merge cannot give you. If you are single-platform on iOS, where the correct implementation is mostly "call the statistics query properly" and the build is genuinely small. Or if data residency or contractual constraints rule out a third party regardless.

And if you do buy: ask the vendor what their overlap rule is, whether it is interval-wise or source-wise, and whether re-running it on unchanged data returns an unchanged number. An aggregator does not remove this problem. It moves it behind an API, which is worth a great deal, but only if you know what it is doing.

Where to go next

Deduplication only makes sense on data whose units and semantics already agree — normalizing wearable data is the step that has to happen first, and what a step count actually is explains why two devices honestly disagree about the same walk. Everything on this page assumes you already know which civil day a sample belongs to, which is its own problem entirely.