OpenMHC Benchmark

OpenMHC Benchmark

OpenMHC Leaderboard

Wearable & mobile health benchmark on MyHeartCounts. Track 1 (predictive tasks) predicts self-reported health outcomes — demographics, medical risk, vitals, mental well-being, and lifestyle — from a participant's wearable history, with each model free to choose its own temporal resolution and context length. The generative tasks: Track 2a (imputation) reconstructs masked daily, minute-level signals, and Track 2b (forecasting) predicts future hourly signals. Each method is ranked by skill score vs a track baseline, computed live from the per-user evaluation substrate.

📤 Submit a model ⚙️ Code 📊 Dataset · coming soon 📄 Paper · coming soon 🏠 MyHeartCounts 🤗 Models
Cohort

Track 1 · Predictive Tasks

All participants · 1,762 users evaluated

#MethodType ▾R ↓S ↑S_worst ↑Demo. ↑Medical ↑Vitals ↑Mental ↑Lifestyle ↑Fallback ↓Submitter
1LSM-2Wearable Foundation Models2.4+16.5+14.8+41.1-1.3+27.0+1.6+14.00.0OpenMHC team
2MultiRocketStatistical Models3.5+8.1+5.6+27.7-4.8+9.2+1.6+6.90.0OpenMHC team
3XGBoostStatistical Models3.6+12.3+9.6+47.1-1.8+16.8-6.7+6.20.0OpenMHC team
4WBMWearable Foundation Models4.4+5.3+2.3+23.8-2.6+8.8-2.6-0.762.7OpenMHC team
5Linear (baseline)Statistical Models4.70.00.00.00.00.00.00.00.0OpenMHC team
6GRU-DSupervised Neural Models5.2+3.6+0.6+19.6-4.1+10.4-7.1-0.60.0OpenMHC team
7Chronos-2Time-Series Foundation Models5.9-3.3-4.1+10.8-9.4-15.6-2.9+0.90.0OpenMHC team
8TotoTime-Series Foundation Models6.2-5.5-7.5+1.0-8.8-11.5-6.8-1.50.0OpenMHC team

Metric legend — Track 1 predicts one self-reported health outcome per participant from their wearable history; scores are computed vs the Linear baseline.
R (Average Rank) — mean cross-method rank across the outcome tasks; 1 = best (lower is better).
S (Skill Score) — category-balanced % improvement over Linear across tasks (per-task AUPRC / Spearman / Pearson, paired-bootstrap mean); higher is better.
S_worst (Worst-subgroup skill) — the method's lowest skill score across the four demographic cohorts (age <40 / 40+, female / male): its performance for the worst-served subgroup; higher is better.
Demo. / Medical / Vitals / Mental / Lifestyle — per-category skill on that outcome group's tasks (Demographics; Medical Conditions & Risk; Vitals & Blood Biomarkers; Mental Well-Being; Sleep & Lifestyle), category-balanced like S.
Fallback — % of test predictions substituted by the Linear baseline when the method produced no valid output (lower is better).
Source: MyHeartCounts/OpenMHC-leaderboard-data.
Cohort — the selector above recomputes R, S, and the per-category skills within one demographic cohort (a near-median age split at 40, or sex); the count above each table is the cohort's evaluated users. S_worst is the minimum cohort skill across the four cohorts, so it reads the same in every view. Cohorts are smaller than the full population, so scores are noisier; tasks without enough users (or degenerate within the cohort) drop from the aggregates.

Submit your model

Add a method by opening a pull request on the OpenMHC leaderboard dataset that adds your per-user evaluation substrate (<track>/<method>.parquet) plus a small <method>.meta.json sidecar. Produce the substrate by running the OpenMHC eval with output_dir=…; the maintainers recompute the skill, rank, and worst-subgroup scores from it. See the step-by-step submission guide for the exact file schema.