Misleading Metrics: Global Health Deserves a Higher Bar

After nearly two decades in consumer tech—where entire teams obsessively tracked funnels, retention curves, and growth—I pivoted into global health tech. Five years of building digital tools for frontline workers and patients in low-resource settings have taught me how easily top line metrics can be “gamed”[1] or optimized in a way that masks underlying issues—DAU (daily active users) looks impressive, but without retention, it’s meaningless in the long-term. With AI-driven tools on the rise in global health, our field must set higher standards.

Lessons from Consumer Tech

Optimizing for short-term metrics is seductive but hollow. A million DAUs don’t mean users find value, stay engaged, or trust the product. When I worked at Meta (2017–2020), every new PM learned how metrics can be manipulated - consciously or not. We built dashboards with counter‑metrics to catch false wins—because unchecked focus on user growth led to wasted investment, churn, and worst of all- broken user trust.

Why It Matters in Health

In digital health, especially in low- and middle-income countries, we see “active user” counts disconnected from real outcomes—like behavior change or service uptake. The rigor of consumer analytics isn’t considered as relevant in the health world. Many times, we talk about “trust” only qualitatively, missing behavior-based proxies. When budgets are shrinking, and innovation dollars are under increasing pressure—admitting failure or missed user targets can threaten continued funding. Successful consumer organizations like Meta or Microsoft expect 50% of big-bet innovation projects to fail (ideally fast)- and the other 50% to learn, pivot, invest in efficiency, and continue to iterate until they achieve product-market fit (PMF)[2], which unlocks the ability to cost-effectively scale. This can take time, patience, and importantly - an honest interrogation of metrics to build a deep understanding to chart a responsible path forward.

Audere’s Approach: Measuring What Matters

At Audere, we’ve adapted the discipline of consumer tech analytics to the realities of global health—because in life-or-death contexts, metrics must reflect real value.

We don’t just count how many devices run our tools. We ask: What changed because of them? For instance, our computer vision (CV) technology for malaria diagnostics isn’t evaluated by deployments or users alone. We track whether it impacted the test positivity rate (TPR)—a meaningful system-level outcome. In one multi-country study in sub-Saharan Africa[3], TPR improved in study sites - very likely due to digital monitoring. CVs specific value was creating a trusted, scalable “ground truth” for test results—crucial for surveillance and quality assurance. Metric shift attribution - sometimes at a granular feature level within digital products - is important to get right, and isn’t always clear. This level of attribution is important to inform product investment strategy - where we may be able to deduplicate efforts, and where our organization’s unique contribution to the ecosystem lies.

Where the CV directly supports clinical workflows, we go deeper:

Did it improve human decision-making? (or importantly - adversely impact it?)
Did it reduce time to result during high-volume testing campaigns?
Did it reduce friction? (or introduce friction, and what was the impact?)
Did it lead to the right patient treatment?
Do the healthcare workers trust it?

We obsess over latency and error rates—because every second matters in a pharmacy during rainy season malaria surges.

The same discipline applies to our consumer-facing AI. We don’t define success as someone simply interacting with a chatbot. Our engagement threshold is 2+ meaningful back-and-forth exchanges with the AI companion, completion of a self-test and result upload, or verified linkage to a clinician. Each tells us important information about where consumers find value, and actions such as a test result upload are an important proxy for intention to uptake services. But the follow-through on figuring out how to track service uptake, one of the hardest metrics in fragmented health systems - is crucial.

In South Africa, a user may uptake services via private doctor, public clinic, local CBO, or choose to self-test—so we track a variety of avenues to ensure we’re capturing as accurately as we can via backend integrations for confirmed service uptake, referrals, and using AI agents to identify self-reported care.
We also recognize that for some users, especially in stigmatized care contexts, information is the service. But unless we can confidently identify intent, we don’t count those interactions toward impact metrics.

This kind of measurement discipline with real-time monitoring, lets us course-correct fast. It also gives us the confidence to say: this feature isn’t working as intended yet—and that’s okay. We’ve built internal systems and partner expectations around learning, not just storytelling. It saves us time, builds trust, and helps us focus our energy where it matters most.

Learning from Failure

Often, sound strategy is built upon tough lessons - I learned this the hard way in 2017, when a new feature my team launched in Messenger’s group calling led to an exciting DAU surge that within a couple of months, reversed due to a memory leak. Performance suffered, retention dropped, and user trust tanked. It took nearly a year of rigorous technical investment and experimentation to regain users - and even longer to regain trust. That taught me: top line growth means nothing if the underlying experience isn’t stable and performant. Additionally, you can’t fix what you don’t measure - and that lesson informs how I think about success, metrics, and counter metrics - and subsequently our intense focus on performance, meaningful usage, retention, and outcomes.

Culture of Transparency

We ask tough questions upfront: Are users returning? Did behavior change? Do we have budget and time to measure meaningful impact? In weekly Data Deep Dives led by our data scientist Sam Smedinghoff, we “wallow in the data” as a team (yes, everyone from our QA team to our CEO): cohort analysis, hypothesis generation, action-oriented discussion, and decisions on what early signals mean, who to share with ASAP to inform time-sensitive decisions, even at the hypothesis stage, with implementing partners, governments, and donors. Sometimes these sessions are a run-of-the-mill review of dashboards, identifying where we may need to investigate, but the really exciting sessions are when Sam’s had time to go spelunking for insights to add color to the metrics, and inform our thinking (like the time he identified RDT Re-use - more on that in a future blog!). Every session ends with actionable next steps—whether additional analysis/research questions, product tweaks, partnership insights, or a need for new instrumentation to capture additional data.

Building Truth, Not Just Traction

Metrics should tell the full story: not just growth milestones, but sustainable value. Yes, it’s tempting to cherry-pick stats for reports and grants - inflating impact through misleading metrics. But if we do, we risk undermining sector credibility—consumers lose trust, funders make misguided decisions, governments scale ineffective tools, and we continue to miss SDG targets collectively.

Looking ahead

In our next blog, we’ll introduce our Hybrid M&E framework, one we first developed to evaluate computer vision (CV) tools for diagnostics and have since adapted and expanded for real-time, real-world LLM-powered interventions.

Audere doesn’t always get it right—but we try to be relentless in measuring what we want to improve, not just what makes us look good. We’ve borrowed and adapted the rigor from consumer tech to apply to global health solutions. Together, in digital health, we need to build the muscle — and the courage — to report the real numbers. To design for real value. To measure what matters. To fail fast and pivot. Let’s build tools—and metrics—that reflect real impact and honor the lives we aim to change.

Citations
[1] https://arxiv.org/pdf/2002.08512
[2] https://stripe.com/resources/more/what-is-product-market-fit-what-startups-need-to-know
[3] https://malariajournal.biomedcentral.com/articles/10.1186/s12936-025-05459-7#citeas