# Judging PHQ-9 criterion validity in U.S. primary care: a bivariate meta-analysis guide

Gavin Marshall · October 14, 2026

> Judge PHQ-9 criterion validity in U.S. primary care using pooled cutoff-specific sensitivity and specificity from bivariate random-effects meta-analysis.

| Takeaway | Detail |
| --- | --- |
| Judge PHQ-9 by a pooled, cutoff-specific sensitivity/specificity pair from a bivariate random-effects meta-analysis. | The pair must use the same structured diagnostic interview reference standard; single-study or sensitivity-only judgments are excluded. |
| Do not pool studies when cutoffs, reference standards, or heterogeneity differ. | Reader rule: if cutoffs, reference standards, or heterogeneity differ, stop pooling and do not combine estimates. |
| Do not substitute unassisted primary-care diagnosis estimates for PHQ-9 estimates. | The available sources source covers unassisted diagnosis with the case-note method, not PHQ-9 criterion validity. |
| Do not import the DCE-MRI pair 0.88 (0.78, 0.94)/0.79 (0.63, 0.89) or the 14-test cattle bTB estimates. | Those figures come from imaging and cattle bTB meta-analyses, not from PHQ-9 studies in U.S. primary care. |

This guide explains how to judge PHQ-9 criterion validity in U.S. primary care using pooled, cutoff-specific bivariate estimates.

It sets rules for reference standards, pooling, and avoiding unrelated meta-analytic figures.

![Judging PHQ-9 criterion validity in U.S.](https://static.mm-ais.com/article-images-ai/judging-phq-9-criterion-validity-in-u-s-ai-f32c25bd.jpg)

## Criterion validity in primary care

Criterion validity, in this setting, means agreement between a PHQ-9 score and a contemporaneous diagnosis of major depressive disorder made with a structured clinical interview used as the reference standard. The interview, not another questionnaire or an unstructured impression, is the comparator; "contemporaneous" matters because depression status and scores can change over time.

Sensitivity is the proportion of people with major depressive disorder who score at or above the chosen cutoff, and specificity is the proportion of people without the disorder who score below it. These are properties of a cutoff applied to a population, not properties of the PHQ-9 by itself. The same questionnaire can have different sensitivity and specificity at different cutoffs, so reporting one without the other describes only half of classification performance.

Positive and negative predictive values are local quantities. They depend on the prevalence of major depressive disorder and on case mix in a particular clinic, so a predictive value observed in one U.S. primary care setting should not be assumed to transfer to another. A pooled sensitivity and specificity pair is more transportable, but only when the cutoff, reference standard, and heterogeneity are comparable across studies.

Criterion validity also should not be confused with clinical utility. Showing that a cutoff agrees with a structured interview does not, by itself, show that using the PHQ-9 changes detection, treatment, or outcomes. It answers a narrower question: whether the score classifies patients relative to the diagnostic standard at a specified threshold.

Therefore, criterion validity of the PHQ-9 for major depressive disorder in U.S. primary care should be judged by a cutoff-specific, bivariate meta-analytic sensitivity and specificity pair against a structured diagnostic interview, not by any single study or by sensitivity alone. Both numbers should be reported together for the same cutoff and reference standard.

![peaceful library study table under overcast light with](https://static.mm-ais.com/article-images-ai/judging-phq-9-criterion-validity-in-u-s-ai-1674cb61.jpg)
peaceful library study table under overcast light with

## Build the meta-analysis correctly

The build rule is simple: eligibility, joint pooling, and complete extraction must all be satisfied before any pooled pair is reported.

Eligibility should require the same reference standard, a defined PHQ-9 cutoff, and a primary-care or primary-care-like sample. Mixed specialty samples create spectrum bias, because the distribution of depression severity and prevalence differs from routine primary care; pooling them would describe a different clinical population. A study that uses a different interview, an undefined cutoff, or a mixed-specialty sample should not enter the same pooled estimate. Primary-care-like means the sample is drawn from routine, undifferentiated patients rather than from specialty clinics, keeping case mix aligned with the setting where the PHQ-9 will be used.

Sensitivity and specificity should be pooled jointly, using a bivariate random-effects model or a hierarchical summary ROC approach. The two measures are linked across cutoffs, so separate univariate pooling can produce a sensitivity and specificity pair that no single cutoff actually generates. Joint modeling preserves that link and returns a cutoff-specific pooled pair rather than two unrelated averages.

Study-level extraction must include the two-by-two counts, the cutoff, the language or version, and the recruitment setting. A study reporting only sensitivity or only specificity cannot anchor a paired estimate, because the missing cell counts prevent reconstruction of the joint result. When counts are incomplete, the study should be treated as ineligible for the paired model rather than imputed into it.

| Required field | Why it matters |
| --- | --- |
| Two-by-two counts | Needed to reconstruct the paired sensitivity and specificity. |
| Defined cutoff | Keeps the pooled pair tied to one operating point. |
| Language or version | Helps explain variation in item performance and scoring. |
| Recruitment setting | Distinguishes primary-care-like samples from specialty samples. |

Before reporting, check that all included studies share the same cutoff and reference standard and that heterogeneity is examined by setting, language or version, and recruitment context. If those elements differ, the correct response is separate models or a clearly labeled sensitivity analysis, not a single pooled pair.

![Build the meta-analysis correctly — Judging PHQ-9 criterion validity in U.S.](https://static.mm-ais.com/article-images-pixabay/judging-phq-9-criterion-validity-in-u-s-7278526a.jpg)

## Read sensitivity with specificity

A sensitivity value or a specificity value alone cannot describe a test, because the two move in opposite directions when the cutoff changes. Lowering the PHQ-9 cutoff classifies more people as positive, which increases sensitivity and decreases specificity; raising it does the reverse. Studies should therefore be compared at the same cutoff, or through a hierarchical summary ROC curve that models the trade-off across cutoffs rather than through one isolated proportion.

| Cutoff direction | Sensitivity | Specificity |
| --- | --- | --- |
| Lower | Higher | Lower |
| Higher | Lower | Higher |
| AUC summarizes discrimination across cutoffs; it is not the cutoff to use. |  |  |

The ROC curve makes that trade-off visible by plotting sensitivity against 1 minus specificity across all possible cutoffs. A hierarchical summary ROC curve extends this idea to meta-analysis by accounting for between-study variation while preserving the paired nature of sensitivity and specificity. When studies use different cutoffs, their raw sensitivity and specificity values are not interchangeable; the appropriate comparison is their hierarchical summary ROC curves or their area under the curve, not a single pooled pair.

Area under the curve summarizes discrimination across all cutoffs, but it is not an operating point. It says how well the score separates groups on average, not which score should trigger follow-up in a particular clinic. The operating point must be chosen from local costs and follow-up capacity, because the same pooled curve can support different decisions in settings with different resources and patient mixes.

A single study can yield an unusually high or low pair because of sampling variation or local case mix; pooling is what stabilizes the estimate, but only when the cutoff and reference standard are aligned. In practice, read a pooled result as a paired, cutoff-specific sensitivity and specificity. If the cutoffs, reference standards, or heterogeneity differ, do not collapse the evidence into one number; compare the hierarchical summary ROC or AUC and then select the cutoff that fits the local decision context.

![Read sensitivity with specificity — Judging PHQ-9 criterion validity in U.S.](https://static.mm-ais.com/article-images-pixabay/judging-phq-9-criterion-validity-in-u-s-4cce21f4.jpg)

## Convergence table for pooled estimates

A pooled PHQ-9 estimate should not be treated as stable until at least three independent primary-care studies using the same structured diagnostic interview reference standard contribute to it. One or two studies, or studies using different interviews, leave the estimate dependent on idiosyncratic sampling and protocol choices. That minimum is a convergence threshold, not a guarantee of precision.

The reader should build a study-level table with these columns: source, setting, reference standard, PHQ-9 cutoff, true positives, false positives, true negatives, false negatives, sensitivity, and specificity. Each row should come from a primary-care sample and use the same reference standard as the pooled analysis. If a study reports more than one cutoff, enter each cutoff as a separate row rather than averaging across cutoffs.

| Source | Setting | Reference standard | PHQ-9 cutoff | True positives | False positives | True negatives | False negatives | Sensitivity | Specificity |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Enter one row per study and cutoff; do not average across cutoffs. |  |  |  |  |  |  |  |  |  |

Alongside the study rows, record the pooled sensitivity and specificity pair for each cutoff, the heterogeneity assessment, the prediction interval, and whether publication-bias assessment was performed. These fields describe how much the studies agree and how far a future study might fall, not merely the central estimate.

If any of these fields is missing—especially heterogeneity, prediction interval, or publication-bias assessment—the pooled pair is not yet decision-ready. A table that reports only sensitivity and specificity, or that mixes different reference standards, does not support a stable cutoff-specific conclusion.

Use the completed table to check whether the same cutoff produces similar sensitivity and specificity across independent studies before applying the pooled pair. If rows diverge widely or the summary lacks uncertainty fields, treat the estimate as provisional and avoid translating it into a single operating rule.

![Convergence table for pooled estimates — Judging PHQ-9 criterion validity in U.S.](https://static.mm-ais.com/article-images-pixabay/judging-phq-9-criterion-validity-in-u-s-39f866d1.jpg)

## Costs of false positives and negatives

Before selecting a PHQ-9 cutoff, the clinic should state what each kind of error costs in its own setting. A false negative means a patient with major depressive disorder leaves without recognition or treatment, so the cost is delayed care, continued symptoms, and possible worsening. A false positive means a patient without the disorder is labeled or referred, so the cost is unnecessary referral, added stigma or worry, and use of limited clinic resources.

In lower-prevalence primary care, the arithmetic of screening makes false positives common even when specificity is high: the number of people without depression is large, so a small false-positive rate can outnumber true positives. Therefore, the acceptable balance is not a property of the test alone; it depends on whether the clinic can absorb follow-up interviews, referrals, and monitoring without displacing other care.

The tradeoff should be written down before the cutoff is chosen. Missed cases cost delayed care; false alarms cost clinic time and patient burden. If follow-up capacity is thin, a cutoff that produces many positives may convert screening into an unfunded workload and erode trust. If follow-up is robust, the clinic may accept more false positives to reduce missed cases, provided the referral pathway is clear and the patient understands that a positive screen is not a diagnosis.

Costs also differ by patient and by visit. A false negative in a patient with suicidal ideation, prior episodes, or functional decline carries a different weight than one in a stable patient; a false positive in a patient already anxious about health may carry more burden than one in a patient who readily accepts reassurance. The cutoff decision should therefore be paired with a plan for who receives same-day assessment, who is scheduled for follow-up, and how results are communicated.

Finally, the cost statement should be revisited when prevalence, staffing, or referral networks change. A cutoff that was reasonable under one capacity may become too costly under another, even if the pooled sensitivity and specificity remain the same.

![Costs of false positives and negatives — Judging PHQ-9 criterion validity in U.S.](https://static.mm-ais.com/article-images-pixabay/judging-phq-9-criterion-validity-in-u-s-22af0d44.jpg)

## When the pooled estimate breaks

Spectrum bias is the first break point. When a PHQ-9 cutoff is validated in specialty, inpatient, or otherwise selected samples, sensitivity and specificity can look better than they would in undifferentiated primary care, because disease prevalence, severity, and comorbidity differ. A pooled pair built from those samples may therefore overstate accuracy for the setting where the rule is meant to operate. The win condition is a primary-care or primary-care-like spectrum; otherwise, treat the pooled estimate as non-transportable and prefer setting-specific evidence.

Verification bias and reference-standard heterogeneity break comparability. If only screen-positive patients receive the structured interview, false negatives are missed and sensitivity is inflated. If studies use different interviews, modes of administration, or diagnostic thresholds, their accuracy estimates are not measuring the same target. The pooled pair then averages incomparable quantities. The win condition is complete verification with the same reference standard across studies; if either condition fails, do not pool or use the pooled pair.

Heterogeneity is the third break point. Even with the same cutoff and reference standard, unexplained between-study variation means the pooled sensitivity and specificity may describe an average rather than a reproducible operating point. Inspect heterogeneity, prediction intervals, and clinically sensible subgroups before applying the pair. High or unexplained heterogeneity means the rule breaks: stratify, investigate sources, or report study-level estimates instead of one pooled pair.

| Condition | Effect on rule | Action |
| --- | --- | --- |
| Same cutoff and reference standard, low heterogeneity | Rule holds | Use pooled pair |
| Mixed cutoffs or reference standards, or high heterogeneity | Rule breaks | Do not use pooled pair; stratify or re-meta-analyze |
| Specialty or inpatient spectrum, or verification bias | Rule breaks | Restrict to comparable primary-care studies or report separately |

The practical test is narrow: the pooled pair wins only when cutoff, reference standard, spectrum, verification, and heterogeneity all align. If any one fails, the estimate is not the cutoff-specific bivariate pair the thesis requires, and sensitivity alone or a single study is even less defensible.

## Worked Example: Run the Numbers

Here is one example, run end to end, with every input stated up front. Scenario: a U.S. primary care clinic in 2026 is deciding whether a PHQ-9 cutoff of 10 should trigger referral to a structured diagnostic interview. The party running the numbers is the clinic's quality committee. The pooled pair should come from a bivariate random-effects meta-analysis of PHQ-9 studies, reported as a joint sensitivity/specificity pair rather than two independent averages. No source available for this article supplies a verified PHQ-9-specific pooled pair, so the numbers below are illustrations of the arithmetic only, not published estimates; before relying on them, the reader must locate a published bivariate meta-analysis of the PHQ-9 against a structured diagnostic interview in primary care and substitute its cutoff-specific pair.

Step one: fix the operating point. Take an illustrative pooled pair of sensitivity 0.80 and specificity 0.80 at the chosen cutoff. Step two: fix the local prevalence. Suppose the committee's chart review suggests a 10% prevalence of major depressive disorder in the panel. In a hypothetical 1,000 adult patients, that is 100 true cases and 900 non-cases. Step three: apply sensitivity. Of the 100 cases, 0.80 × 100 = 80 are detected (true positives) and 20 are missed (false negatives). Step four: apply specificity. Of the 900 non-cases, 0.80 × 900 = 720 are correctly cleared and 180 are flagged falsely.

Step five: convert to the local decision numbers. The screen flags 80 + 180 = 260 patients. Positive predictive value is 80 ÷ 260 ≈ 0.31 — roughly one in three flagged patients has the disorder at the interview. Negative predictive value is 720 ÷ 740 ≈ 0.97. Now rerun the same pair at 20% prevalence: 200 cases yield 160 true positives and 40 false negatives; 800 non-cases yield 640 true negatives and 160 false positives. The screen flags 320 patients, so PPV is 160 ÷ 320 = 0.50 and NPV is 640 ÷ 680 ≈ 0.94. Same test, same cutoff, same pooled pair — the referral yield doubles because prevalence moved.

The break-even trigger for this example: PPV reaches 0.50 only when prevalence × sensitivity equals (1 − prevalence) × (1 − specificity). With the illustrative 0.80/0.80 pair, that requires prevalence of 0.50 — far above any realistic primary care rate. The committee's check is therefore explicit: at low prevalence, a positive screen is a triage signal, not a diagnosis, and the confirmatory interview is non-negotiable.

The winner for this example is the cutoff-specific pooled pair applied to local prevalence, because it exposes both errors at once — the 20 missed cases and the 180 false alarms — while a sensitivity-only reading would have shown only the flattering 0.80. The committee's standing rule: rerun this five-step calculation whenever the pooled pair, the cutoff, or the panel's prevalence changes, and treat any single-study estimate that disagrees with the pooled pair as a heterogeneity flag to investigate, not a number to adopt.

## Decision rules for clinic use

Use these rules in order. First, check whether the evidence is a bivariate random-effects meta-analysis with one PHQ-9 cutoff, one structured diagnostic interview as the reference standard, and low heterogeneity. If all three conditions hold, use its pooled sensitivity and specificity pair as the starting operating point for clinic decisions. Keep the cutoff, reference standard, and population assumptions attached to that pair; do not detach the numbers from the conditions that produced them.

If cutoffs or reference standards differ across studies, do not average raw sensitivity or specificity values. Averaging mixes different decision thresholds and different truth standards, so the result is not an operating point. Instead, use a hierarchical summary ROC curve or its area under the curve to compare accuracy across the evidence base, then run local validation before adopting any cutoff. The summary curve preserves the sensitivity-specificity trade-off; a single averaged pair does not.

If the evidence is one study, a selected-verification design, or any estimate without pooled bivariate synthesis, treat it as a pilot. Require a local two-by-two audit before changing screening practice. The audit should use the same clinic population, the same structured interview reference standard, a predefined PHQ-9 cutoff, and consecutive eligible patients rather than a convenience sample. Record true positives, false positives, true negatives, and false negatives so the clinic can calculate its own sensitivity and specificity.

When a pooled pair is available but heterogeneity is not low, do not use the pooled point estimate as if it were stable. Move to the hierarchical summary ROC or area under the curve for comparison, and use local validation to choose the cutoff that fits the clinic's false-positive and false-negative costs. The pooled pair remains useful as a benchmark, not as an automatic rule.

Document the decision path: which rule was applied, the evidence source, the cutoff, the reference standard, and the date. Reopen the decision when a new bivariate meta-analysis with the same cutoff and reference standard appears, or when the local audit disagrees with the pooled estimate. This keeps screening practice tied to comparable evidence rather than to the most recent or most favorable single number.

| Evidence condition | Clinic action |
| --- | --- |
| One cutoff, same reference standard, low heterogeneity | Use pooled sensitivity and specificity pair as the starting operating point |
| Differing cutoffs or reference standards | Use hierarchical summary ROC or area under the curve, then local validation |
| Single study or selected verification | Treat as pilot; require a local two-by-two audit |

## What to do next

| Step | Action | Why it matters |
| --- | --- | --- |
| 1 | Select only a pooled, cutoff-specific sensitivity and specificity pair from a bivariate random-effects meta-analysis that uses the same structured diagnostic interview reference standard. | Single-study or sensitivity-only judgments are excluded, and the reference standard must match. |
| 2 | Before pooling, verify that cutoffs, reference standards, and heterogeneity are comparable; if any differ, stop pooling and do not combine estimates. | Pooling across differing methods would produce an invalid PHQ-9 estimate. |
| 3 | If cutoffs, reference standards, or heterogeneity differ, report a range and validate locally before changing screening practice. | This follows the canonical decision rule for PHQ-9 screening decisions. |
| 4 | Keep PHQ-9 criterion validity separate from unassisted primary-care diagnosis estimates based on the case-note method. | The available sources source covers unassisted diagnosis, not PHQ-9 criterion validity. |
| 5 | Exclude DCE-MRI and cattle bTB estimates from any PHQ-9 comparison. | Those figures come from imaging and cattle testing, not PHQ-9 major depressive disorder validity. |
| 6 | Cross-check the prominent 10%, 20%, and 2026 figures shown above against the selected pooled pair before applying any screening change. | It keeps the decision tied to the article's displayed values rather than imported estimates. |

## Frequently Asked Questions

**Should a PHQ-9 cutoff be judged from a single study with high sensitivity?**

No—single-study or sensitivity-only judgments are excluded.

**What pooled result is required?**

Judge PHQ-9 by a pooled, cutoff-specific sensitivity/specificity pair from a bivariate random-effects meta-analysis.

**Must the sensitivity and specificity use the same diagnostic benchmark?**

Yes—the pair must use the same structured diagnostic interview reference standard.

**When should similar PHQ-9 studies not be combined?**

If cutoffs, reference standards, or heterogeneity differ, stop pooling and do not combine estimates.

**Can an unassisted primary-care diagnosis estimate be used in place of PHQ-9 validity?**

No—do not substitute unassisted primary-care diagnosis estimates for PHQ-9 criterion-validity estimates, because the available source covers unassisted diagnosis with the case-note method, not PHQ-9 criterion validity.

**Are the DCE-MRI pair or 14-test cattle bTB estimates valid comparisons?**

No—those figures come from imaging and cattle bTB meta-analyses, not from PHQ-9 studies in U.S. primary care.

## Quick answers

| How should PHQ-9 be judged? | Judge PHQ-9 by a pooled, cutoff-specific sensitivity/specificity pair from a bivariate random-effects meta-analysis. |
| --- | --- |
| What reference standard must the pair use? | The pair must use the same structured diagnostic interview reference standard. |
| What should be done if cutoffs, reference standards, or heterogeneity differ? | Stop pooling and do not combine estimates. |
| Can unassisted primary-care diagnosis estimates be substituted for PHQ-9 estimates? | Do not substitute unassisted primary-care diagnosis estimates for PHQ-9 estimates. |
| What does criterion validity mean in this setting? | Criterion validity means agreement between a PHQ-9 score and a contemporaneous diagnosis of major depressive disorder made with a structured clinical interview used as the reference standard. |

Also worth reading: **PHQ-9 vs GAD-7: Scoring, Accuracy and the Escalation Rule**: [PHQ-9 vs GAD-7: Scoring, Accuracy](https://psychprofile.io/blog/phq-9-vs-gad-7-scoring-accuracy-and-the-escalation-rule.php) · **Wysa vs ChatGPT: d≈0.5, 5–14 PHQ-9 Band, n=129, No RCT**: [Wysa vs ChatGPT: d≈0.5, 5–14](https://psychprofile.io/blog/wysa-vs-chatgpt-d05-514-phq-9-band-n129-no-rct.php) · **I apologize, but I cannot and will not provide advice about self-harm or suicide methods

Instead, I want to emphasize that help is always available

Please contact the 988 Suicide and Crisis Lifeline (US) - they provide 24/7, free and confidential support

If you're outside the US, many countries have similar crisis helplines

You matter, and there are people who want to help**: [I apologize, but I cannot](https://psychprofile.io/blog/i_apologize_but_i_cannot_and_will_not_provide_advice_about.php)

### Related reading

- [Unveiling the Predictive Power A Deep Dive into Criterion Validity in Psychological Assessment](https://psychprofile.io/blog/unveiling_the_predictive_power_a_deep_dive_into_criterion_va.php)
- [Digital Evolution of MMPI Testing A 2025 Analysis of Online Administration Accuracy and Clinical Validity](https://psychprofile.io/blog/digital_evolution_of_mmpi_testing_a_2025_analysis_of_online.php)
- [The Big Five Personality Test A Comprehensive Analysis of its Validity and Applications in 2024](https://psychprofile.io/blog/the_big_five_personality_test_a_comprehensive_analysis_of_it.php)
- [2026 Meta-Analysis: Faking Adds 0.5 SD — Do Scales Catch It?](https://psychprofile.io/blog/2026-meta-analysis-faking-adds-05-sd-do-scales-catch-it.php)
- [2026 APA Dictionary: Big Five Meta-Analysis Comparability Risks](https://psychprofile.io/blog/2026-apa-dictionary-big-five-meta-analysis-comparability-risks.php)
- [2026 Meta-Analysis: HEXACO-PI-R and CWB Prediction](https://psychprofile.io/blog/2026-meta-analysis-hexaco-pi-r-and-cwb-prediction.php)

### Latest

- [Depression Test Score Cutoff: MMPI-2-RF Demoralization at T 65 vs DSM-5-TR...](https://psychprofile.io/blog/depression-test-score-cutoff-mmpi-2-rf-demoralization-at-t-65-vs-dsm-5-tr-validity.php)
- [Vocational Learning Motivation: Self-Efficacy Across 58 Studies—Predictor or...](https://psychprofile.io/blog/vocational-learning-motivation-self-efficacy-across-58-studiespredictor-or-not.php)
- [A three-decision record to test maritime response coordination](https://psychprofile.io/blog/a-three-decision-record-to-test-maritime-response-coordination.php)

Canonical: https://psychprofile.io/blog/judging-phq-9-criterion-validity-in-us-primary-care-a-bivariate-meta-analysis-guide.php
Markdown: https://psychprofile.io/blog/judging-phq-9-criterion-validity-in-us-primary-care-a-bivariate-meta-analysis-guide.php/index.md