INTRODUCTION

A key element of primary care, “comprehensiveness” is the extent to which the primary care clinician, as part of the primary care team, recognizes and meets the majority of each patient’s physical and common mental health care needs, including prevention and wellness, and acute, chronic and comorbid condition management.1,2 Comprehensiveness includes both the scope of services offered and the depth and breadth of conditions managed by the primary care team, based in part on the prevalence of health concerns and conditions in the population served. Comprehensiveness, along with accessibility, continuity, coordination and accountability for the whole person, differentiates primary care from other forms of care (e.g. specialty care, hospital care).1–5

Comprehensiveness is associated with numerous benefits,1,5–12 yet several studies suggest it is declining in the U.S. and Canada.13–17 Here, we discuss the rationale for measuring comprehensiveness in primary care, enumerate the challenges, including available data sources, then describe approaches to measuring comprehensiveness and suggest areas for future research.

Why Measure Comprehensiveness of Primary Care?

Comprehensiveness, measured primarily by survey and administrative data, is associated with better outcomes for patients and populations. Greater efficiency, defined as a wider range of services, with better health outcomes, provided at lower costs,5–10 lower hospitalization rates for ambulatory care-sensitive conditions,8,11 improved health and better self-reported health outcomes,6,8,9,11,12 and greater equity (i.e., reduced disparities in disease severity as a result of earlier detection and prevention across different populations),5–10 are all linked to more comprehensive primary care. Without comprehensive primary care, patients experience fragmented care across numerous different specialists, resulting in higher healthcare costs, more diagnostic tests and interventions, and more types of medications.1,8,18

The decline in comprehensiveness of primary care is not an inevitable aspect of modern health care. In the Netherlands, for example, comprehensiveness appears to be increasing.19 In fact, many countries lead the U.S. in comprehensive primary care.1,20 The U.S. decline may be due to numerous factors, including the dramatic growth in the number of specialists, health plans that enable direct access to specialty care for common conditions, specialist-oriented medical centers, and insufficient support of and payment for practices to comprehensively manage the expanding population of patients with comorbidities.1,21–32

In addition, patient-centered medical home (PCMH)33,34 standards include other primary care features, such as accessibility and coordination,35–38 but underemphasize comprehensiveness.39,40 Some include a one-time health risk assessment and reminders about individual services for “individual conditions.” Although some PCMH recognition tools include care management plans that begin to address the notion of comprehensiveness, they seldom highlight the scope of services offered and the extent to which conditions and comorbidities are managed by the primary care practice. While standards are increasingly emphasizing the important element of care coordination, primary care practices may be tracking specialist referrals for every problem patients have (including common primary care problems) without managing any of these problems directly.

Primary care providers may need more resources to deliver comprehensive care, and without measurement and support for its improvement, comprehensiveness may further whither as other measured aspects of primary care improve.

General Challenges to Measuring Comprehensiveness

The under-measurement of comprehensiveness results from several factors, including the lack of a common vocabulary, patient population differences, insufficient agreement across specialties, and inadequate data sources.

Lack of Common Vocabulary.

Some studies have measured comprehensiveness by the “scope” or “range of services.”1,30,41 For this paper, “scope of services” means the types of services a primary care practice provides (e.g., nutrition counseling, family planning, minor laceration repair, etc.). However, we also emphasize a factor that has received even less attention: the depth and breadth of conditions and problems the primary care clinician and team manage. This includes not only preventive care and acute, chronic and multi-morbid conditions, but also the range of problems and conditions within these areas that a generalist manages without immediate referral to a specialist.1,5

Needs of Population Served.

Common comprehensiveness definitions focus on meeting the large majority of patients’ needs, meaning “a range of services broad enough to care for all health needs except those too uncommon to maintain competence.”1,2,4 Population needs determine the depth and breadth of conditions and scope of services a practice should provide. If demand for particular services is insufficient for a practice to maintain the necessary skills and equipment, referral elsewhere may be needed.

Training May Not Equal Practice.

Practicing clinicians with primary care training do not necessarily provide comprehensive care. Despite receiving training in obstetrics, many family physicians do not provide obstetrical care,42 and some internists trained in primary care competencies practice inpatient care exclusively31,32 or in other narrow venues of care.43,44

Lack of Professional Agreement on Respective Roles and Indications for Referral.

Comprehensive primary care providers make fewer referrals to specialists.1 The range of practitioner types in a practice and their combined skill sets affect the patient needs the practice can meet.29 Despite efforts to outline the problems that are most common in primary care1,41,45,46 and to study referrals,22,47,48 disagreement remains on when referrals are appropriate.49,50 The limited number of metrics on referrals further complicates measures of comprehensiveness.22

Inadequate Data Sources.

Different data sources capture different aspects of comprehensiveness.51 Claims and surveys can provide some information on what services have been provided, but like electronic health record (EHR) data, they have limitations. Various types of data currently available for measuring comprehensiveness have relative advantages and disadvantages.

Survey-Based Measures of Comprehensiveness

Patient, provider and practice surveys have measured different aspects of comprehensiveness and have provided varying perspectives.

Leading patient surveys with items on comprehensiveness include the Primary Care Assessment Tool (PCAT),52 the Primary Care Assessment Survey (PCAS),53 and the Ambulatory Care Experiences Survey (ACES).54 The adult and core Consumer Assessment of Healthcare Providers and Systems (CAHPS)-PCMH surveys, which are commonly used to evaluate patient experiences in PCMH demonstrations, include items on some aspects of comprehensiveness, (e.g., whether the clinician pays attention to patients’ mental and emotional health and whether preventive, acute and chronic care is provided).55 None of these, however, collect information on the breadth and depth of conditions managed.

Most provider surveys that touch on comprehensiveness focus on the scope of services provided. The PCAT52 has both physician and facility versions that ask whether the practice provides nutrition counseling, immunizations, family planning, behavioral health counseling and minor laceration repairs. The Medical Home Implementation Quotient (MHIQ)56 asks about the scope of procedures and whether the practice provides acute, chronic and prevention services and end-of-life care. While questions address such practice functions as identifying patient medication barriers and monitoring progress on outcomes, the MHIQ does not probe specific patient conditions, and provides less insight into the breadth and depth of managing specific problems or a range of conditions. Other national physician and practice surveys57,58 have robust items on practice supports for selected chronic conditions, but do not ask specifically about the aspects of comprehensiveness that are the focus of this paper.

The Quality and Costs of Primary Care in Europe (QUALICOPC) project, which extends beyond Europe but does not include the U.S., analyzes and compares primary care in 35 countries.59 This ongoing effort includes numerous surveys, including one of primary care physicians (“GPs”) that covers comprehensiveness.59 After a review of existing instruments, the QUALICOPC study measured aspects of comprehensiveness in its physician questionnaire, including available equipment and the GPs’ task profiles (e.g., range of problems for which the GP is the first point of contact). The survey thus resulted in some conceptual overlap between the concepts of first-contact accessibility3 and comprehensiveness.

A new measure, developed but not yet validated, for the Agency for Healthcare Research and Quality's (AHRQ) nationally representative physician survey pilot60 asks about five common symptoms and chronic conditions (new-onset low back pain, amenorrhea, depression, sore throat, and diabetes). For each of these, both primary care and specialist physicians are asked about the extent to which the physician/practice (1) conducts the history and physical exam for the initial assessment, (2) orders and interprets the necessary diagnostic tests, (3) initiates treatment, and/or (4) refers the patient to a different health professional. These new items address depth of management for specific conditions, enabling comparisons on comprehensiveness among different specialties.

Advantages and Limitations of Survey Measures of Comprehensiveness

Clinician surveys on scope of services enable clinicians to report on services provided by practices, whereas claims data reflect only services billed under a specific plan benefit. Clinicians' reports on their degree of management of conditions, however, may be more subject to social desirability bias (over-reporting of the extent to which they provide comprehensive care) than items on the types of procedures a practice offers. Table 1 compares various forms of data collection on comprehensiveness of care.

Table 1 Quality Comparison of Survey- vs. Claims- vs. Chart-Based Measures of Comprehensiveness of Care
Full size table

While patient surveys are useful for determining whether patients feel that their needs have been met, they do have limitations. Patients may give clinicians high ratings on comprehensiveness, as they do with factors such as satisfaction and trust, in part because the clinician does things the patient values or because they have a long-standing relationship.61,62 Patient ratings may also be influenced by their beliefs about the relative merits of primary care physicians vs. specialists and by the prevailing views in their community,63,64 rather than whether the clinician is providing the professionally appropriate level of comprehensiveness.65 Additional modes of data collection to capture other aspects of comprehensiveness are likely needed.65

Claims-Based Measures of Comprehensiveness

Some studies have measured comprehensiveness across physician specialties using Medicare claims and visit abstraction data from the National Ambulatory Care Medical Survey (NAMCS).41,45,46,66–70 Several potential claims-based approaches may be useful for estimating a practice’s comprehensiveness.

  1. a.

    Range of conditions treated:

Investigators at the Robert Graham Center have proposed an approach based on the theory that clinicians who treat a larger number of conditions provide more comprehensive care. This method entails analyzing the number of conditions listed for evaluation and management (E&M) visits, as identified by the International Classification of Diseases, Ninth Revision, Clinical Modification (“ICD-9-CM") codes.71 Because claims may list co-conditions that physicians are not necessarily treating, the Graham Center measure utilizes a ranked frequency distribution of all conditions treated over the course of a year by each physician. It then tallies the number of ICD-9-CM codes that account for the distribution below a given threshold value (e.g., 80% of ICD-9 codes). Flatter distributions indicate more comprehensive care, while those skewed to the left indicate less comprehensive care. For example, the group determined that 91% of family physicians billed for 12 or more ICD-9-CM codes for 80% of their practice. Using this method, 85% of internists and 95% of geriatricians would be deemed to be providing comprehensive care.72 Developed for broad comparisons of different types of physicians in different regions, this approach is being tested for use in comparing the relative comprehensiveness of individual physicians.

Rosenblatt et al. used an alternative approach in which 120 separate “diagnostic clusters” were identified in order to capture the range of conditions listed on claims.41,45,69 Diagnostic clusters group individual diagnostic codes to bring together conditions with similar pathophysiologic characteristics that tend to be treated in similar ways. They simplify the ICD system to manageable proportions and compensate for idiosyncratic labeling and coding by individual physicians.45 The top 20 clusters were found to represent more than 50% of all non-referred visits to physicians, and captured the most common ambulatory conditions for which people sought care. The three primary care specialties (family medicine, internal medicine, and pediatrics) deal with the broadest range of diagnostic clusters. Other specialists, including the areas of obstetrics-gynecology, cardiology, and oncology, typically treat one to three major diagnostic clusters.41

Bindman and Forrest used the expanded diagnostic clusters from the Johns Hopkins University ACG case-mix system73 to describe the comprehensiveness of primary care in an international comparison of data abstracted from office visits.20

These approaches cannot capture whether the physician has actually treated all of the conditions designated by the ICD codes on claims, and physicians may fail to identify some important health conditions (e.g., depression) during the primary care encounter. Thus there are limitations to measures based on the range of conditions documented by the primary care provider.

  1. b.

    Involvement in patient conditions:

An alternative approach estimates the degree to which the primary care practice provides services within the constellation of care that the patient receives. The assumption is that when patients receive more services from specialists, the primary care physician is providing less comprehensive care.

To assess “involvement in patient conditions,” one could identify all conditions (ICD codes) in each patient’s E&M claims for all physicians using claims for Medicare fee-for-service beneficiaries over a 1-year period. Determining the percentage of various conditions managed by each clinician for a beneficiary and ordering clinicians based on who saw the greatest number of different conditions would identify the top clinician as the “comprehensive clinician” for that beneficiary. Clinician scores could be assigned based on the ratio of the number of beneficiaries for whom they were the comprehensive clinician to the total number of beneficiaries they saw.

  1. c.

    New problems managed:

A comprehensive primary care practice should be able to manage most new problems that patients might experience.1 In this context, another approach estimates the extent to which a clinician manages vs. refers patients with new problems commonly seen in primary care. Primary care clinician encounters for new issues could be identified by examining a defined “look-back period” (e.g., 3 years) to ascertain that the problem (ICD code) was not previously listed for a patient. Looking forward from the index claim for that new problem in order to see who billed for it would indicate who managed it. One would then calculate the percentage of total E&M visits associated with the new problem as provided by the clinician (numerator) vs. by all clinicians (denominator).

  1. d.

    Scope of services provided:

Claims data can also be used to assess the scope of services a primary care practice delivers. Chan et al. (2002) reviewed the settings in which services were provided (e.g., outpatient, inpatient, ED, etc.) using FFS claims from practices in Ontario as a comprehensiveness indicator.13 The Robert Graham Center is also using subsets of Berenson-Eggers Type of Service (BETOS) codes)74 to count the different types of services, procedures and locations of care (e.g. office, hospital, home, ER) for which clinicians bill.72

Combinations of claims-based measures could be complementary to ensure that both scope of services and the breadth and depth of conditions managed are captured.

Advantages and Disadvantages of Claims-Based Methods:

Advantages to using claims data include their availability, relatively low cost, and, for Medicare data, their national scope. Claims include both clinical and procedural codes and site of care (outpatient, inpatient, ED, etc.), are consistent across settings, and may capture the breadth and depth of conditions recognized and treated by a practice better than surveys. Claims also include services patients have received outside their usual provider’s or health system’s EHR, something the current lack of interoperability makes challenging to gather with EHR data across providers.

Disadvantages to using claims are numerous, and the extent to which different approaches are subject to them varies. Tables 2, 3, and 4 compare the four main claims-based measures described above. One challenge is identifying the degree to which the clinician is truly managing conditions s/he lists on a claim. Clinicians or their practice may be assuming responsibility only for some of the multiple conditions reflected by ICD codes on a patient’s claim. This may be inadvertently exacerbated by current risk adjustment documentation requirements to maximize reimbursement.75,76 Because traditional Medicare claims do not require the identity of the referring provider or whether specialist visits were due to referrals, it can be difficult to tell whether a primary care clinician managed a condition or simply recognized and documented a problem and then referred the patient to a specialist (“document and refer”).

Table 2 Comparison of Relevance and Feasibility of Claims-Based Measures of Comprehensiveness of Care
Full size table
Table 3 Likelihood that Various Measurement Approaches Capture Various Aspects of Comprehensiveness
Full size table
Table 4 How Claims-Based Comprehensiveness Measures would Differ for Different Clinician Types
Full size table

Systematic differences in coding behaviors among provider organizations or local markets might also skew claims-based measures of comprehensiveness. For example, research suggests regional market variation in health services use,63,64,77 and in markets with high use of services, patients may have more ICD codes identified. Thus, clinicians in some markets might appear more comprehensive when they are just collectively doing more for patients.

A related problem is the lack of sufficient clinical information in claims data to assess the clinical appropriateness of services delivered. There are a few exceptions to this, e.g., management of low back pain, for which guideline-based claims algorithms exist.78

Given that 75% of Medicare beneficiaries are in the fee-for-service program and are not required to identify a primary care provider, we focused the above discussion on claims-based methods that would not first require attribution of beneficiaries to a usual clinician. Such attribution algorithms79 might create circularity if the same claims were also used to assess comprehensiveness. Additional measures might be available, however, in health plans or systems where patients have an identified primary care provider. For example, after adjusting for case mix, primary care providers whose patients had a higher proportion of specialist visits in a given time period could be designated as less comprehensive.

Directions for Future Research on Measuring Comprehensiveness of Care

Identification of Population-Based Health Needs.

To determine whether a practice meets the majority of its patients’ health needs, ideally, one would first identify those needs (ranging from preventive to comorbid) and the services required to address them. This would optimally include data that is not subject to current fee-for-service payment incentives, as current claims fail to capture the full extent of population health needs (e.g., people without access to care or care for unreimbursed services.) A future gold standard for what comprehensive primary care would entail could be developed once a population’s needs are identified. Pairing epidemiologic research on population health needs with concurrent measures of provider comprehensiveness could advance discussions on where practices fall short, or do well, in meeting patient needs.

How do we Validate a Measure when there is no Gold Standard?

In the absence of a “gold standard” against which to assess concurrent validity of comprehensiveness metrics, an examination of relative characteristics of different measures could help determine their criterion and predictive validity. For example, a comparison of claims-based measures to either (1) survey and EHR measures of comprehensiveness for the same individuals, or (2) important patient outcomes, including quality measures, hospitalizations, efficiency and costs, could be informative.

Another approach to validating comprehensiveness measures might compare competency standards of certification bodies, such as the American Board of Internal Medicine, the American Board of Family Medicine, or the Accreditation Council for Graduate Medical Education, to physician survey and claims-based measures of comprehensiveness. Competing visions of primary care competencies by different generalist disciplines46 and an inability to capture from claims some activities outlined by professional associations (e.g., end-of-life planning) might present challenges.80 The core services and conditions managed in primary care in other Western industrialized democracies could also inform our consideration of a “gold standard” for comprehensive primary care.

What is the Potential Role of Clinical Information Systems in Measures of Comprehensiveness?

EHR data has potential relevance for several research and future quality measures, including comprehensiveness. Unlike claims data, the EHR can track care delivered outside of face-to-face encounters, although it may be susceptible to some of the same weaknesses as claims data (e.g., the requirement of documentation for fee-for-service billing). Studies have used EHR data to identify care processes within and across providers in closed delivery systems and regional health information exchanges.81,82 Once more clinically meaningful data (e.g., on referrals, condition management over time, and comorbidities) become routinely collected, standardized and aggregated across providers, EHR data could help to determine which providers are the most comprehensive. In the meantime, the respective strengths, weaknesses and validity of both claims-based and e-measures of comprehensiveness should continue to be assessed.

While most ambulatory care clinicians are still paid on a fee-for-service basis, CMS and other organizations are developing various alternative payment models. As these models still rely on information in fee-for-service claims, this data source will be available for years to come,83 but claims may become less informative over time, necessitating greater emphasis on EHR measures.

Other systems for classifying patients and services might also facilitate an understanding of how to support comprehensiveness. The International Classification of Primary Care (ICPC), for example, incorporates not just billing data, but also data on symptoms and reasons for encounters in the patients’ own words in addition to provider assessments.84 EHR versions of the ICPC are available in Europe, but have been used in only a few institutional EHRs in the U.S. ICPC has been cross-walked to ICD-9 and ICD-10.85,86 The future iteration, ICPC-3, may provide a structure to link the patient’s reason for a clinical encounter with the processes of care and clinician assessment, and then aggregate data across series of encounters to capture care over time. Merging the ICPC-3 with more clinically meaningful EHR data from clinicians has the potential to improve our understanding of the content and comprehensiveness of primary care.84

CONCLUSION

Comprehensiveness is an important aspect of primary care. Developing valid and reliable measures of comprehensiveness could inform quality improvement efforts and help identify providers in need of additional support. Developing an understanding of needs and expectations for comprehensive primary care may also inform efforts to prepare the primary care workforce and shape expectations for maintenance of primary care competencies. If comprehensiveness remains an under-measured expectation, the current incentives that discourage it may exacerbate its decline.