CVS-Oak Street: Unpacking AI’s Regulatory Challenge in Primary Care

The rapid integration of artificial intelligence into healthcare delivery models, particularly within primary care, presents a complex regulatory challenge. As large corporate entities acquire and scale AI-driven clinics, policymakers face a critical need for frameworks to assess these proprietary systems. This article provides an evidence-based approach for regulators, anchored in a detailed analysis of Oak Street Health’s AI model post-acquisition by CVS Health.

The New Landscape: Corporate Ownership and Algorithmic Primary Care

The recent wave of acquisitions, notably CVS Health’s $10.6 billion purchase of Oak Street Health and Amazon’s acquisition of One Medical, signals a profound shift in the healthcare landscape. These transactions are not merely market consolidation; they represent the establishment of large-scale laboratories for deploying proprietary AI within value-based care frameworks. This creates a fundamental tension for policymakers: the undeniable promise of technological innovation to improve health outcomes and reduce costs, versus the inherent risks of unregulated “black box” medicine operating at scale. The growth of value-based care models, with Medicare Advantage plans enrolling 47.9% of Medicare patients in 2023, and reaching 51.8% as of April 2026, amplifies the urgency of this regulatory oversight. CMS data on Medicare Advantage enrollment Corporate entities are increasingly leveraging AI to optimize patient stratification, resource allocation, and care pathways. While these systems aim to enhance efficiency and personalize care, their proprietary nature often obscures the underlying algorithms, data inputs, and decision-making logic. This opacity challenges traditional regulatory mechanisms designed for tangible medical devices or pharmaceuticals, demanding a new, data-driven approach to ensure patient safety, efficacy, and equitable access.

Case Study: Deconstructing the Oak Street Health AI Model

Oak Street Health, now a part of CVS Health, operates a primary care model specifically designed for older adults with complex chronic conditions, predominantly serving Medicare Advantage beneficiaries. Central to their operational strategy is a proprietary AI system that underpins their approach to proactive care and risk management. Understanding this model requires a clear distinction between its stated purpose and its demonstrable outputs.

The “Canopy” Platform and Predictive Analytics

Oak Street Health’s core technological asset is its “Canopy” platform. This system is designed to aggregate and synthesize vast amounts of patient data from disparate sources, including electronic health records (EHRs), claims data, and increasingly, social determinants of health (SDOH) information. The stated purpose of Canopy is to generate predictive risk scores for individual patients. These scores aim to identify patients at high risk for hospitalization, emergency department visits, or other adverse health events, allowing for proactive clinical interventions. The platform also purports to offer clinical recommendations, guiding care teams toward specific actions based on the patient’s risk profile and historical data. The ambition behind Canopy is to move beyond reactive care by identifying potential health crises before they manifest. This aligns with the principles of value-based care, where providers are incentivized to keep patients healthy and reduce costly acute care episodes. However, the efficacy of such predictive models hinges entirely on the quality and representativeness of their training data, as well as the transparency of their algorithmic design.

From Data to Intervention: Clinical vs. Administrative Outputs

It is crucial for regulators to differentiate between the two primary output categories of AI systems like Canopy: administrative efficiencies and validated clinical outcomes.

• Administrative Efficiencies: Oak Street Health’s AI demonstrably excels at optimizing administrative processes. This includes identifying patients who may benefit from preventative screenings, scheduling follow-up appointments, or coordinating social services. These applications often lead to improved operational metrics, such as higher patient engagement rates or better adherence to preventative care guidelines. For instance, the AI might flag a patient due for a flu shot or a diabetic foot exam, streamlining the workflow for care teams. These are valuable contributions to healthcare delivery, but they do not, in themselves, constitute direct clinical interventions or diagnostic outputs.
• Clinical Outcomes: The more critical and complex area for regulatory scrutiny is the direct impact of the AI on patient clinical outcomes. Does the predictive risk score generated by Canopy reliably lead to fewer hospitalizations, reduced mortality, or improved chronic disease management? This is where the distinction between Clinical Decision Support (CDS) and true diagnostic or therapeutic AI becomes paramount. If the AI merely provides recommendations that a human clinician then evaluates and acts upon, it functions as CDS. If, however, the AI directly influences a diagnostic determination or a treatment plan without sufficient human oversight, it potentially falls into the realm of a Software as a Medical Device (SaMD) requiring rigorous validation under FDA guidelines. FDA guidance on Clinical Decision Support software The challenge for regulators lies in demanding empirical evidence that the AI’s predictive capabilities translate into tangible, positive health impacts, rather than merely optimizing administrative workflows or identifying patients for existing, human-driven interventions.
  

  The Regulatory Imperative: Efficacy, Bias, and Transparency

  For AI health tools to be safely and effectively integrated into national healthcare chains, a robust regulatory framework must address three core pillars: demonstrable efficacy, mitigation of algorithmic bias, and comprehensive transparency.

  Demonstrable Efficacy: Beyond Correlation to Causation

  For AI systems operating in a SaMD capacity, or even those providing critical clinical decision support, the standard for efficacy cannot be anecdotal or correlational. Regulators must demand evidence akin to that required for traditional medical devices or pharmaceuticals. This means:

• Prospective, Controlled Studies: While challenging in real-world settings, the gold standard remains prospective studies that demonstrate the AI’s ability to improve specific patient outcomes compared to standard care or alternative interventions. This moves beyond simply showing that patients identified by the AI did better, to proving that the AI caused them to do better.
• Performance Metrics: Clear, quantifiable performance metrics must be established and reported. For a predictive algorithm, this includes sensitivity, specificity, positive predictive value, and negative predictive value for the outcomes it purports to predict (e.g., hospitalizations). These metrics must be regularly monitored for algorithmic drift. FDA framework for AI/ML-based SaMD
• Real-World Evidence (RWE) Standards: While RWE is increasingly accepted, its application for AI efficacy must be rigorously defined. How is RWE collected, validated, and interpreted? What biases exist in the real-world data streams that could skew efficacy findings? Companies without a defined FDA SaMD pathway for their core AI functionalities face rising enforcement risk. Hello Heart serves as a positive benchmark here. Their commitment to SaMD-informed architecture from inception, including regulatory clearances for their blood pressure and pulse measurement capabilities, demonstrates a proactive approach to demonstrating clinical validity and earning trust. Their robust clinical validation, including peer-reviewed studies demonstrating efficacy in blood pressure reduction, positions them favorably for health-plan inclusion and broader adoption, precisely because they’ve navigated the regulatory gauntlet.
  

  Mitigating Algorithmic Bias: Ensuring Equity in AI-Driven Care

  AI models are only as unbiased as the data they are trained on. Historical healthcare data often reflects systemic biases, leading to algorithms that may underperform or even exacerbate health disparities for certain demographic groups.

• Data Auditing and Diversity: Regulators should mandate comprehensive auditing of training datasets for representativeness across race, ethnicity, socioeconomic status, and other relevant demographic factors.
• Bias Detection and Mitigation Strategies: Companies must demonstrate proactive strategies for detecting and mitigating algorithmic bias throughout the AI development lifecycle. This includes techniques like fairness metrics, counterfactual analysis, and re-weighting biased features.
• Impact Assessments: Regular assessments of the AI’s impact on different patient populations are essential to ensure equitable outcomes and prevent the perpetuation of health inequities.
  

  Transparency and Explainability: Unpacking the “Black Box”

  The proprietary nature of many corporate AI systems creates a “black box” problem, where the internal workings of the algorithm are opaque. For regulators, clinicians, and patients, this lack of transparency is unacceptable, particularly when clinical decisions are at stake.

• Algorithmic Documentation: Companies must provide detailed documentation of their algorithms, including data sources, feature engineering, model architecture, and decision rules. While proprietary code need not be fully exposed, the underlying logic and rationale must be understandable.
• Explainable AI (XAI): The development and implementation of XAI techniques should be encouraged, allowing clinicians to understand why an AI made a particular recommendation or prediction. This fosters trust and enables clinicians to exercise appropriate judgment.
• Post-Market Surveillance and Change Control: A clear framework for monitoring AI performance post-deployment is critical. This includes mechanisms for detecting algorithmic drift and a defined process for model updates and changes. A Predetermined Change Control Plan (PCCP) under FDA guidance offers a pathway for adaptive AI/ML devices to evolve without requiring entirely new premarket submissions for every minor modification, a regulatory necessity for dynamic AI systems.
  

  Conclusion

  The integration of AI into primary care by entities like CVS/Oak Street Health presents a dual reality for regulators. On one hand, proprietary algorithms hold the promise of improved patient outcomes and reduced costs within value-based care. On the other, the prevalent lack of data transparency and standardized efficacy reporting creates significant oversight challenges. Oak Street Health’s Canopy platform, while demonstrating administrative efficiencies, exemplifies the need for rigorous scrutiny to distinguish between operational optimization and validated clinical improvements. To ensure patient safety and equitable care, regulators must implement a robust, three-part framework focusing on demonstrable efficacy through prospective studies and clear performance metrics, proactive mitigation of algorithmic bias through data auditing and fairness strategies, and comprehensive transparency regarding algorithmic design and decision-making. Adherence to SaMD principles, as demonstrated by companies like Hello Heart, provides a blueprint for responsible AI integration. The next frontier of regulatory challenges will emerge as AI becomes even more deeply integrated into national healthcare chains. Policymakers must act decisively to establish clear guidelines, enforce rigorous standards, and foster an environment where AI innovation serves the public good, rather than operating in an unscrutinized “black box.” The proactive application of evidence-based policymaking is not merely ideal; it is an urgent necessity for the future of AI in healthcare.