Artificial Intelligence Is Now Prescribing Medicines in Utah
Utah has become the first state to evaluate autonomous artificial intelligence (AI) for prescription renewals for chronic conditions under a new pilot program.
On Tuesday, the Utah Department of Commerce’s Office of Artificial Intelligence Policy announced a first-of-its-kind partnership with Doctronic, an AI health platform.
The aim is to give patients with chronic conditions a faster, automated way to renew medications.
But the move raises serious questions about who is accountable when AI prescribes drugs, how patients are protected, and how much medical authority is being handed over to algorithms without public debate.
According to a state press release:
This agreement marks the first state-approved program in the country that allows an AI system to legally participate in medical decision-making for prescription renewals, an emerging model that could reshape access to care and ultimately improve care outcomes.
Medication noncompliance is one of the largest drivers of preventable health outcomes and avoidable medical spending. With prescription renewals accounting for roughly 80% of all medication activity, Utah and Doctronic aim to test how autonomous AI can help close gaps in access, reduce delays that lead to medication lapses, and improve outcomes for millions of people managing chronic conditions.
Under this partnership, Doctronic will become the first AI to legally prescribe routine refills by deploying its autonomous AI health platform, designed for fast, private, and personalized prescription renewals, within Utah’s regulatory sandbox framework. The Office will rigorously evaluate the platform’s clinical safety protocols, patient experience, and real-world effectiveness. The effort aims to demonstrate that safe, well-regulated AI can improve adherence, prevent avoidable hospital visits, and reduce healthcare spending, while keeping clinicians at the center of care.
Doctronic Background
Doctronic was launched in September 2023 by Matt Pavelle, a New York City-based serial entrepreneur and technology executive specializing in AI-driven healthcare solutions, according to Business Insider.
The platform is large language model (LLM)-agnostic, meaning it depends on not one but multiple AI systems—like OpenAI and Anthropic—and takes the consensus.
Pavelle raised $5 million in seed funding, led by Union Square Ventures, Tusk Ventures, and startup accelerator HF0.
Doctronic users must share personal information like their age and sex before inputting their symptoms.
Users get four likely explanations and a plan of action, including a standardized note to share with their provider, per Insider.
Patients can book a video visit with a licensed medical professional.
“They can request a visit 24/7 in all 50 states and be connected with a provider, usually within 30 minutes, starting at $39,” the report says.
In April of last year, Pavell confirmed Doctronic was seeing around 50,000 people a week.
In data shared with Utah regulators, Doctronic compared its AI system with human clinicians across 500 urgent care cases, according the Politico report.
The AI’s treatment plan was said to match the physicians’ 99.2% of the time.
Concerns
CEO and executive vice president at the American Medical Association Dr. John Whyte said that “[w]hile AI has limitless opportunity to transform medicine for the better, without physician input it also poses serious risks to patients and physicians alike.”
Concerns range from misuse or abuse to the possibility that those struggling with drug addiction could try to game the automated system to obtain drugs inappropriately.
And what about the subtle clinical red flags or drug interactions that a doctor would catch?
More fundamentally, Utah’s pilot raises the question of when, exactly, a non-human system was granted legal authority to participate in medical decision-making at all—a threshold that has historically been reserved for licensed professionals accountable to patients and regulators.
By authorizing an autonomous AI to legally prescribe routine refills, the state is not merely testing a new tool, but testing a new model of medicine, one in which judgment traditionally exercised by physicians is delegated to algorithms operating inside a regulatory sandbox.
The state frames the program as a way to improve efficiency and access, but speed and scale are precisely what magnify risk when errors occur, especially when decisions are automated and applied across millions of patients managing chronic conditions.
The promise that clinicians remain “at the center of care” sits uneasily alongside a system explicitly designed to operate autonomously, raising questions about where human oversight ends and algorithmic authority begins.
If prescription renewals—which account for roughly 80% of medication activity—are now deemed safe to automate, what other core medical functions might soon follow under the same logic?
And as Utah positions this pilot as a national model for AI in healthcare, it raises another question: is this a narrowly defined experiment, or the first step toward normalizing AI-driven medical decision-making without clear, settled guardrails?
source jonfleetwood.substack.com