Wednesday, January 8, 2025

How AI’s bias snowballs

The ideas and innovators shaping health care
Jan 08, 2025 View in browser
 
POLITICO Future Pulse Newsletter Header

By Daniel Payne and Carmen Paun

TECH MAZE

Post-doctoral researcher Tofunmi Omiye looks over chatbots in his office at the Stanford School of Medicine in Stanford, Calif., Tuesday, Oct. 17, 2023. A new study, co-led by Omiye, cautions that popular chatbots are perpetuating racist, debunked medical ideas, prompting concerns that the tools could worsen health disparities for Black patients. (AP Photo/Eric Risberg)

Research suggests that bias grows faster when using biased AI than when working work with biased humans. | AP

Artificial intelligence and humans can both be biased — but together, their biases can create a feedback loop, according to new research.

A feedback loop occurs when a system’s output is fed back into the system as input, which can then influence future output.

A human user’s biases can grow when using a biased AI system — more so than when humans work with humans who hold similar biases, the study, published in Nature Human Behaviour, found.

For example, people interacting with biased AI systems were more likely to underestimate women’s performance or overestimate men’s probability of holding a high-status job.

That can create a “potential snowball effect” of bias that further amplifies human behavior, one researcher said in a press release, partly because the authors said people seem to believe AI is more accurate than humans.

Why it matters: AI’s rapid rise across the health sector has raised concerns about its impacts on care. Bias is a key concern, and the new research suggests bias from systems could be more impactful than bias from clinicians.

Several AI systems in health settings have been shown to be biased in ways that could directly reduce some quality of care. Understanding how AI can change clinician behavior is key to making sense of algorithms that can be reliable in a research setting but unpredictable in the real world.

WELCOME TO FUTURE PULSE

Martha's Vineyard, Mass.

Martha's Vineyard, Mass. | Richard Schumaker

This is where we explore the ideas and innovators shaping health care.

Chlamydia is decimating the koala population on Australia’s east coast. Researchers there hope to save many of the animals with vaccines, the BBC reports.

Share any thoughts, news, tips and feedback with Carmen Paun at cpaun@politico.com, Daniel Payne at dpayne@politico.com, Ruth Reader at rreader@politico.com, or Erin Schumaker at eschumaker@politico.com.

Send tips securely through SecureDrop, Signal, Telegram or WhatsApp.

FORWARD THINKING

FILE - In this Thursday, Dec. 20, 2012 file photo, Dr. Terry Rabinowitz, right, talks with nurse Leslie Orelup at Helen Porter Nursing Home in Burlington, Vt. Doctors have used video feeds and other technology for years to treat patients in remote locations. But experts say growing smartphone use and customer demand are fueling a rapid expansion of telemedicine into everyday care the family doctor used to handle. (AP Photo/Toby Talbot, File)

Widespread demand for weight-loss drugs has accelerated telehealth investment. | AP

The convergence of widespread telehealth investment, growing GLP-1 demand and drugmakers, who increasingly look to reach patients directly could create a perfect storm for new care delivery in 2025.

At least that’s the case, according to Michelle Davey, co-founder and CEO of Wheel, a company that offers a slate of virtual care services.

“More and more consumers are coming online,” Davey said, though cautioning that “it’s still really early innings.”

It’s not just relatively new companies like hers looking to shake up the current models. Providers nationwide invest heavily in remote care, and pharmaceutical giants look to use that trend to take a more active role in connecting patients to their products.

“They’re really moving their commercialization to meet patients where they’re at,” she said, noting that widespread demand for new weight-loss drugs has accelerated that trend.

Washington’s role: Though major health players — both new and established — are also pushing for support of telehealth and other new care models, they say they need lawmakers’ backing.

Congress will need to renew pandemic-era telehealth rules again this March to continue the current use of the technology going forward, for example. And government payers must consider how to handle a potentially huge cost of GLP-1s, a new class of weight-loss drugs.

THE NEXT CURES

The prescription medicine OxyContin is displayed.

The FDA is considering approving a new pain drug that could be an alternative to opioids. | Darren McCollester/Getty Images

A new treatment for pain without using drugs that can cause addiction could be on the horizon.

The Food and Drug Administration will decide later this month whether to approve a novel drug for acute pain that could offer an alternative to opioids — if patients can afford it, Carmen reports with our Lauren Gardner.

How so: Suzetrigine, the candidate drug by Vertex Pharmaceuticals, would be the first selective sodium channel blocker approved for pain. The drug works differently than existing sodium channel blockers like lidocaine — a local anesthetic commonly used by dentists — by targeting specific pain signals in the body, enabling patients to maintain abilities like motor function. Because it doesn’t affect the central nervous system directly, the company said it expects the drug won’t pose an addiction risk for patients.

But it’s unclear how much the medication would cost consumers at the pharmacy counter and whether drug-pricing middlemen will favor it on their formularies over well-known and cheap opioids.

Why it matters: The market entry of a nonopioid pain reliever would be a landmark development for the category, which hasn’t seen a new product offering in decades. Meanwhile, the opioid crisis killed more than 81,000 Americans in 2023, according to the CDC.

Those deaths mostly involved illicit fentanyl, a synthetic opioid. But many people initially became addicted when they started taking prescription opioids to treat pain after surgery or for other conditions.

Remaining questions: Vertex is still studying suzetrigine’s efficacy for chronic pain, an area with a higher need. Mid-stage trial results released last month showed it didn’t perform better than a placebo in treating sciatica, a lower-back nerve pain that can linger for months or longer.

What’s next: The FDA is slated to make a final call on Vertex’s application by Jan. 30.

 

Follow us on Twitter

Carmen Paun @carmenpaun

Daniel Payne @_daniel_payne

Ruth Reader @RuthReader

Erin Schumaker @erinlschumaker

 

Follow us

Follow us on Facebook Follow us on Twitter Follow us on Instagram Listen on Apple Podcast
 

To change your alert settings, please log in at https://login.politico.com/?redirect=https%3A%2F%2Fwww.politico.com/settings

This email was sent to edwardlorilla1986.paxforex@blogger.com by: POLITICO, LLC 1000 Wilson Blvd. Arlington, VA, 22209, USA

Unsubscribe | Privacy Policy | Terms of Service

No comments:

Post a Comment

You’ve never heard anything like this before

This is bigger than my prediction of the 2008 financial crisis… ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏  ͏...