Where docs beat AI: Showing their work

The ideas and innovators shaping health care

Jul 26, 2024 View in browser   By Erin Schumaker, Toni Odejimi, Ruth Reader and Daniel Payne TECH MAZE Doctors do a better job explaining their work than AI does, a new study found. | AFP via Getty Images Human doctors largely bested an AI model in a new study that involved a head-to-head patient diagnosis challenge. Where do docs still shine? Showing their work. How so? The National Institutes of Health and Weill Cornell Medicine researchers tasked doctors and an AI model with completing 207 challenge quizzes in which they looked at clinical images and a short description of a patient’s symptoms, then selected a diagnosis from multiple choice answers. The nine doctors in the study had different medical specialties, including dermatology, gastroenterology and infectious diseases. The doctors answered questions in their specialty open-book style, where they could use outside material, including online sources, and closed-book style, in which they had no help. Researchers asked the AI model to provide a written explanation for each diagnosis, including a description of the image, a summary of relevant medical context and step-by-step reasoning for the answer it chose. The study was published this week in the journal npj Digital Medicine. How’d it go?  — The AI model and doctors scored high when selecting patient diagnoses correctly. — The AI model beat doctors in the closed-book test, selecting more correct diagnoses. — Doctors won the open-book test, particularly on the most difficult questions. The bottom line: The AI model floundered, becoming confused and making mistakes when explaining how it arrived at a diagnosis. For example, it didn’t correctly identify that two lesions on a patient’s arm were linked to the same diagnosis when photos of the lesions were taken from different angles. Why it matters: While the study was small, the researchers say it’s important because it highlights risks for doctors relying on AI in a clinical setting. “Integration of AI into health care holds great promise as a tool to help medical professionals diagnose patients faster, allowing them to start treatment sooner,” Stephen Sherry, acting director of the NIH’s National Library of Medicine, said in a statement. But that promise comes with a key caveat. “AI is not advanced enough yet to replace human experience, which is crucial for accurate diagnosis,” Sherry said.   Live briefings, policy trackers, and procedural, industry, and people intelligence from POLITICO Pro Analysis gives you the insights you need to focus your policy strategy this election cycle. Secure your seat.      WELCOME TO FUTURE PULSE Chappaquiddick, Mass. | Erin Schumaker/POLITICO This is where we explore the ideas and innovators shaping health care. Don’t drink and drive a boat: Drinking alcohol is the leading cause of fatal boat accidents, accounting for almost 20 percent of boating deaths, according to the U.S. Coast Guard. Share any thoughts, news, tips and feedback with Carmen Paun at cpaun@politico.com, Daniel Payne at dpayne@politico.com, Ruth Reader at rreader@politico.com, Erin Schumaker at eschumaker@politico.com, or Toni Odejimi at aodejimi@politico.com. Send tips securely through SecureDrop, Signal, Telegram or WhatsApp.   WORLDVIEW The health tech sector in Europe needs some upgrades, a study found. | AFP via Getty Images Investors think the European health tech sector lacks leadership and management skills, a new report has found. How so? The study, from the European Investment Fund and the WorkInHealth Foundation, surveyed 472 venture capital fund managers across 371 EU-headquartered firms. According to the report, investors think Europe suffers from gaps in hard skills, such as science, technology, data analysis, engineering and mathematics. Other reported soft skills gaps include entrepreneurialism and strategic planning, our European colleagues report. What do investors care about? The top three factors health-focused venture capitalists consider when planning investments are the management team (61 percent of respondents), a product’s value proposition (33 percent) and market size (28 percent). Less important: Environmental, social and governance considerations (8 percent) and diversity and inclusion (3 percent).   SUBSCRIBE TO GLOBAL PLAYBOOK: Don’t miss out on POLITICO’s Global Playbook, our newsletter taking you inside pivotal discussions at the most influential gatherings in the world. Suzanne Lynch delivers the world's elite and influential moments directly to you. Stay in the global loop. SUBSCRI     PROBLEM SOLVERS Refugees are among those facing greater risks during pregnancy. | AFP via Getty Images More could be done to help vulnerable women during pregnancy, researchers said at an online forum sponsored by the National Academies of Science, Engineering and Medicine this week. How’s that? The focus was on strategies that could help ensure healthy births for disabled, incarcerated and refugee women. Dr. Carolyn Sufrin, director and founder of the group Advocacy and Research on Reproductive Wellness of Incarcerated People, cited an initiative with eight state prison systems across the East Coast to hire more doulas. Willi Horner-Johnson, a professor at Oregon Health & Science University, said the National Center for Disability and Pregnancy Research that she works with created a postpartum mental health toolkit that allows providers to identify postpartum depression symptoms in those with intellectual or developmental disabilities. Dr. Gunisha Kaur, director of the Human Rights Impact Lab at Weill Cornell Medicine, touted a program that gives refugee women access to providers. Her research used AI to predict who’d be at risk for high blood pressure, relatively common during pregnancy, and connect them with providers either in person or virtually. “This is a group that is so fearful of punitive immigration policies, of family separation or deportation or immigration detention that they avoid accessing almost all health care,” said Kaur.   Follow us on Twitter Carmen Paun @carmenpaun Daniel Payne @_daniel_payne Ruth Reader @RuthReader Erin Schumaker @erinlschumaker   Follow us

To change your alert settings, please log in at https://login.politico.com/?redirect=https%3A%2F%2Fwww.politico.com/settings This email was sent to edwardlorilla1986.paxforex@blogger.com by: POLITICO, LLC 1000 Wilson Blvd. Arlington, VA, 22209, USA Unsubscribe | Privacy Policy | Terms of Service