aMedtech development is in crisis, with developers from some of the world’s largest and most successful companies leaving high-paying jobs to start medtech companies. Healthcare providers, regulators, and industry leaders are (rightfully) seeking regulatory frameworks that ensure AI applications are trustworthy and patient-centric.
Not every regulatory framework can make AI truly revolutionary. At a recent hearing before the House Energy, Commerce and Health Subcommittee, congressional leaders expressed concern about a third-party review process for AI proposed by the Coalition for Health AI (CHAI), a nonprofit led by representatives from incumbent companies such as Google, Microsoft and the Mayo Clinic. The review process would give large tech companies developing medical AI models control over market entry.
As Reps. Mariannette Miller-Meeks (R-Iowa) and Brett Guthrie (R-Ky.) pointed out at the hearing, there is significant potential for regulatory capture inherent in this approach.
Big tech companies trying to regulate the medical AI industry on behalf of the federal government could create an unfair playing field for startups and small companies. Startups will innovate quickly, while incumbents may struggle to keep up. Greater scrutiny of startups’ intellectual property by competing incumbents would do nothing to improve safety and would slow down real innovation that could change the lives of patients and clinicians for the better.
An alternative framework is localized quality assurance. One example of this framework was developed by the Health AI Partnership, a collaboration of individuals from decentralized academic medical centers that aims to provide health systems and other organizations looking to incorporate health AI into their operations with the tools and technical expertise to help them make their own decisions about which AI models are best suited for their patients.
Rather than handing over control of sensitive data review to large technology companies looking to compete in the AI ​​space, localized quality assurance would allow participating organizations to validate healthcare AI applications on their own datasets to determine which technologies work best for the people they serve. The framework would provide resources for all providers to operate their own review processes, rather than consolidating these reviews into a few large technology companies and academic medical centers.
Democratized access and localized assurance of AI is on the rise. In addition to the Health AI Partnership tools, electronic health record company Epic recently released an open-source tool on Github for providers to validate their AI models. In a world of multiple voluntary assurance tools, the most innovative companies can compete on the merits of their products, and clinicians can evaluate these tools based on their effectiveness and practicality in real-world clinical settings.
Of course, full democratization of AI review requires more support for practices, large and small. Not all practices have the capacity or expertise to deploy AI review tools, making it more difficult to evaluate AI models for their specific patient populations and clinical needs. Without additional support, full democratization of AI access may be delayed.
Open, competitive markets are essential for technological development. The winner of AI in healthcare is yet to be determined, and large companies are trying to prove themselves. In the ongoing battle between big and small tech companies, stabilizing the market with regulations that favor incumbents and drive the best talent, ideas, and funding away from healthcare is not the right way to proceed.
AI has the potential to improve healthcare for millions of Americans, ease the burden on clinicians, and alleviate healthcare disparities. But innovators need to test and validate novel ideas quickly. Big companies have a strong incentive to seize the market under the guise of safety. It’s critical to prevent this from happening and allow health systems and providers to determine the right solutions for their unique practices and unique patient populations, encouraging innovation and investment in the process.
Julie Yeo is a general partner in the Bio + Health team at venture capital firm Andreessen Horowitz.