ChatGPT Passed a Major Medical Exam, but Just Barely

Anyone anxiously holding their breath for a competent robot doctor may need to wait a bit longer. A group of AnsibleHealth AI researchers recently put OpenAI’s ChatGPT to the test against a major medical licensing exam and the results are in. The AI chatbot technically passed, but by the skin of its teeth. When it comes to medical exams, even the most impressive new AI still performs at a D level. The researchers say that lackluster showing is nonetheless a landmark achievement for AI.

David Byrne on New Tech and AI | Gizmodo Interview

The researchers tested ChatGPT on the United States Medical Licensing exam (USMLE), a standardized series of three exams required for U.S. doctors vying for a medical license. ChatGPT managed to score between 52.4% and 75% across all three levels of the exam. That might not sound great to all of the overachievers out there, but it’s about on par with the 60% passing threshold for the exam. Researchers involved in the study claim this marks the first time AI was able to perform at or near the passing threshold for the notoriously difficult exam. Crucially, ChatGPT was able to pass without any extra specialized inputs from human trainers.

“Reaching the passing score for this notoriously difficult expert exam, and doing so without any human reinforcement, marks a notable milestone in clinical AI maturation,” the authors wrote in the journal PLOS Digital Health.

Mediocre test scores aside, the researchers praised ChatGPT for its ability to craft authentic sounding, original answers. ChatGPT managed to create, “new, non-obvious, and clinically valid insights,” for 88.9% of its responses and appeared to show evidence of deductive reasoning, chain of thought, and long term dependency skills. Those findings appear somewhat unique to ChatGPT and its particular style of AI learning. Unlike previous generations of systems that use deep learning models, ChatGPT relies on a large language model trained to predict a sequence of words based on the context of the words that came before. That means, unlike other AIs, ChatGPT can actually generate sequences of words that weren’t previously seen by the algorithm and that could make some coherent sense.

The tricky USMLE exams test participants on basic science, clinical reasoning, medical management, and bioethics. They’re most often taken by medical students and physicians in training. These exams are also standardized and regulated, which makes them particularly well suited to test out ChatGPT’s capabilities, the researchers said. One thing the exams definitely aren’t is easy. Human students typically spend around 300-400 hours stressfully pouring over dense scientific literature and testing material in preparation just for the Step 1 exam, the first of the three.

Surprisingly, ChatGPT managed to outperform PubMedGPT, another large language model AI trained exclusively on biomedical literature. That may seem counterintuitive at first, but the researchers say ChatGPT’s more generalized training may actually deliver it a leg up because it’s potentially exposed to a broader range of clinical content like patient-facing disease primers or drug package inserts. The researchers optimistically believe ChatGPT’s passable grade could hint towards a future where AI systems can play an assisting role in medical education. That’s already happening on a small level, they write, citing a latest example of AnsibleHealth clinicians using the tool to rewrite dense, jargon filled reports.

“Our study suggests that large language models such as ChatGPT may potentially assist human learners in a medical education setting, as a prelude to future integration into clinical decision-making,” the researchers said.

In a rather meta twist, ChatGPT wasn’t just tasked with taking the medical exam. The system was also involved with drafting the eventual research paper documenting its performance. Researchers say they interacted with ChatGPT, “much like a colleague” and leaned on it to synthesize and simplify their draft and even provide counterpoints.

“All of the co-authors valued ChatGPT’s input,” Tiffany Kung, one of the researchers wrote.

ChatGPT has added an impressive amount of passing grades to its educational trophy wall in latest months. Last month, ChatGPT managed to score between a B and B minus on a MBA-level exam given to business students at the prestigious Wharton School of the University of Pennsylvania. Right around the same time, the AI achieved a passing score on a law exam given to students at the Minnesota University Law School. In the law exam case, ChatGPT skirted by with a C+.

“Alone, ChatGPT would be pretty mediocre law student,” lead study author Jonathan Choi said in an interview with Reuters. “The bigger potential for the profession here is that a lawyer could use ChatGPT to produce a rough first draft and just make their practice that much more effective.”

ChatGPT might be able to eke out passable scores in exams focused on writing and memorizing comprehension, but mathematics is another beast entirely. Despite its impressive ability to bust out academic papers and semi-conceiving prose, researchers say the AI only performs at roughly a 6th grade level when it comes to math. ChatGPT fares even worse when it’s asked basic arithmetic problems in natural language format. That stumbling stems from its predictive large language model training. ChatGPT will, of course, confidently provide you an answer to your math problem, but it could be completely divorced from reality.

ChatGPT’s at time wacko answers are what senior Google engineers and other in the field have referred to, cautiously, as AI “hallucinations.” These AI hallucinations create answers that seem convincing but are partially or completely made up, which isn’t exactly a great sign for anyone looking to authoritative AI’s in high-stakes fields like medicine and law.

“It [ChatGPT] acts like an expert, and sometimes it can provide a convincing impersonation of one,” University of Texas professor Paul von Hippel said in a latest interview with The Wall Street Journal. “But often it is a kind of b.s. artist, mixing truth, error and fabrication in a way that can sound convincing unless you have some expertise yourself.”

What Is No-Exam Life Insurance?

There’s a lot to consider when you’re buying life insurance, not to mention the hassle of getting a medical exam, which most policies require. If you’re looking for a simpler, quicker alternative, no-exam life insurance may be a good fit.

A traditional life insurance policy typically involves a battery of tests and may take more than a month to come into place. If you’re one of the 6 in 10 Americans that have a

By contrast, no-exam policies can be done online usually in just a few days. “Medical exams can be off putting,” says Mark Scafaro, co-founder of Afficiency, an insurance technology company. “No-exam life insurance is a great alternative for customers who aren’t interested in completing a medical exam—or waiting weeks for an underwriting decision.”

To be sure, no-exam policies have their drawbacks. While it’s generally easier to get approved, there is no ensure you will be. Coverage limits also tend to be far lower. And because the insurance company has less insight into your health, you will almost certainly pay more. 

Still it’s no secret these have become more popular. Here is what you need to know.

No-exam life insurance is usually a term policy, which means you typically pay a premium to be covered for a specific term such as 10, 20 or 30 years. Afterward the insurance ends. But there are whole life policies, which stay in place as long as you maintain premiums, available as well.

Here are several common types:

You can be approved for this type of life insurance policy with minimal medical questions and no exams. Coverage starts as soon as your application is submitted and you’ve selected a premium.  

As its name suggests, guaranteed issue life insurance is a type of whole life policy that guarantees you won’t get denied when you apply. You’ll skip all the health-related questions as well as a medical exam. However, these policies tend to have a long waiting period of around two to three years and a lower death benefit ranging from $2,000 to $25,000. 

Accelerated underwriting relates to the process underwriters use to approve your application for insurance quicker. This is commonly used with no-exam life insurance and involves using third-party data and algorithms to determine a person’s approval rate and premium. Accelerated underwriting policies tend to have a coverage limit of $1 million and for anything higher, you’d need a medical exam. 

No-exam life insurance may be a good choice if you are:

Traditional life insurance typically requires an in-person medical exam that involves drawing blood and collecting other samples, along with a review of your body-mass index, medical conditions and whether you smoke or drink alcohol. “It can be a rigorous process,” says Susana Zinn, an independent life insurance agent based in Miami. “Even an inch or two in your height could impact your BMI and premium rate during the medical exam,” she adds.

With no-exam life insurance you’ll still need to answer a few basic questions and disclose any medical conditions you have. No-exam life insurance doesn’t ensure coverage, but it’s more forgiving. “Having some major health issues or risk factors in your history such as nicotine use, high blood pressure or high cholesterol, can make you a strong candidate for no-exam,” says Zinn.

No-exam life insurance can also make sense if you are young and healthy–and simply looking for a hassle-free way to get insured quickly. While you may pay slightly more, your premiums are likely to be low regardless of what kind of insurance you pick, so the convenience may be worth the trade off. Keep in mind no-exam life insurance policies tend to have smaller coverage limits than traditional policies, so the option works best if you are looking for a policy that pays less than $500,000.

With a term insurance policy—which is far more common for no-exam life insurance, your premium is often lower and fixed for the duration of the term length. With a permanent or whole life insurance policy, your premiums will be higher but stay the same so long as you keep the policy. 

The cost of life insurance depends on several factors including your:

Because life insurance pricing is based on so many individual factors, it’s difficult to find data about average prices. However, here are some representative quotes provided by Quotacy through Bestow Life Insurance.

Quotacy through Bestow Life Insurance

There are a few ways to shop for individual no-exam life insurance:

Online is a great place to start, and you can quickly find quotes from well-known national brands, as well as startups like Bestow, Haven Life and Fabric, which are usually backed by major insurance companies. If you want guidance you can find Buy Side from WSJ’s picks for Best Life Insurance companies here.

Most of what you find online will be term life insurance options, but some major companies like Nationwide or Progressive will offer whole life insurance options as well. To get a quote, you’ll have to submit a form that includes some basic information about yourself including your name, ZIP Code, height and weight, and gender.

Experts recommend gathering at least three quotes to determine how much no-exam life insurance would cost based on your demographic. “No two life insurance companies are alike, meaning one company may rate someone a smoker if they use smokeless tobacco while another won’t,” says Brian Carden, a Brentwood, Tenn., insurance and financial advisor.  

After receiving a quote, a representative from the insurer’s site may call you or you’ll be directed to continue completing an application form online. 

If you work with a financial advisor, your advisor should be able to consider all aspects of your financial life to pick a policy that works for you. But remember some financial advisors earn commissions for selling insurance so they will have an incentive to recommend certain insurers and policies. 

Working with a fee-only and fiduciary advisor means they won’t earn a commission and are more inclined to prioritize acting in your best interest. It’s always wise to ask a financial advisor if they have specific partners or earn commissions for making certain recommendations early on. 

A life insurance agent sells policies for a living, so they can be some of the most knowledgeable professionals out there. While insurance agents traditionally work in person, there are also online brokers like Sproutt, if you prefer a digital experience.

Like many financial advisors, insurance agents typically work on commission which means they might have an incentive to recommend products that pay the highest commissions. They earn a commission that can range from 40% to 100% of the first year’s annual premium, along with a 2% to 5% commission each additional year when the policy renews. 

In addition, some life insurance agents work with only one insurer or a handful of insurers, meaning they may only present you with a narrow slate of options. Other agents are “independent” meaning they have a broader reach and can help you get a policy with any life insurance company. 

You may already have free no-exam life insurance through your employer. Check with your human resource department to see if life insurance is an employee benefit and what your coverage is. 

Usually, your policy amount is equivalent to one year’s salary and your policy is only active so long as you remain an employee of the company. So keep in mind, if you need a higher coverage amount or don’t plan to stay at your job forever, this is when you should consider a policy from a private company. 

Once you pick a company and policy that you’re happy with, it’s time to move forward and sign up for life insurance. All in all, signing up for a no-exam life insurance policy should take one to three days at most. Here is how the process works in four steps.

The advice, recommendations or rankings expressed in this article are those of the Buy Side from WSJ editorial team, and have not been reviewed or endorsed by their commercial partners.

Best No-Exam Life Insurance Of August 2023

A no-exam life insurance policy may not be able to provide the full coverage amount you need, especially if you’re looking to cover many working years or the years of raising a family.

Before you start getting life insurance quotes, calculate how much life insurance you need. A no-exam policy alone may not be able to provide sufficient coverage.

Getting life insurance without a long application process is appealing, but don’t jump into a no-exam life insurance application without understanding your chances of getting approved. Many no-exam policies require very good or excellent health for approval. A denial goes on your insurance record and could hinder future applications.

Don’t be surprised if you end up doing a more traditional application process, including a life insurance medical exam if you’ve had some health issues.

Of course you don’t want to overpay for life insurance, but research more than cost. There are coverage options that can be very valuable long after you buy the policy, such as the option to convert term life to permanent life insurance. Also, look at whether there’s an accelerated death benefit, which gives you access to money from your own death benefit if you become terminally ill.

No matter what type of life insurance you’re applying for, be thorough and truthful on the application.

“Pay attention to each question carefully and be ready to respond with information around specific medical conditions you have. Giving more detailed information will help streamline the process as well as get a more accurate underwriting decision,” says Tavan of Legal & General America.

Intentional misrepresentations can lead to application denials or, worse, denial of a claim after you pass away. Life insurance companies have many ways to verify application information. Technology on the backend allows them to verify data from additional sources.

If the company rejects you for no-exam life insurance because of your health, don’t deliver up your life insurance search. If you need life insurance, you likely have other routes.

Looking for traditional term life insurance? See Forbes Advisor’s ratings of the best term life insurance.