ChatGPT answers more than half of software engineering questions incorrectly

ChatGPT's ability to provide conversational answers to any question at any time makes the chatbot a handy resource for your information needs. Despite the convenience, a new study finds that you may not want to use ChatGPT for software engineering prompts.  

Before the rise of AI chatbots, Stack Overflow was the go-to resource for programmers who needed advice for their projects, with a question-and-answer model similar to ChatGPT's. 

Also: How to block OpenAI's new AI-training web crawler from ingesting your data

However, with Stack Overflow, you have to wait for someone to answer your question while with ChatGPT, you don't. 

As a result, many software engineers and programmers have turned to ChatGPT with their questions. Since there was no data showing just how efficacious ChatGPT is in answering those types of prompts, a new Purdue University study investigated the dilemma. 

To find out just how efficient ChatGPT is in answering software engineering prompts, the researchers gave ChatGPT 517 Stack Overflow questions and examined the accuracy and quality of those answers. 

Also: How to use ChatGPT to write code

The results showed that out of the 512 questions, 259 (52%) of ChatGPT's answers were incorrect and only 248 (48%) were correct. Moreover, a whopping 77% of the answers were verbose. 

Despite the significant inaccuracy of the answers, the results did show that the answers were comprehensive 65% of the time and addressed all aspects of the question. 

To further analyze the quality of ChatGPT responses, the researchers asked 12 participants with different levels of programming expertise to deliver their insights on the answers. 

Also: Stack Overflow uses AI to deliver programmers new access to community knowledge

Although the participants preferred Stack Overflow's responses over ChatGPT's across various categories, as seen by the graph, the participants failed to correctly identify incorrect ChatGPT-generated answers 39.34% of the time.  

According to the study, the well-articulated responses ChatGPT outputs caused the users to overlook incorrect information in the answers. 

"Users overlook incorrect information in ChatGPT answers (39.34% of the time) due to the comprehensive, well-articulated, and humanoid insights in ChatGPT answers," the authors wrote. 

Also: How ChatGPT can rewrite and Excellerate your existing code

The generation of plausible-sounding answers that are incorrect is a significant issue across all chatbots because it enables the spread of misinformation. In addition to that risk, the low accuracy scores should be enough to make you reconsider using ChatGPT for these types of prompts. 

Best Online Couples Counseling of 2023

Online therapy provides a private, convenient and often affordable way to access mental health help without requiring you to visit a counseling center or therapist’s office in person. Instead, you can connect with your therapist via video call, phone call and/or text message conversation, depending on your needs and preferences.

Online therapy is a safe space in which you can address Topics like depression, anxiety, stress, anger management, insomnia, panic attacks, eating disorders, trauma, relationship issues, life transitions, bereavement and more.

Different online therapy platforms support different methods of therapy delivery, so first consider the way(s) in which you would like to receive help. The best online therapy options in their evaluation were broad telehealth sites that include therapists: Amwell, Doctor on Demand and MDLive. These services mirror a traditional therapy appointment, focusing on live video sessions with a therapist.

Some online therapy platforms allow you to purchase one video therapy session at a time while others require a monthly subscription, which usually includes a single video therapy session a month and access to unlimited text messaging with your therapist. If ongoing communication with your therapist sounds beneficial to you, consider a platform with this subscription option. Just note that “unlimited messaging” means that you can text message your therapist as much as you like, but your therapist might respond only once or twice a day on weekdays.

Some platforms, such as Online-Therapy.com, focus heavily on the skill-building components of therapy, offering tools like worksheets that your therapist reviews, journals, activity plans to help you schedule and keep track of certain behaviors, and even yoga videos.

Many online therapy services have their own mobile apps as well.

“Each [communication] format plays a significant role in the total therapy process,” says Lisa Henderson, a licensed professional counselor expert at the American Counseling Association and co-founder of Synchronous Health in Nashville, Tennessee. “If you’re doing anything that’s what I would consider deep work—trying to resolve trauma, getting into the roots of addictions or eating disorders, anything where you’re doing a lot of processing—I prefer video. You need to be able to see and read body language.”

Meanwhile, Henderson says texting is fantastic for check-ins around skill building and using those new skills. “Texting is much better for the coaching side when I want to deviate away from the processing side.” Texting or check-ins between live sessions are often used in dialectical behavior therapy to reinforce the client’s practicing of evidence-based skills when they encounter difficult situations.

As far as phone calls go, Henderson suggests this format is best for navigating gray areas. “It’s harder to coach without getting into processing on the phone, but if it’s complicated and you need to work through why something didn’t work, then the phone would certainly be better than texting,” she adds.

The best online therapy platforms connect you with licensed providers, which can include psychiatrists, psychologists, licensed marriage and family therapists, licensed clinical social workers and licensed professional counselors.

It can be challenging to compare online therapy platforms due to their wide range of plans and prices. Based on their research, here are several ways to identify the best online therapy for you:

20% off your first month

BetterHelp – Affordable therapy anytime, anywhere

Many people are likely to need some mental health support due to the effects of the COVID-19 pandemic. The number of adults with depression symptoms more than tripled between March 2020 and September 2020—from 8.5% to 27.8%—according to a study published in JAMA.

Certain disorders and issues may be better suited for virtual therapy than others. It’s usually appropriate and effective to address anxiety disorders, body image issues and guilt issues with online therapy. People seeking personal growth and the children of alcoholics can benefit from the online therapy format as well, according to a study in the Journal of Clinical Psychology.

Situations not appropriate for online therapy, according to the study, include:

Mary Alvord, a psychologist in Maryland who teaches mental health professionals about telehealth, Jay Shore, Ph.D., a psychiatrist and director of telemedicine at the Helen and Arthur E. Johnson Depression Center at the University of Colorado Anschutz Medical Campus, and Lisa Henderson, a licensed professional counselor expert at the American Counseling Association, offer these pros and cons for online therapy. Other sources of this advice include Psychology Today and the American Psychological Association.

Circumvents mental health stigma. “For people whom stigma is a concern, especially if they live in a tight-knit community, parking their car outside a counseling center or therapy office can really violate their privacy,” Henderson says. “But online therapy is really discrete and can protect people’s privacy and confidentiality in ways that in-person [therapy] simply cannot.”

Convenience and safety. If you’re unable to travel safely during bad weather or can’t take time out of your workday to travel to and from a mental health professional’s office, a virtual visit can be a good substitute.

Sense of intimacy. Shore says some patients may prefer their familiar at-home surroundings versus an “artificial clinic environment.” Henderson echoes these sentiments. “In some ways, video is more intimate than being in the same room because we’re in each other’s space,” she says. “You might be in my office, but it’s in my home, so it feels like you’re in my home just as I am in your home. That really bridges a gap, as opposed to being on my turf when you come into my office.”

Similar outcomes. In-person and video visits hold the potential to deliver similar results, according to Shore. Henderson agrees: “We see just as much, if not more, improvement in online therapy settings. Apples to apples, in-person therapy versus telehealth, there’s really no difference between which one is more effective.”

Easier access. For people who live far from the nearest therapist office or counseling center, online therapy can provide a readily available alternative.

Little to no wait time. A virtual appointment may be able to begin on time while an in-office appointment may be delayed by paperwork and other bureaucratic hurdles.

Nonverbal communication. A therapist may not pick up on a patient’s nonverbal cues during a virtual appointment. Alvord explains that much of their communication is nonverbal. However, Henderson points out that the proximity of the camera lens during video appointments can provide more visual communication through facial expressions than an in-person appointment where a greater physical distance exists between the therapist and the client.

Limited effectiveness for some. Certain patients, such as some children or people with autism spectrum disorder, may not respond well to virtual therapy, Alvord notes. Patients with dementia or other cognitive issues also may not do well in virtual sessions without modifications, such as a caregiver being with the patient, Shore says..

Technology. Some patients’ homes may not be equipped with high-speed internet service, or the patient may not be comfortable with technology, making virtual therapy difficult or even impossible to carry out.

Insurance coverage. In some cases, your health insurance provider may cover an in-person therapy session but may not cover a virtual session. Such policies are constantly changing, though, especially in light of the COVID-19 pandemic. Speak to an agent at your insurance company to confirm what your coverage currently includes.

Alvord suggests asking these questions when selecting an online therapist:

Regulation of online therapy sites and mobile apps—an area of mental health known as telebehaviorial health or telemental health—is a bit of a hodgepodge.

Both the Food and Drug Administration (FDA) and the Federal Trade Commission (FTC) regulate some, but not all, medical apps. The safety of medical apps “is an emerging public health issue,” say researchers in a 2020 study in the Journal of the American Medical Informatics Association. They called for establishment of “vigilant regulatory frameworks” to govern these apps.

As the FDA “continues to develop a framework for oversight, industry professionals have noted that the agency has taken a hands-off approach for mental health apps in particular,” say researchers in The Regulatory Review, a publication by University of Pennsylvania’s Program on Regulation.

At the state level, regulation typically focuses on doctors who deliver telehealth services, but state agencies have stepped up their regulation of psychologists, counselors and other mental health professionals who use telehealth. For its part, the American Counseling Association emphasizes that counselors who offer telebehavioral services must adhere to state licensing requirements. Many online therapy sites stress that all of their therapists are licensed.

On top of state regulations, compliance with professional ethical standards and HIPAA may come into play with online therapy.

Online therapy also raises questions about regulation when a therapist is in one state and the patient is in another. Most of the time, therapists can only practice in the state they’re licensed, which means their client must be in the same state even if the sessions are virtual–an exception would be if they obtained some type of temporary or provisionary permit with the state board where the client resides. This is important because therapists must abide by rules and regulations overseen by their specific licensing boards, and interstate cases can cause issues in investigating problems involving a therapist’s professional conduct.

If you need to lodge a complaint about an online therapy platform, first alert the site to your concerns. Next, you can reach out to the FDA and FTC. To file a complaint about a specific therapist, contact the agency in your state that’s in charge of licensing the therapist’s profession. Licensing rules vary for psychiatrists, psychologists, therapists and counselors.

Before you commit to online therapy, ask yourself:

Some sites market their services as therapy, but those claims may be false or misleading, says the American Psychological Association (APA), because some therapists may not be professionally licensed.

According to the APA, “therapist” and “psychotherapist” are not “legally protected” words in some states, meaning someone who promotes themselves as a therapist may not be licensed. Numerous online therapy providers promote the fact their therapists are licensed.

Online therapy may not be right for everyone in every situation. For example, online therapy may not be ideal for people with several mental disorders or who pose a threat to themselves or others, according to a Frontiers in Psychiatry study.

However, research suggests professionals and patients view telehealth favorably and that teletherapy can be effective. “I would say—and have heard some of my clients say, too—that in some ways, online therapy is even a little bit more comfortable than being in the same room,” Henderson says. “And that has implications on people feeling relaxed and opening up.”

Private and government-sponsored health insurance don’t cover telehealth services, including online therapy, evenly. The U.S. Department of Health and Human Services says most private health insurance covers at least some telehealth offerings. Meanwhile, Medicare and Medicaid cover telehealth in certain cases.

If an online therapy provider accepts insurance, they can bill your insurer directly. You’ll still be responsible for your copays and deductibles. If the site doesn’t accept insurance, you may be able to submit your bills to the insurer for reimbursement, depending on your plan. You may also need a diagnosis in order to use insurance.

Meanwhile, you may be able to use your health savings account (HSA) or flexible savings account (FSA) to pay for online therapy.

Alvord and Shore provide these tips for Preparing for an online therapy session:

Not all “online therapists” are legit. To be safe, make sure your therapist has licensing in your state, such as a licensed marriage and family therapist or licensed clinical social worker. The best online therapy services will also have psychiatrists and/or psychologists available.

Not all therapists can prescribe medication, regardless of whether you meet with them online or in person. A couple of online therapy services they evaluated have plans that specifically include medication in the pricing: Cerebral and Brightside. Other online therapy services, such as Amwell, can connect you with therapists who can prescribe.

Telehealth and teletherapy practices must adhere to the same codes of ethics, regulations (federal and state) and professional standards as in-person treatment. These responsibilities include maintaining confidentiality and storing electronic patient records securely.

20% off your first month

BetterHelp – Affordable therapy anytime, anywhere

Filing Bankruptcy Without a Lawyer | Step-By-Step Guide