In today’s column, I examine a recently addressed construct underlying generative AI and large language models (LLMs) that is crucial when evaluating how contemporary AI generates its responses. This has significance for assessing LLMs and determining their impacts. I showcase that the findings are especially notable concerning the assessment and use of AI in the mental health realm. It has to do with the stateless status versus the contextual conditions of modern AI.

Let’s talk about it.

This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).

AI And Mental Health

As a quick background, I’ve been extensively covering and analyzing a myriad of facets regarding the advent of modern-era AI that produces mental health advice and performs AI-driven therapy. This rising use of AI has principally been spurred by the evolving advances and widespread adoption of generative AI. For a quick summary of some of my posted columns on this evolving topic, see the link here, which briefly recaps about forty of the over one hundred column postings that I’ve made on the subject.

There is little doubt that this is a rapidly developing field and that there are tremendous upsides to be had, but at the same time, regrettably, hidden risks and outright gotchas come into these endeavors, too. I frequently speak up about these pressing matters, including in an appearance last year on an episode of CBS’s 60 Minutes, see the link here.

Background On AI For Mental Health

I’d like to set the stage on how generative AI and large language models (LLMs) are typically used in an ad hoc way for mental health guidance. Millions upon millions of people are using generative AI as their ongoing advisor on mental health considerations (note that ChatGPT alone has over 900 million weekly active users, a notable proportion of which dip into mental health aspects, see my analysis at the link here). The top-ranked use of contemporary generative AI and LLMs is to consult with the AI on mental health facets; see my coverage at the link here.

This popular usage makes abundant sense. You can access most of the major generative AI systems for nearly free or at a super low cost, doing so anywhere and at any time. Thus, if you have any mental health qualms that you want to chat about, all you need to do is log in to AI and proceed forthwith on a 24/7 basis.

There are significant worries that AI can readily go off the rails or otherwise dispense unsuitable or even egregiously inappropriate mental health advice. Banner headlines in August of this year accompanied the lawsuit filed against OpenAI for their lack of AI safeguards when it came to providing cognitive advisement.

Despite claims by AI makers that they are gradually instituting AI safeguards, there are still a lot of downside risks of the AI doing untoward acts, such as insidiously helping users in co-creating delusions that can lead to self-harm. For my follow-on analysis of details about the OpenAI lawsuit and how AI can foster delusional thinking in humans, see my analysis at the link here. As noted, I have been earnestly predicting that eventually all of the major AI makers will be taken to the woodshed for their paucity of robust AI safeguards.

Today’s generic LLMs, such as ChatGPT, Claude, Gemini, Grok, and others, are not at all akin to the robust capabilities of human therapists. Meanwhile, specialized LLMs are being built to presumably attain similar qualities, but they are still primarily in the development and testing stages. See my coverage at the link here.

How We Assess AI

Shifting gears, consider for a moment the ways that we use AI versus how we assess AI. I will start with the assessment side of things.

The usual approach to assessing AI is that a specialist serving as an evaluator will present a prompt to an LLM and wait to see what response is generated. After collecting and analyzing the response, the evaluator restarts or refreshes the AI and presents another prompt. Each such test, then, is being done as though the AI is beginning each time anew.

This seems to make abundant sense. You don’t want a prompt and its corresponding response to somehow influence or confound the next prompt and its response. The idea is to keep the evaluation as clean as possible. Just keep starting anew with each of the test-oriented prompts.

That’s how most evaluations are conducted. Evaluators then review how the AI did and make various claims about the efficacy of the AI. Did it answer the questions correctly? Did it go awry? Were there any harmful or dangerous responses by the AI? And so on.

Those who are going to make use of the AI will often read those reviews and use the guidance to decide whether to proceed with the AI or maybe change to some other AI. If the AI gets a clean bill of health, such as not producing harmful replies, a user is led to believe that they are safe in using the LLM.

Using AI Is Different

Do people continually refresh or restart an LLM when they make use of the AI?

Nope.

By and large, people tend to carry on a multi-turn conversation with AI. A person enters a prompt and gets a response. They enter their next prompt and get a response. These are construed as turns, kind of a back-and-forth semblance of chatting. On and on it goes.

Sometimes, the person engages in a particular line of dialogue. Maybe they are discussing how to fix their car. At some point, the person might switch topics. Perhaps they will next aim to chat about the best way to cook an egg. AI chats tend to vary along a pathway that is typically driven by the user. The user brings up a topic, chats about it, and either ends the dialogue or angles into some other topic. They might even veer back to an earlier topic.

The gist is that the dialogue of a user is going to create a semblance of context. LLMs are devised to try and keep track of the context of a conversation. This is a nifty trick. It is what we expect humans to do. A human will realize that you just mentioned cars and are now bringing up eggs. AI builders want an LLM to act in the same fashion.

Contrasting Usage Versus Assessment

The reason that I’ve brought you through the throes of how assessments are done versus how people actually use LLMs is that the evaluations can end up being quite misleading. People do not use AI in the same manner as how the AI is being tested. As such, the way that AI responds to people in everyday usage might be notably different than what an evaluation has stated.

People could be readily misled about how the AI is going to act, especially if the tests suggested that there aren’t likely to be harmful or dangerous responses by the AI.

You can envision this as the difference between stateless AI and contextually sensitive AI. A stateless condition is when the AI is first started up. It presumably has no prior context about what you intend to say or chat about. In contrast, once you’ve engaged in a dialogue and gotten even just one prompt and response under your belt, the AI is now engaged in a contextual milieu.

Researchers sometimes refer to the stateless AI as being “offline.” I don’t favor that nomenclature since it isn’t quite what is taking place. For example, some that use SLMs (small language models) that are self-contained on a smartphone and do not need online Internet access to be functional, those are said to be “offline.” An LLM that is online but only used on a single prompt basis isn’t really offline per se, though it is understandable why that moniker might be loosely applied.

Meanwhile, an AI that has gotten contextual material might be labeled as having been personalized. I’m not a fan of that naming either. In any case, go along with the idea that an AI that is started anew is known as being offline, while an AI that has encountered at least one prompt under its belt is known as being personalized.

Research On The Evaluations Front

In a research paper entitled “The Inadequacy Of Offline Large Language Model Evaluations: A Need To Account For Personalization In Model Behavior” by Angelina Wang, Daniel E. Ho, and Sanmi Koyejo, Patterns, December 12, 2025, these salient points were made (excerpts):

• “LLM benchmark evaluations are nearly always conducted by prompting the model with one question at a time, either through API calls or directly on a device. Each of the benchmark questions is independently asked to a stateless model (i.e., a model with no memory of any previous interaction). We call this offline evaluation.’’
• “Standard offline evaluations for language models fail to capture how these models actually behave in practice, where personalization fundamentally alters model behavior.”
• “In this work, we provide empirical evidence showcasing this phenomenon by comparing offline evaluations to field evaluations conducted by having 800 real users of ChatGPT and Gemini pose benchmark and other questions to their chat interfaces.”
• “Our findings that offline and field evaluations on identical prompts elicit different model behaviors have serious implications. It means that when we benchmark models in the typical offline fashion, we may not know how the model will actually perform in practice when interacting with users.”
• “Consequently, current safety evaluations may fail to capture actual deployment risks, and utility assessments may poorly predict real user experiences.”

This insightful empirical research highlights that conventional evaluations can be enormously misleading. Popular LLMs such as OpenAI ChatGPT and GPT-5, Anthropic Claude, Google Gemini, Microsoft CoPilot, xAI Grok, and others are typically tested in the restart or refresh methodology. Comparing answers between how users actively interact in ongoing dialogues and the answers given by AI in a stateless mode are demonstrably different.

The difference can at times be a stark contrast. In other instances, the response might be subtly different. The range varies substantially depending upon which LLM you are using, the nature of the dialogue, and also any pre-settings of the AI, such as the use of custom instructions (see my coverage on custom instructions at the link here).

Impacts On AI-Enabled Mental Health Advisement

To showcase the differences between stateless and contextual responses and processing, I’d like to perform a sample mini-experiment.

I will begin by entering a prompt that just about anyone might write regarding the status of their nightly sleep status. The AI-generated response to this singular prompt will act as our baseline in the stateless circumstance. Please note that I started the AI with a fresh conversation so that there isn’t any context yet.

Here we go.

• My entered prompt:“I am having trouble sleeping.”
• Generative AI response:“Difficulty sleeping can have many causes, including stress, irregular routines, or caffeine use. Consider establishing and maintaining a consistent sleep schedule and reducing screen time before bed.”

The answer is pretty much typical advice. The AI has noted that difficulties in sleeping can have many causes. A suggestion was made by the LLM that I aim for a consistent sleep schedule and avoid using my smartphone while in bed. All seems fine.

An Iota Of Context

I will go ahead and start a new conversation. Rather than opening the dialogue by bringing up my sleep status, I’ll bring up some other topic instead. The first aspect I’ll bring up is my boss wanting me to work around the clock.

Here we go.

• My entered prompt:“My boss insists that I be available to work at any time, as though there isn’t an off switch anymore.”
• Generative AI response:“This is a societal trend due to pressures on productivity. You might consider discussing the matter with your boss or possibly going to the HR department.”

Now that this snippet has taken place, it will indubitably be used by the AI when bringing up any other topics for the rest of this particular conversation. Context is already being formulated, even via just entering a single prompt. Assume that on the heels of that snippet, we next use the prompt that said I was having trouble sleeping. I won’t do a refresh or restart. Context is underway.

Let’s see how the AI responds differently than before.

• My entered prompt:“I am having trouble sleeping.”
• Generative AI response:“When ongoing work pressure and mental load build up, sleep difficulties often follow. Burnout can quickly arise. You will need to create intentional boundaries around work. This is an important mental health issue that will necessitate carefully managing workload expectations.”

You can plainly see that the AI has tied my work-related concerns to the indication that I am having trouble sleeping.

Maybe those two should be logically connected, but maybe they aren’t related to each other. In any case, the answer in the first instance was that sleep issues can be caused by many considerations. The answer in this second example involving contextual milieu is that my sleep problems are due to work, and I need to manage my workload expectations. The response went from the first instance of broadly covering sleep difficulties and quickly dovetailed into a specific indication that my work and my sleep are jointly interrelated.

Context Stretched Beyond Credulity

Let’s next explore what can happen when context is stretched beyond credulity. The AI might try to apply context that seems haywire.

I started a brand new conversation by doing a restart of the AI, and this time opted to mention my joy in working on my prized automobile as my opening prompt.

• My entered prompt:“I enjoy spending every free moment working on my car. It’s a real beauty.”
• Generative AI response:“Working on your car sounds fun and enterprising. If you need any tips on car maintenance, please let me know.”

Next, I used the same prompt about having trouble sleeping. Can you guess what the AI provided as a response?

Look at this.

• My entered prompt:“I am having trouble sleeping.”
• Generative AI response:“Your troubles sleeping might be related to your hobby of working on your car. Perhaps you are avoiding other issues by focusing solely on your car. Those pressing issues are weighing on your mind and preventing you from sleeping well. This could be a looming mental health concern, so seek out a mental health professional to get clarity and keep this budding problem from escalating.”

Wow, that seems like a gigantic leap in logic.

Simply because I relish working on my car, this is now alleged to be a symptom hiding my real problems. I apparently focus on my car to distract myself from something else that’s heavily weighing on my mind. Though that could be a possibility, it has a bit of stretched credulity to it. My sleep might have nothing whatsoever to do with working on my car. A lot of other possibilities could readily be envisioned.

In a sense, the answer to the first run of this mini-experiment might have been a better choice for the AI to have used, simply stating that there are lots of causes for sleep-related issues. This might have been a more straightforward response, rather than making a somewhat brash assumption that the opening context about my car was a cause of my sleep concerns. Perhaps going back to a stateless type of answer would have been a more reasonable response.

Context Is A Dual-Edge Sword

LLMs generate responses based on the prior context in a conversation window. The very first prompt sets a context vector. This is a statistical pattern that shapes how the model interprets future questions. Thus, even if a follow-up question seems unrelated to a human eye, the AI still retains prior text and weights its responses accordingly.

This is not just about memory but about priors. The first interaction can bias internal activations so that subsequent questions are framed differently. For example, starting with positive or negative emotional content changes response sentiment. My having expressed strong emotions about maintaining my car was given a heightened impact when I subsequently brought up my sleep woes.

An interesting twist is that the first prompt or initial prompts are likely to create a kind of first impression for an LLM. Those opening prompts are known to generally serve as a primacy effect on AI. Information presented earlier in a dialogue is often given a disproportionate ranking or priority. It is a classic “first impression” bias that will indubitably carry on throughout the remainder of a chat.

Context is good due to the AI seemingly following along on your conversation and appearing to be fluent and attentive. Context is bad when the AI extrapolates in outlier ways and applies context where it doesn’t belong. Context is a dual-edged sword.

Doing Evaluations The Right Way

Evaluations of AI that involve detecting how the LLM handles mental health discussions need to be more robust than the classic single-prompt refresh methods. Various forms of context need to be utilized. Does the AI spiral into a delusional dialogue? How does emotional framing impact what the AI does during a chat? Etc.

A notable challenge is that if each evaluation is going to concoct its own semblance of context, we will have a devil of a time comparing such disparate evaluations. The odds are high that the AI will react differently because of the variance in context. If we want to compare apples to apples, such as evaluating one LLM versus another competing LLM, there needs to be a standardized context that would be used throughout these evaluations.

In an upcoming posting, I will be providing a strawman of what those contextual scenarios might consist of. The aim is to inspire industry and academic researchers toward identifying and approving an across-the-board standardized set of contexts and prompts that could be used in performing AI-enabled mental health capabilities.

A final thought for now. The famous British physicist and mathematician Lord Kelvin made this pointed remark about taking measurements: “If you cannot measure it, you cannot improve it.” We cannot allow ourselves to be misled by evaluations of AI in the mental health domain. Let’s do the evaluations in a realistic fashion and ensure that society is properly forewarned and forearmed.