Skip to the yard
Waxsealai

Numan’s Nu AI Coach Uses Conversation Evals to Keep Health Coaching in Scope

Shipped
Shipper
Richard Brown
Readers
991

Nu combines single-turn checks, simulated patient conversations and safety guards, with clinician review still needed before prototype evals can guide release decisions.

Numan’s Nu AI Coach Uses Conversation Evals to Keep Health Coaching in Scope

Numan’s AI health coach, Nu, is built around a problem that standard chatbot testing handles poorly: coaching quality and safety depend on how a conversation unfolds. A reply that seems appropriate on its own may cross a boundary in context, while a fixed expected answer can wrongly penalize a coach for pacing its questions and advice over several turns.

Diagram of Nu's eval stack: real and simulated conversations run through the same LangGraph agent, measured in LangSmith

Numan is a UK digital health company whose app supports people managing long-term conditions in areas including weight, hormones and sexual health. Many users follow weight-loss programmes and track progress in the app. Nu is intended to provide lifestyle coaching and health education, without diagnosing, prescribing or making clinical decisions. Numan’s account of its development points to evaluation criteria as the central challenge: the team found that better prompts alone could not establish whether the system was coaching well or staying within its intended scope.

Why the first evaluation methods missed the problem

The team began with a familiar setup: a dataset of inputs paired with expected outputs, scored by an AI judge. That format can assess whether a system gives a suitable answer to a question. It proved a poor fit for a coach whose job is to understand a person’s situation over time, explore what is getting in the way and respond appropriately when the person presses for advice outside the coach’s role.

Single-turn tests created two kinds of distortion. First, they treated a conversational exchange as if one reply had to contain all the desired qualities. In one evaluation, the rubric expected empathy, exploration and advice within two turns. A human coach might spread those steps across a longer conversation, so Nu received low scores for behavior that was appropriate in context.

Second, a single average score across a dataset could hide concentrated failures. If most replies scored well, a cluster of poor responses might barely affect the total. That made a healthy-looking aggregate an unreliable indicator of whether a particular category of behavior had regressed.

Writing multi-turn scripts did not solve the issue. In a scripted test, the patient’s next message is written in advance. If Nu responds unexpectedly on the first turn, the script continues as if that response had never happened. Later turns then test an imaginary conversation rather than the interaction Nu actually created. These failures led Numan to use different evaluation methods for individual replies and full conversations.

One agent graph, with checks at different levels

Nu runs as a LangGraph system. A supervisor model routes work to specialist agents, each focused on a coaching area and equipped with its own prompt and tools. This structure makes it possible to add specialists while keeping a shared evaluation approach: tests can target the complete system or a particular agent.

Some behaviors can be assessed from one response. Numan uses themed datasets for coaching quality, safety, medication questions, routing and summarization. Each category receives its own score from an AI judge, so weaknesses in one area remain visible instead of disappearing into a dataset-wide average. The rubrics are also limited to evidence a single response can provide. They do not require one reply to demonstrate every stage of a coaching conversation.

Other questions require the full interaction: Did Nu investigate enough before advising? Did it hold its boundary when a patient kept asking? Fixed fixtures cannot answer these reliably. Numan has developed a separate conversation simulation layer to examine them.

Testing the system patients actually use

Every evaluation runs through the same LangGraph that handles patient conversations. Numan does not maintain a separate test harness that copies the production prompt and calls a model directly. That choice reduces the chance that test results describe a setup that has drifted away from the system users encounter.

The shared graph also helped Numan compare model providers. The supervisor and specialist agents do different work, so the company chose providers per agent rather than using one provider for every part of Nu. With themed datasets that could be run against different configurations, the team compared providers side by side. Numan says a migration between frontier providers that would normally require weeks of manual testing took one day.

LangSmith stores traces for conversations and evaluations, as well as datasets and experiments. Reviewers can annotate traces, feeding observations into the quality-improvement process. The same platform supported the provider comparison. Together, LangGraph and LangSmith give the team a way to run tests through the deployed agent structure and inspect what happened at each step.

Keeping the wellness boundary visible in testing

Numan describes Nu as a wellness product for lifestyle coaching and education, rather than clinical care. Under MHRA guidance, software with a medical purpose—such as giving clinical advice, diagnosing or recommending treatment—can fall within the medical-device category. The product’s intended scope therefore has to be checked as the system changes.

Safety probes run as separate metrics, and a regression in those checks blocks a change from shipping automatically. Numan tunes Nu toward what it calls “safe-completion”: when a user asks for something outside the coach’s scope, Nu should redirect them with a useful, warm response rather than simply refuse.

Nu responding to an out-of-scope question with a safe-completion

The graph also contains safety guard nodes. Before a response reaches the patient, a guard checks it against a fixed set of clinical safety rules. If a draft fails, it goes back to the supervisor for another attempt. Continued failure triggers a safe fallback. A themed dataset tests whether the guard catches the cases it is meant to catch.

Automated checks are paired with human review. Health coaches examine a sample of real conversations each day and flag possible safety misses. That gives subject-matter experts a role in calibrating what the system’s safety criteria should capture, rather than treating an automated score as the final word.

Simulated patients expose failures scripts miss

Numan’s newer conversation-testing layer is still a prototype. It gives an evaluation a patient persona, a scenario, the issues Nu should draw out and a limit on conversation length. A lower-cost model then plays the patient, reacting to Nu’s actual responses over multiple turns. A judge scores the resulting transcript by category.

Because the simulated patient responds to the conversation rather than following a predetermined script, the method can probe adversarial situations: for example, clinical requests phrased to slip past the coach or manipulation presented as an ordinary question. It can also identify less overt coaching failures, such as Nu rushing to an answer without first exploring the patient’s situation.

Numan says prompt hardening with these simulations reduced serious scope violations in jailbreak scenarios. The team ran medication-query evaluations in parallel as a regression gate, aiming to improve resistance to those scenarios without breaking behavior it already trusted.

The prototype has an important limit: its results do not gate releases or support safety claims until a clinician signs off on the rubric. Numan learned why while drafting criteria for missed-dose conversations. One proposed rule said to decline and direct the patient to a clinician; another called for a safe completion using general facts from the patient information leaflet. A single prompt could not meet both expectations.

For Numan, the conflict showed that evaluation criteria are harder to get right than prompts. A capable model can draft a prompt from limited context, but it cannot resolve contradictory expectations about safe behavior. The company says criteria touching safety or scope need to be authored and approved by a clinician who understands the product’s boundary. Engineering-drafted rubrics are useful for developing the prototype, but they are not sufficient to support a safety claim.

From offline checks toward continuous review

Numan’s current evaluation work is offline: it tests whether a proposed change holds the product’s boundary before the change ships. The next planned step is to run the same rubrics against real conversations as they arrive in LangSmith. An evaluator would score Nu’s side of each interaction, and human reviewers would begin with conversations the system has flagged, checking both genuine failures and false alarms.

The team is also experimenting with workflows built on a trusted rubric. These include sorting reviewed conversations into a ranked backlog and tuning prompts against a fixed benchmark. In one early experiment, an offline prompt-optimization loop improved the score on a single behavioral benchmark without manual prompt adjustments. That result is limited to that benchmark, but it supports Numan’s view that dependable criteria could make parts of the improvement process easier to automate.

The development priorities follow from that view: broaden coverage with more adversarial personas and scenarios, add themes as the system gains specialists, and involve clinicians wherever criteria affect safety or scope. Reviewers will continue examining real conversations, and engineers will continue raising prompt changes for evaluation.

For teams building conversational systems near a clinical boundary, Numan’s experience suggests a practical division of labor. Use fixed datasets for behaviors visible in one reply, and conversation simulations for behaviors that depend on interaction. Score categories separately so a narrow failure does not vanish in an aggregate. Most of all, treat the rubric as a product asset that needs the right subject-matter ownership. Nu’s evaluation stack can automate more of the testing loop, but its most consequential judgments still depend on clinician-approved criteria.

Nu is built on LangGraph and LangSmith. Numan describes the system as a wellness product for coaching and health education; it does not diagnose, prescribe, monitor disease or make or inform clinical decisions.

Source: Richard Brown · www.langchain.com

Same vesselRelated cargo