If an AI Vendor Can't Answer These Questions, Keep Looking

If you threw a pebble out of our San Francisco office window, you'd hit 20 AI companies. Walk any healthcare conference floor and it feels the same. Every booth promises to fix your phones, faxes, referrals, prior auths, and claims and denials. Most of them sound identical.

For most independent provider groups, the question is no longer whether to use AI. It's where to start, and how to tell the partners who will deliver from the ones who won't.

Software made nearly every other business easier over the last 30 years, while healthcare got harder. That's because provider office work is exactly the kind software can't handle. It arrives as phone calls, faxes, and handwritten forms. Every task changes with the payer, the plan, the state, and the provider's preferences. And our systems rarely talk to each other.

AI is good at all three. It can read a faxed referral, pull out the patient's details, and attach it to the right chart. It can follow your scheduling rules on every call, every time. That's why healthcare is adopting it about twice as fast as the rest of the economy [Menlo Ventures]. But a fast-moving market is also a crowded one.

After 20 years on the provider side as an operator, founder, and investor, I've landed on four rules for choosing well:

1. Dream big, start small.

2. Don't leave AI alone.

3. Find a true AI partner, not a vendor.

4. Demand day-one ROI.

At the end, you'll find the questions to bring to every vendor call.

Rule 1: Dream big, start small

Start by picturing what your front office could look like: every referral processed the day it arrives, every call answered the first time, every fax in the right chart. Then pick one problem you can draw a box around and measure.

Where you start depends on what matters most right now. If your priority is to grow patient volume, start with referrals. If it's to free up staff time, start with your phones. To improve collections, start with denial management.

You want a first project big enough to matter and small enough to prove. Upperline Health, one of the nation's largest podiatry groups, started with us on referrals in a single Florida clinic. Staff were spending 10 to 20 minutes doing data entry on each referral, with a regular backlog of over 500 referrals. We automated the process, cleared the queue, and in doing so increased patient conversion 7%.

Upperline then expanded across Florida, then nationally, and later added call center support.

Each step earned the next one. A good partner should be glad to work this way, because they're confident the first step will work.

Rule 2: Don't leave AI alone

I've bet my career on this technology, and even I don't trust AI on its own. It still gets things wrong. Anyone who's used a chatbot has watched it hallucinate and answer, or generate a hand with six fingers. The problem isn't that it's wrong; it's that it doesn't know it's wrong.

In a lot of industries, 90% accuracy is fine. A mistake costs a refund or an annoyed customer. In healthcare, the other 10% is a patient’s PHI filed in the wrong patient's chart, an urgent referral marked as routine, or an incorrect insurance ID that creates a denial. The bar has to be 99% or better, because there's a patient on the other side of every task.

The way to close that gap is a human-in-the-loop. When the AI isn't confident, the task goes to a person who checks it, and that correction makes the AI better next time. Self-driving cars work the same way: when the car is unsure, a remote team takes over.

The real question is whose person that is. Some vendors hand you a tool and leave the uncertain tasks to your staff. That means another login, more training, and someone at each clinic checking the AI's work, which is a lot to ask of a team that's already stretched.

At Valerie, we do that review ourselves, so every fax, referral, and call is completed in your EHR without your team touching it. One CIO told us that was the deciding factor: he no longer had to worry about integration, training, or FTEs.

Either model can work. What I wouldn't trust is a vendor who says AI will handle everything with no people involved, because today that isn't true.

Rule 3: Find a true AI partner, not just a vendor

There are two reasons this matters.

First, AI is improving at an exponential pace. The tasks that models can complete on their own keep getting longer and more complex. You're not just buying what a partner does today; you're betting they'll keep up. Pick one that exists because of AI, not one that bolted it on.

Second, the front office is one connected system. A fax arrives with a referral. The eligibility check shows the visit needs a prior auth. Then the patient has to be scheduled, and maybe rescheduled. Split those steps across four vendors and patients fall through the gaps between them. Leaders of independent provider groups tell me all the time that they don't want a vendor for every step, and they're right.

You can usually tell on the first call whether a vendor can be that partner. They should talk specifically about how your group runs; if you hang up unsure whether they understand your workflows, trust your gut. They should explain exactly how they work inside your EHR, because if your team ends up copying data between systems, you lose most of the benefit. And when you ask for references, the answer should simply be yes.

Rule 4: Demand day-one ROI

With most software, the first day is the least valuable one. You pay, you train your team, you work through the kinks, and the payoff arrives months or years later, if it arrives at all. You go into the red to eventually get into the black.

AI should be different, because your partner isn't handing you a tool; they're doing the work. There will be rules to iron out in the first few weeks, but you should see results in weeks, not months or years.

When we launched with the largest pediatrics group in the country, their team told us: "You guys are on day 2 and it's where a new hire would be after more than a month."

ROI should quickly come from three places you can measure:

Lower labor costs: the work gets done without having to hire new staff.

More patients: more referrals converted, fewer abandoned calls, and shorter waits for an appointment.

Increased margin: new sites without new headcount, and every slot filled with the right patient for your services and payer mix.

Pinnacle Medical Group, a 9-site primary care group in Reno, cut its labor costs by 40% while opening a new clinic. As CEO Chris King put it: "With Valerie, we are saving money, my team is happier and patients are getting better care." Labor savings rarely mean layoffs. Most groups use them to cover turnover or avoid the next hire.

Labor matters even more in smaller groups. Here, AI might free up a quarter of someone's week rather than a whole role. The value then comes from what that person does with the time, like calling every patient you haven't seen in a year. If your schedules are already full, your payer mix is perfect, and nobody has time to spare, I genuinely don't know what the ROI is. I've just never met that group.

Finally, ask whether the partner will put their fees at risk, and be wary of anyone who wants a big upfront commitment before proving anything. When you only pay for work completed, the partner carries the risk instead of you.

If a partner can't answer these clearly, keep looking. If they can, pick the one problem costing your group the most today and start there.

We built Valerie to answer every one of these questions. We start with your most urgent problem, work natively inside your EHR, and keep our own team in the loop so nothing lands back on your staff. We only get paid when the work gets done. Then we earn the right to do more.

See how Valerie works for groups like yours.