Skip to content

CX without the BS

Honest takes on customer experience and UCaaS for the SMB and mid-market buyers enterprise vendors forgot... hosted by Brian Nichols.

Your Chatbot Is Only as Smart as What It Knows: Kate Nelson and Vicki Brackett on the Biggest CX Mistake Companies Make

Brian Nichols | Host, CX Without the BS

A contact center in Texas has 950 agents and a beautiful SharePoint site. Four people manage it.

Behind it sit 85 subject matter experts, four team leaders, three quality people and an operations manager. Their job? Answering the questions agents can't find in that beautiful SharePoint site.

The bill, according to Vicki Brackett, comes "to the tune of over $4.2 million in labor costs to support the agents when they can't find the answer in the SharePoint."

That's not a technology problem. That's a decision problem. And it's the thread that ran through my conversation with Kate Nelson and Vicki Brackett on CX Without the BS.

Vicki has led customer service organizations for more than 20 years and now runs Knowledgely, a technology-agnostic knowledge management practice that also puts fractional knowledge managers inside companies. Kate, at the time we recorded, was with CX Consulting Partners, helping businesses roadmap their customer journey and pick the right tools, so they're "not getting lied to."

I asked them what the biggest CX mistake companies make is. They had a few.

Listen to the full episode: What Is the Biggest CX Mistake Companies Make?

Mistake 1: Buying the Bot Before You Fix the Brain

Everybody wants AI right now. Kate sees the same pattern over and over. A company gets excited, buys a chatbot, and then it doesn't work the way they wanted.

What the vendor never told them, in Kate's words, "because then they wouldn't buy their solution," is that without "one single source of truth," the bot is just as chaotic as the old knowledge library it's pulling from.

Her fix is to map the full customer journey first, current state and future state, and then layer technology on top where it makes sense. "So it needs to be building blocks, or else you're just spending a lot of money on AI that's going to blow up."

Vicki drew a line most buyers never draw. "A knowledge base is different than a knowledge management system." A base is a pile of articles. A system pops the answer at conversational speed, runs decision trees, integrates with talk tracks and the CRM, reports on what's being used, and gives employees a feedback loop. And it sits behind everything: the chatbot, the agents, the company app and the website. One engine.

I said it on the show. It doesn't matter how great the AI is if it's feeding off bad information.

Mistake 2: Starting With the Tool Instead of the Customer

When I asked where a company should start from ground zero, Kate didn't name a single product.

Start with how you want the customer to feel before, during and after they buy. Then decide what to measure. She called out one shift she's grateful for: brands moving off average talk time and toward first call resolution.

"I'd rather have a 20 minute call and not have the customer have to call back, versus having a customer call three times."

Vicki added the step everybody skips. "Ask the employees, the frontline folks that are interacting with customers, ask the customers." Find the friction. Fix that first.

I've watched advisors walk in with a playbook that ignores all of that. It's like a hotshot offensive coordinator who installs his deep-bomb play on a new team. Great play. Except the team doesn't have a fast wide receiver. If you don't know what the boots on the ground are dealing with every day, it doesn't matter how good your playbook is.

Mistake 3: Forgetting Your Agents Are Customers Too

Kate remembered the first time a boss told her about "internal customers." She didn't get it. Then she realized the salespeople she supported were her customers.

Agents are the same. "If your employees are miserable, that's going to bleed over and vice versa."

The fastest way to make an agent miserable? Make them swivel between systems that don't talk to each other, put the customer on hold, and go hunting through SharePoint. Kate's line on where AI fits: "technology is here to support us, not replace us."

I brought up a story Jason Lowe shared on the show last year about Klarna, the buy now, pay later company that set out to replace around 700 agents with AI. Fast forward, and they're bringing people back. It's not AI that replaces people. It's people who use AI who replace the ones who don't.

Mistake 4: Letting the Knowledge Rot

So how do you keep the information agents and bots use from going stale? Vicki's answer started with one word: governance.

Someone owns the knowledge. Different people can author it. Legal, marketing or HR review it before it goes live. Then build a feedback loop where an agent clicks a button to flag a typo or a missing step, and hears back when it's fixed. Keep the return policy in the same spot on the screen every time, so the agent's eyes go straight to it. And when an article changes, every bookmark updates with it.

Why it matters: "if the agents aren't confident in the information, they're not going to use it." They'll ask the person next to them, ping the chat room or escalate to tier two. That costs money. That's the $4.2 million in Texas.

She also called out the ritual of making agents click a box so the manager knows they saw an update. Agents are moving so fast they click the box, "the report says you understand it, but then you make the mistake when you're talking to a customer."

It matters even more when something goes wrong. Vicki walked through a product safety call. The first caller gets put on hold while everyone scrambles for legal. Then a second caller. Then a third. "Now we're on the news. We have to be saying the same thing."

Mistake 5: Buying in the Wild West

Kate hears the same thing from CX leaders constantly. They're told to build an AI roadmap, they don't know where to start, and every vendor says they can do everything.

"AI is like the Wild West," she said. Pricing comes in tokens, per interaction or by the minute, and buyers have no baseline to compare. So they take the vendor's word for it.

Her advice: "let's slow this down. Stop talking to every single vendor that tries to pitch you." Write down the "top 10 to 20 things that are absolutely critical." Then make vendors prove it with a structured demo of your use case, in your environment.

Skip that, and you end up with the story she hears too often. The tool is missing two features you really needed, "but we already bought it. Now we're in a two year contract."

Vicki added why this is so hard from the inside. Leaders spend their days putting out fires, and sometimes "we as CX leaders might not even know what questions to ask." She also warned that "any kind of technology acquisition can be a deal breaker for an executive's career."

The Biggest BS: 45 Minutes for a 25-Second Fix

I close every episode by asking for the biggest pile of BS in CX. Vicki brought a story from three weeks before we recorded.

She spent 45 minutes fighting a chatbot to get a phone number that wasn't on the website. She called the IVR. She explained her problem again. And the bot told her, "if I transfer you to a human, they're going to tell you exactly what I've just been telling you."

When she finally reached a person, the fix was uploading her driver's license. "It was a 25 second fix." She had been stuck with the bot for over an hour.

Kate's reaction: "That's enough to leave a brand."

The Real Mistake: Leaving CX Out of the Room

Kate's pick for the biggest BS ties the whole conversation together. Companies are buying customer-facing technology without the customer experience team in the room. Not the frontline agents, and often not even the CX leaders who have to roll it out and manage the change.

"CX is now a revenue driver," Kate said. So "they need a seat at the table."

Vicki's final words were the same idea in fewer words: "let's get a CX seat at the table."

If you're about to sign for an AI tool, a few questions worth asking first:

  • Do we have one source of truth, or are we about to point a bot at a messy SharePoint?
  • What do our agents and customers say the biggest friction is?
  • Are we measuring how fast calls end, or whether the problem got solved?
  • Who owns our knowledge, and how does an agent flag something that's wrong?
  • What are our top 10 to 20 must-haves, and did we see them work in our environment?
  • Was the CX team in the room when we picked this?

Listen to the Full Episode

Kate and Vicki bring decades on the front lines and zero patience for snake oil. If you're buying, selling or advising on CX technology, this one is worth your time.

Listen to Kate Nelson and Vicki Brackett on CX Without the BS

Subscribe wherever you get your podcasts. Send it to the person on your team who's about to sign an AI contract.