Troubleshooting with AI in Support: What Actually Works, What Breaks

On a Tuesday morning a customer of a fintech startup had their scheduled payment fail. The app fires off a notification: “payment didn’t go through, we’ll retry soon”. Standard recovery flow, working exactly the way it’s supposed to.

A few hours later she writes in. The retry hasn’t happened. Or if it has, nothing on her end shows it.

The agent who picks up the ticket pulls up the knowledge base article on failed payment retry behavior. It’s a real article. Readable, even. It says a retry will fire within an hour. That was true two revisions ago.

The current retry handler runs on a different schedule, and nobody in support has any way of knowing, because nobody’s read the handler and nobody was invited to the PR review where the behavior last changed.

The agent tells the customer what the KB says. She checks. Still nothing. She writes back, less patient this time.

By the time the ticket works its way to engineering, she’s been waiting two days. A senior engineer (increasingly annoyed) opens the codebase, finds the branch, and writes back a two-paragraph explanation of what the retry logic actually does now.

Forty-odd minutes of expensive engineering time for a troubleshooting question the agent should have been able to answer themselves, if anyone had given them the tools.

At a company that ships weekly this is a regular event.

Most of the troubleshooting questions support gets asked aren’t answerable with the tools they have, but AI is changing this rapidly. This isn’t about replacing the KB. The KB is fine. It’s about handing support a flashlight for a room they’ve been navigating by feel for a decade.

Docs Aren’t Going to Catch Up

The reflex, once you’ve watched this type of ticket land enough times, is to fix the docs. Get the KB current. Add a stricter PR review step. Hire a technical writer. You know the drill. (We can fix this with headcount!)

I’ve watched a handful of teams try this. None of them quite get there, and I don’t think any of them really can. Code has a fast, painful feedback loop. When it’s broken, customers complain, charts go red, someone gets paged at 2 a.m. Docs don’t have anything like that.

When a help center article is three revisions behind the product, nothing happens. The page sits there, still getting traffic, still reading confidently, still wrong. The KB rots at the speed the product ships, and the people who understand the new behavior are the same ones too busy shipping the next version to document the last.

So keep the docs alive. Table stakes. But if the KB is the only knowledge source your support team can actually reach, every ticket that touches anything it doesn’t cover (conditional logic, feature flags, per-account behavior, grandfathered configs, the usual pile) lands on engineering by default.

Writing won’t close that gap. The thing support needs to see lives somewhere they can’t get to.

The Four Things You Need to See

When a customer writes in with a problem, there are four questions worth asking before you answer. Most support teams can reach one of them directly, maybe two. The other two are gated behind engineering, which is most of why so many tickets escalate.

First, what actually happened. Logs, the database, event streams, all read-only. Whether the webhook fired, whether the row exists, whether the event the customer’s asking about hit your system at all.

Second, what should have happened. The code. Scoped to the repos that explain customer-visible behavior. Your support team probably doesn’t even have read access.

Third, who else has hit this. Ticket and customer history. Past tickets on the account, past tickets with the same symptom across accounts, engineering notes from the last time this surfaced. It exists; it just isn’t searchable in any useful way.

Fourth, what the customer was told. The KB article the agent would cite, the help center page the customer probably read before writing in. Sometimes the bug isn’t a bug. It’s that the docs promised something the code stopped doing two releases ago.

Three Workflows You Can Run This Week

Wire up any one of those and ticket patterns shift a little. Wire up two and the support job starts looking materially different. Three plays, all runnable with tools that already exist.

Bug, edge case, or working as designed? 

Back to the retry logic.

With Claude connected to the repo, a support agent opens the ticket, points Claude at the retry handler, asks when a retry actually fires for a payment that failed the way this one did.

Claude walks through the schedule logic, names the branch that applies, cites the function and line number.

The agent reads the summary, eyeballs the cited line to confirm it exists, writes back to the customer in ten minutes with the real answer.

Bread and butter. Cursor handles it. Claude Code handles it. Codex handles it. The tool matters less than whether your support lead can operate one competently.

The retry logic I’m thinking of got revised a handful of times over a couple of years and the docs didn’t keep up. Claude doesn’t care. It’s reading the current code, which is the whole point.

One variant worth running before you write back.

Ask Claude to compare the behavior the code actually implements against what the help center article on the topic claims.

A big chunk of the tickets you’d otherwise escalate aren’t bugs. They’re doc drift.

The KB promised something the code doesn’t do, or stopped doing. That’s a writing ticket, not an engineering one, and once you can see the mismatch in thirty seconds you stop sending it to the wrong team.

Has Anyone Seen This Before? 

Most teams already have the answer to a given ticket buried in a closed thread from six months ago. Nobody remembers it. Helpdesk search is bad enough that nobody finds it either.

(Quick aside: somebody should write a whole piece on how bad helpdesk search has been. Zendesk, Intercom, Freshdesk, all of them have been promising to fix it for longer than I’ve been working in support. The words your customer uses are never the words the last agent used to close the duplicate ticket. Off-the-shelf search gives up at the second hop. The problem sat there for a decade waiting for something. Turns out the something was embeddings plus a language model.)

Plug Claude into Intercom through their MCP server (or Zendesk through Swifteq’s, or whatever your helpdesk offers, the MCP ecosystem has gotten notably better in the last six months) and the retrieval problem mostly solves itself.

The agent pastes in the customer’s symptom, asks for the five most similar closed tickets, gets a summary of how each was resolved. Two used the same workaround. That’s the lead. That’s often the whole ticket.

Go a step further and connect your team’s Notion and Slack MCPs to your AI of choice. Now all of the context of an issue with onboarding that for some reason happened in a thread for last year’s off-site is available to your team.

Did This Actually Happen? 

Half of what gets reported as a bug turns out to be misperception. Wrong environment. Cached UI. Customer looking at yesterday’s data wondering where today’s went.

Most of these questions aren’t a SQL problem. “Did the webhook fire?” “Did the user actually log in?” “Did the charge go through?” They’re answerable through tools you’re probably already running.

PostHog has an MCP server for product-event data. Stripe has one for payment state. Your observability stack almost certainly has something for errors the app hit but never surfaced to the customer. Plug the right one in and the agent asks, in plain English, whether the thing the customer’s asking about actually happened.

Raw database access is the backstop for the cases your analytics layer doesn’t cover. It carries more risk than the other workflows, because an AI-generated query can be wrong in ways that don’t look wrong. A bad filter. A misread join.

Keep it scoped to a read replica, keep whoever’s operating it fluent enough to sanity-check the query before running it, and don’t let raw query results into a customer reply without a human read. That’s the whole guardrail. It works if you respect it.

Where I’ve Watched This Break?

Two places, both things I hit personally, both of which shaped how I roll this out.

The first was Claude explaining a refund button. A team I was on had Claude plugged into the repo, and support asked what happened when a customer clicked refund in a less common flow.

Claude produced a confident, thorough, completely wrong answer. The code it cited was real. It was also from a deprecated path nobody had gotten around to deleting. The current refund logic lived in a different file, and Claude surfaced the wrong one.

AI tools don’t know what’s dead code unless you tell them, and mature codebases have piles of it lying around for exactly the reason I just gave.

The fix is a verification habit. Ask Claude for the file path, open it, confirm the handler’s actually wired to the current route.

Two extra minutes, a few times a day, and you dodge the category of error that would torch your team’s credibility in front of a customer the first time it hits.

The second was Fin, and complaint detection. I tried using Fin to auto-flag complaints inside tickets. The concept was fine. The execution wasn’t, because Fin couldn’t be fed enough context to reliably distinguish a complaint from a feature request from a frustration vented in passing.

False positives were high enough the signal was, functionally, noise. We moved to a purpose-built classifier and let Fin keep doing what it was actually good at.

(Poor Fin. Wrong tool for the job.)

The lesson’s broader than Fin. If the model can’t reliably get the context it needs, don’t force it. AI is excellent at narrow problems with rich context, merciless at broad problems with thin context. Pick your spots. The wins are at the narrow end.

The Prep Move That Saves You Later

Before you wire any of this up, do the single highest-leverage thing I’ve watched distinguish teams that got this working from teams that spent months debugging confident-but-wrong AI answers.

Write the model a short set of context notes about your codebase. (This is something you might need help from your engineering team but it is perhaps the most impactful way they can help you. Plus, they might’ve already done this for their AI workflows.)

Not for humans. For the model.

Call out the landmines. Which paths are dead code that nobody’s gotten around to deleting. Which service is confusingly named the opposite of what it does. Where the one genuinely weird conditional lives.

The fact that the payments table has both legacy and current rows with different schemas, and the legacy ones are still getting read by a cron nobody wants to touch. A page or two, dropped at the root of the repo as a CLAUDE.md or similar, whatever file your tool of choice reads first.

This directly prevents the category of failure the refund-button story describes. The team I was on hadn’t written anything like it.

If we had (“warning: the refund handler in the old path is dead code, current refund logic is in billing/refunds/v2”), Claude wouldn’t have cited the wrong file. The fix wasn’t more verification.

It was giving the model the same context the engineers had in their heads, in a form the model could read.

The Security Question

Is exposing your source code and data a risk? It is always the first question, and, frankly,  it’s the right one to ask.

Risk gets managed, not eliminated. Lean into read-only scopes. Enterprise deployments where your prompts and outputs are contractually excluded from training data. MCP servers that are scoped to specific repos or tables, not the whole system. Audit logs on every query. In truth, none of that buys you zero risk.

You’re weighing risk against the current practices of the team, which at a lot of companies looks like support pasting customer data into Slack DMs with on-call engineers, ad-hoc database queries from whoever still has production credentials from their last rotation, Zoom screen-shares with sensitive customer data visible for the whole call.

That process has real security problems. It just doesn’t get scrutinized as security, because it’s familiar.

Do the security review. Start narrow with the lowest-risk workflow you picked. Widen scope as trust compounds. Don’t skip the review; also don’t pretend the status quo is the zero-risk baseline it quietly gets treated as.

Wiring It Up Without Burning the Team

Start with the workflow that can’t hurt anyone. If the team’s been burned before, they have reason to be skeptical, and customer-facing hallucinations end experiments fast.

We started with Intercom ticket-history triage, researching tickets against our codebase to solve tricky problems and then helping draft escalations for identified bugs. (If you’re on Zendesk, Swifteq’s Zendesk MCP Server plugs straight into assistants like Claude for this exact workflow, without you building the connector yourself.)

Measure quality, not count. Escalation rate was the obvious metric. Escalation quality was the useful one: how thorough was the writeup, did engineering have to come back, did the ticket move forward without a hard stop. Engineering felt the difference before the dashboards caught up.

Widen scope in slow, visible steps while checking in along the way. Don’t be afraid to change course, or experiment as you learn and the landscape of tooling changes. The future of support is AI enabled humans, not AI only.