What callers actually said: bot journeys, turn-by-turn replay and the no-match backlog

A parts and ordering line runs a Genesys Dialog Engine voice bot. It asks what the caller needs, captures a part number and either answers or transfers to the right queue. On Monday at 09:10 the queue supervisor notices more callers arriving at agents without a captured part number, and asks the bot owner a fair question: what are callers actually saying to the bot, and where is it losing them? Is the problem the intent model, the part-number capture, the reprompt wording, or callers who simply ask for a person?

Answering that needs three views at once. The aggregate journey, so you can see which ask is leaking. One caller's session, turn by turn, so you can hear the failure in context. And the full list of things callers said that the bot did not understand, ranked by how often they said them, because that list is the tuning backlog. Bot Flow Diagnostics puts all three on one harvested population.

Genesys provides a strong set of bot insights. The Insights and Optimizations menu in Dialog Engine bot flows and digital bot flows brings together the Optimization dashboard, Intent Miner, Journey Flows, bot performance reports and the Virtual Agent dashboard. The Optimization dashboard shows total interactions, average duration, average turns and end states such as contained, transferred, agent escalation, recognition failure and abandoned, with trend lines against the previous period, a Top Bot Flow Actions with Issues list, an intents breakdown, the Utterance History view and a Bot Conversation Library.

Intent Miner mines intents, utterances, topics and phrases from chat and call recordings, which is the right tool for designing new intents. The Bot Performance Detail view in Analytics Workspace reports entries, durations, turns and intents over custom ranges of up to six weeks. Our app does not replace any of this. It adds what an analyst tuning one bot keeps asking for: a full-population journey diagram, an exact replay of any session, and a ranked corpus of failed utterances that outlives the platform's turn-level retention.

Genesys collects Bot Performance data automatically for every Architect bot flow, with no per-flow opt-in, for voice and digital bots alike. Each bot session ends with a bot result, such as TransferToACD or DisconnectRecognitionFailure, grouped into a result category. Inside it, each reporting turn carries the bot's prompts, the caller's utterance, the ask action that ran, the matched intent with its confidence and slot values, and the ask result: SuccessCollection, NoMatchCollection, NoInputCollection, AgentRequestedByUser and so on. On voice, the utterance is the speech-to-text output, exactly the text the language model judged.

That makes every session a path and every turn a small conversation. Project each session into a bounded path of prompt, ask and result steps ending at a terminal, and the population becomes a directed graph. Keep the turns, and any path can be replayed. Collect the no-match utterances, and you have the backlog. Third-party bots do not report through these APIs, so they are out of scope; for Architect bots the coverage is complete.

A run is one bot flow and one UTC window of up to 42 days. The app compares the window with the flow's stored coverage, polls only the gaps plus a six-hour overlap at the tail so late-settling analytics refresh, and walks two cursors against Genesys at 250 items a page: /api/v2/analytics/botflows/{id}/sessions and the matching division-aware reporting turns endpoint. Turns are grouped by session id, each session is projected into its journey path and stored once, keyed by flow and session, so a re-harvest reuses everything already downloaded. A typical window finishes in seconds to a couple of minutes, because there are no per-conversation jobs.

Safety bounds keep it honest. Each uncovered gap's session walk has a cap, 1,200 by default and adjustable up to 20,000, and the turns walk has a derived bound. If either is hit, the run says so and deliberately does not record coverage for that sweep, so nothing is silently missed. A best-effort POST to /api/v2/analytics/bots/aggregates/query corroborates the tiles but is never a dependency. The Genesys client refuses every non-GET request except a short allowlist of read-only analytics queries, runs behind a per-organisation rate limiter and honours Retry-After on 429 responses. The OAuth client needs analytics:botFlowSession:view, analytics:botFlowDivisionAwareReportingTurn:view and architect:flow:view.

The journey map is a zoomable directed graph rendered on the server from the run's stored paths. Ask actions are decision nodes, intent matches, no-matches and no-inputs are result nodes, milestones and nested bot calls get their own nodes, and end states such as Transfer to ACD, Caller hung up and Recognition failure are terminals. The bot's spoken prompts are a layer that is on by default, because for a bot its speech is the product. Reprompt loops are drawn rather than dropped. Node labels show reach as a count and percentage of all sessions, edge width encodes volume, and hovering an edge shows its share of the source node's outflow.

Node labels come only from bounded sets, such as ask names, intent names, result taxonomies and prompt texts, never from raw utterances, so the diagram cannot explode in size. A Summary panel adds client-ready material from the full population: a clean partition of sessions into transferred, hung up in bot, transferred elsewhere and self-served, friction hotspots, top successful outcomes, the last prompt heard before callers abandoned, the busiest journeys and a fidelity score for how faithfully one aggregate diagram reproduces real step order. The map and the summary both export as single-page branded PDFs.

Session replay lists the run's sessions with their end category, final matched intent, channel and step count, and replays the selected one as a conversation: bot bubbles for each prompt, a caller bubble with the utterance or no input, a colour-coded ask result, an intent confidence bar, green at 80% and above, amber from 55%, red below, and slot chips with their own confidence. The header carries the Genesys conversation id so any session can be cross-checked elsewhere, and a map node filter narrows the list to sessions that passed through that node.

The Utterances page aggregates every stored turn. Tiles show sessions, turns, utterances captured, match rate, no match and no input. The no-match table merges phrases case-insensitively, ranks them by frequency and links up to three sessions to hear each in context. On voice that includes the literal misheard forms, the pna for P&A, which are exactly what to add as grammar entries. Per-ask cards show result mixes and match rates, low-confidence matches under 70% are listed for mis-route checks, and an intents table gives average and minimum confidence. Match rate excludes agent requests and guardrail results, because a caller asking for a person is an escalation, not a recognition failure. The whole corpus exports as a five-sheet Excel workbook.

A Bot tuning panel on the Utterances page can hand a large language model the run's per-ask digest, including every no-match utterance with its count, and returns prioritised recommendations with click-to-copy additions ready for Architect: grammar additions, STT variants, intent gaps, reprompt wording, escalation handling and format hints. It is opt-in per click, cached per run, and sends nothing until a user presses Generate. A separate analyst narrative uses aggregate metrics only, never caller speech. The warehouse itself holds caller utterances and slot values, so we advise clients to treat it, and its exports, with the same care as call recordings.

Bot Flow Diagnostics was built the QVCCS way. Our Senior Business Consultant and Business Analyst defined the questions bot owners ask after go-live; the Solution Architect designed the path projection and the match-rate semantics; Senior Developers built the harvest and renderers under the Senior Platform Practice Lead's engineering standards; and our Systems Integration Tester proved the caps, coverage rules and failure paths against a deterministic synthetic bot, so the demo exercises every feature without customer data. For clients, it closes the loop from what callers said to what to change in Architect next.