Bot Flow Diagnostics

1What it is

Bot Flow Diagnostics answers the questions Genesys’ own Bot Performance screens make hard to see at a glance: where do conversations with our bot go, what does the bot actually say, what do callers actually say back, and where does recognition break down? Pick a bot flow and a UTC window; the app walks every bot session and every reporting turn in that window and turns them into:

• The journey map — a zoomable diagram of the whole population: the bot’s spoken prompts as a toggleable layer, ask actions as decision nodes, intent matches / no-matches / no-inputs as result nodes, milestones and end-states (transfer, contained, recognition failure, hang-up) as terminals. Node labels show reach (“count (% of all sessions)”); edge labels show volume and the source node’s branch split.
• Session replay — any session behind the map, turn by turn: bot bubbles, caller bubbles, colour-coded ask results, intent confidence bars and slot captures.
• The utterance corpus — the run’s ranked no-match utterances (what real callers said that the bot didn’t understand — the tuning backlog), per-ask match rates and result mixes, low-confidence matches and intent confidence stats, exportable as a five-sheet Excel workbook.
• Bot tuning recommendations — an opt-in LLM analysis of the failed utterances that returns concrete strings to paste into Architect (intent utterances, slot synonyms, STT variants, reprompt wording, escalation handling).

Why the Bot Performance APIs

Unlike flow-execution capture (the IVR Sankey data source), Bot Performance data collects automatically for every Architect bot flow — no per-flow opt-in — and covers voice and digital bots. Its trade-off is retention: Genesys deletes bot sessions and reporting turns after approximately (but not before) 10 days. Bot Flow Diagnostics harvests into a private, gap-aware warehouse, so history you have polled once stays available after the platform ages it out — harvest regularly and the corpus outlives the API horizon.

2Quick start

1. Sign in — region + OAuth client-credentials Client ID/Secret (14-region picker). The client needs analytics:botFlowSession:view, analytics:botFlowDivisionAwareReportingTurn:view and architect:flow:view (see §10).
2. Pick a flow + window — filter the bot/digital-bot list by name, set a UTC window (at most 42 days; Genesys retains only about 10 days of session and turn detail, so older days show data only if they were harvested while still available). Once a flow is picked, the cache-coverage calendar appears: click one day to start a range, another to complete it.
3. Build journey map — the harvest streams progress live (sessions walked, turns walked, per-window logs) and links to the map when done. Typical windows finish in seconds to a couple of minutes: two cursor walks, no per-conversation jobs.
4. Explore — toggle the Asks / Prompts / Intents & results / Milestones & endings layers, hover any line for its volume and branch share, open the Summary panel, download the Diagram PDF.
5. Drill — Session replay for the turn-by-turn view; Utterances for the corpus, the tuning panel and the Excel workbook.

No credentials to hand? View demo renders the full product — map, summary, replay, corpus and a canned tuning report — from a synthetic bot, signed out (§9).

3Harvesting & the warehouse

How a harvest runs

1. One run = one bot flow + one UTC window (≤ 42 days). The requested window is compared against the flow’s stored coverage intervals; only the gaps are polled, plus a 6-hour recency overlap at the tail of covered time so late-settling analytics get refreshed (re-saves are keyed by session id, so nothing double-counts).
2. Per gap, two cursor walks against Genesys (250 items per page, after cursor): sessions (every bot session with its end result) and reporting turns (every prompt/utterance/ask/result exchange). Turns are grouped by session id.
3. Each session + its time-ordered turns is projected into a bounded journey path (§8) and stored — the full path with slimmed turns in the detailed layer, the same path without turns in the summary layer (which powers the calendar’s per-day counts). Sessions are stored once, keyed by (org, flow, session id, layer) — a re-harvest reuses everything already downloaded.
4. If nothing was truncated, the swept window is merged into the coverage set. A final best-effort aggregates cross-check runs (§10), then the run completes with counts: sessions in window, newly polled vs reused from cache, and turnless sessions.

Caps and safety bounds

• Max sessions per sweep (default 1,200; 50–20,000) bounds each uncovered gap’s session walk. The turns walk is bounded too (40 turns per allowed session, between 10,000 and 250,000). If either bound is hit the run reports it and deliberately does not record coverage for that sweep, so nothing is silently missed — narrow the window and re-run.
• Re-poll the whole window (ignore cache) re-walks covered time too — use after an extraction change. Re-polling only adds or refreshes; cached sessions are never lost.
• Abort stops the run cleanly (“Aborted by user (progress saved)”): sessions already saved stay in the warehouse, but no new coverage is recorded.

Progress, restarts and the calendar

Progress is reported live and survives a reload — navigating away and back (or a dropped connection) resumes cleanly; if the job is no longer running, the page reports the run’s real final state instead of hanging. A service restart interrupts an in-flight run, which is then marked as an error (“Interrupted by a server restart — please re-run the harvest”). Everything saved before the restart is kept and reused.

The coverage calendar (UTC, month-paged) shades each day — green fully cached, amber partial, white uncovered — and shows the cached session volume per day, so a quiet overnight day reads as low-volume rather than “broken”. Click a day to anchor a range, a second day to span; the pickers stay in sync.

4The journey map

The map is a Graphviz-rendered SVG built from the run’s stored paths and shown in a scroll-and-zoom container, so a wide bot flow can be panned end to end. It is a real directed graph — cycles (reprompt loops) are drawn rather than dropped, and convergence nodes like Transfer to ACD show their true reach.

Layers & navigation

• Layer toggles (top-left): Asks (the ask-action decision nodes), Prompts (what the bot said — on by default: for a bot, its speech is the product), Intents & results (match / no-match / capture outcomes) and Milestones & endings. The flow backbone always shows; toggling a layer keeps your zoom.
• Hover any edge — it highlights and a panel shows origin, target, session count and the share of the source node’s outflow taking that hop (a node’s outgoing edges sum to ~100%). Edge width encodes absolute volume.
• Node labels show reach: distinct sessions passing through, as “count (% of all sessions)” — well-defined even when callers loop.
• Auto-pruning: for populations of 200+ sessions, edges below ~0.5% of the population (minimum 3) are pruned so a few pathological loops don’t stretch the diagram.
• Diagram PDF (top-right) exports the diagram exactly as configured — same layer toggles — as a single-page, 100%-scale, branded PDF (title, window, colour-key legend); Graphviz sizes the page to the diagram, so nothing tiles or shrinks.
• Zoom −/Reset/+ buttons (0.3×–3×); the scroll area grows with the zoom so the full width stays pannable.

The header links to Utterances and Session replay for the same run, shows the harvested UTC window, and the footer strip carries the colour key: cyan tags are what the bot said, indigo nodes are asks, teal/red notes their match results.

Client summary panel

The Summary toggle opens a right-hand panel of client-ready material, all computed from the run’s full population:

• KPI tiles — a clean partition of every session: Transferred to agent queue, Hung up in bot, Handled by agent / Abandoned in queue (bot paths end at the transfer, so these read 0 — cross-check queue outcomes in Conversation Analyser), Transferred elsewhere, and Self-served (reached a success outcome without any transfer).
• Diagram sequence fidelity — how faithfully the single aggregate diagram reproduces the true per-session step order (branching flows genuinely vary; the replay view is always exact).
• Analyst narrative — templated one-line findings always show; Generate AI analysis optionally sends the run’s aggregate metrics (counts, outcome and ask names — no caller speech) to a large language model service configured by QVCCS for a written CX analysis: executive summary, key findings, strengths, areas for deeper review. Opt-in per click, cached per run, Regenerate to refresh; available where QVCCS has enabled it.
• Friction hotspots (failed outcomes such as Recognition failure), top successful outcomes (including milestones reached), “Heard before abandoning” — the last prompt played before sessions that hung up without reaching a success outcome — and the busiest journeys (top full paths with counts).
• Export PDF — a branded one-page summary; it includes the AI narrative only if one has already been generated, and never generates one by itself.

5Session replay

Session replay lists the run’s sessions (newest first, 50 by default) on the left and replays the selected one on the right. Each list card shows start time (UTC), the end-category chip (Transfer sky-blue, BotExit/BotDisconnect green, UserExit/UserDisconnect/SessionExpired amber, RecognitionFailure/Error red), the final matched intent, channel and step count.

The replay renders each reporting turn as a conversation:

• Bot bubbles (cyan) — every prompt the bot spoke or sent that turn;
• Caller bubble — the utterance (STT text on voice), or “— no input —”;
• Result chip — the ask result, colour-coded (Success green, No match red, No input / Confirmation-No / Agent-requested amber, Guardrails/Error red);
• Intent + confidence bar — green ≥ 80%, amber ≥ 55%, red below;
• Slot chips — captured slot name/value pairs with their own confidence.

The session header carries the Genesys conversation id alongside channel, language, raw bot result and result category — so any session can be cross-checked in Conversation Analyser. Sessions with no stored turns show an explicit empty state (they ended before the first ask — real behaviour for immediate hang-ups; they still count in the end-state totals).

Links from the map can filter the list to sessions whose path passes through a given node, and the utterance corpus’s “hear it in context” links open a specific session even when it is outside the visible list.

6The utterance corpus

Utterances (from the map header) aggregates every stored reporting turn in the run. Six tiles head the page: Sessions, Turns (with how many carried an ask result), Utterances captured, Match rate (amber below 85%), No match and No input.

No-match table — the tuning backlog

Every phrase callers said that the bot did not understand, merged case-insensitively, ranked by frequency, with the ask it happened at and up to three “hear it in context” links that jump straight into the session replay at that session. Each row is something real callers say that the bot’s grammar doesn’t yet cover — on voice, including the literal misheard STT forms (“pna” for “P&A”), which is exactly what you need to add as grammar entries.

Per-ask breakdowns & intents

• One card per ask action, ordered by volume: a colour-coded result-distribution bar (green success, red no-match/guardrail/error, amber no-input/agent-request) with a legend, the ask’s turn count and its match rate (amber below 85%).
• Low-confidence matches (< 70%, worst first, up to 25 per ask) — successful matches worth verifying for mis-routes, each with the utterance, matched intent, confidence and a replay link.
• Matched intents table — match counts, average confidence (amber below 75%) and minimum confidence per intent.

Excel workbook

Download utterance workbook (Excel) exports the corpus as a five-sheet .xlsx analysis pack (named after the flow):

7Bot tuning panel

The Utterances page carries a sticky right-hand Bot tuning panel. Press Generate tuning recommendations and the analyser hands a large language model the run’s per-ask digest — the bot’s prompts for context, the result distribution, every no-match utterance with its count, no-input and agent-request rates, low-confidence matches, and samples of utterances that did match (for contrast) — and forces a structured response back.

Each recommendation carries a priority (high/medium/low), a kind, the observed issue (quoting real utterances and counts), click-to-copy addition chips (plus Copy all) ready to paste into Architect, and the rationale. The kinds:

Generation is opt-in per click and the report is cached per run — reopening the page loads it instantly; Regenerate re-runs the analysis. The analysis runs on a large language model service configured by QVCCS; where it is not enabled, the panel says so.

8Concepts & semantics

Session

One conversation’s pass through one bot flow, ending with a bot result (e.g. TransferToACD, ExitRequestedByUser, DisconnectRecognitionFailure) bucketed into a result category: Transfer, BotExit, BotDisconnect, UserExit, UserDisconnect, RecognitionFailure, SessionExpired, Error. The warehouse key is the bot session id (a conversation can hold several bot sessions).

Reporting turn

One exchange inside a session: the bot’s prompts, the caller’s utterance (STT text on voice), the ask action that ran, the matched intent with confidence and slot fills, and the ask result — SuccessCollection, SuccessConfirmationYes/No, PartialCollection, NoMatchCollection (heard but not understood), NoInputCollection (silence), AgentRequestedByUser, GuardrailsViolation, Error.

How the map is built

Each session projects into a bounded path: Start → Bot: <flow> → [“prompt” → Ask → result]* → terminal. Node labels come only from bounded sets — ask names, intent names, result taxonomies, prompt texts (which repeat per ask, truncated at 90 chars) — never from raw utterances, so the diagram cannot explode in cardinality. Utterances live in hover detail and the stored turns. Milestones (AddFlowMilestoneAction) and nested bot calls become their own nodes; pathological looping sessions truncate at 80 steps rather than being dropped.

Terminal nodes

Result category → node: Transfer → Transfer to ACD · BotExit → Bot completed · BotDisconnect → Bot ended session · UserExit → Exited by caller · UserDisconnect → Caller hung up · RecognitionFailure → Recognition failure · SessionExpired → Session expired · Error → Bot error.

Colour vocabulary

Purple = bot flow boundaries · cyan = what the bot said · indigo = ask actions · teal = successful intent/capture results, red = failures, amber = no-input/rejection · green/red = success/failed outcomes · sky-blue = transfers · slate = Start.

Retention & windows

Genesys keeps bot sessions and reporting turns for approximately 10 days; harvest windows are capped at 42 days and the end is clamped to now. The local warehouse keeps whatever it has harvested indefinitely — the coverage calendar is the honest record of what is cached.

Ask actions recognised

AskForNLUIntentAction, AskForNLUNextIntentAction, AskForSlotAction, AskForBooleanAction, AskForAuthenticationAction, AskForPaymentAction, AskSurveyQuestionAction become Ask: nodes; CallBotFlowAction / CallDigitalBotFlowAction / CallAgenticVirtualAgentAction become nested Bot: nodes.

9Demo mode

The demo is a deterministic synthetic corpus — the “Parts & Ordering Bot”, ~200 voice sessions built from 8 weighted blueprints, tuned to look like a real mid-tuning bot: strong intent capture, a no-match tail on part-number capture, an agent-request path and a recognition-failure tail. It exercises every surface signed out — map, session replay and utterances — with full turns behind the replay.

The demo’s Generate buttons for the analyst narrative and the tuning report return canned reports — no AI call — so the opt-in workflow can be demonstrated end to end.

10Genesys endpoints & permissions

One further endpoint exists in the app — GET /api/v2/architect/prompts/{promptId} (architect:userPrompt:view), the diagram’s click-to-play prompt-audio lookup shared with IVR Sankey. Bot prompt nodes are built from reporting-turn text and carry no Architect prompt id, so this lookup does not fire on bot runs.

Traffic shape: all calls run through a per-org rate limiter with a one-shot token refresh on 401 and Retry-After-honouring back-off on 429. A 10,000-session window costs roughly 40 session pages plus ~1 page per 250 turns.

11Storage, privacy & security

• Warehouse — harvest runs (with their aggregates), sessions (path steps, session attributes, slimmed turns) and the covered-interval set per flow are kept in the app’s own private store, with cached AI reports alongside, one per run.
• Owner scoping — everything is keyed to the org credentials that harvested it. Different orgs never see each other’s data; the same org signed in again sees its own cache. Browser sessions are individual, but the warehouse is shared per org — a colleague’s harvest benefits everyone.
• Credentials — the Client Secret is held encrypted on the server, never written in plain text and never sent to the browser; tokens refresh automatically. Only an encrypted sign-in is kept, so a service restart does not force re-login. Sessions end after a period of inactivity.
• What the warehouse contains — per-session journey paths and slimmed turns including caller utterances (STT text) and slot values. Utterances are caller speech and can contain personal data: treat the harvested data and the Excel exports with the same care as call recordings. Nothing is redacted; the mitigations are restricted access, owner scoping and the read-only Genesys client.
• What leaves the service — nothing, except the two opt-in AI calls to a large language model service configured by QVCCS: the analyst narrative sends aggregate metrics only; the tuning analysis sends the failed-utterance corpus (raw text — the analysis subject). Both fire only on an explicit button press and cache their result.

12Technology

• Web application — the bot-session warehouse, a harvest engine with live progress, a per-org rate limiter in front of the Genesys APIs, and the utterance workbook builder.
• Renderers — the journey map and its single-page PDF are rendered with Graphviz; the branded summary PDF is rendered by the app.
• Browser interface — map, replay and corpus pages; this guide ships with the app and is readable without signing in.

13Troubleshooting

The flow picker shows “0 of 0 flows”.

The org has no Architect bot/digital-bot flows visible to this OAuth client. Per-type listing failures are logged rather than surfaced, so a missing architect:flow:view (or division scoping) presents as an empty list, not an error — check the client’s role and divisions.

The harvest fails naming a missing scope.

403s carry the exact permission: grant analytics:botFlowSession:view and analytics:botFlowDivisionAwareReportingTurn:view to the OAuth client’s role. Token scopes are minted at sign-in — sign out and back in after changing permissions.

The harvest log warns “Bot aggregates unavailable…”.

The best-effort aggregates cross-check (§10) failed — most often a 403 because the OAuth client lacks analytics:botAggregate:view. Every tile and KPI computes from the harvested sessions themselves, so nothing is missing from the analysis; grant the scope (then sign out and back in) if you want the cross-check stored with the run.

0 sessions found.

The bot did not run in the window, or the window is older than the ~10-day Genesys retention for bot sessions and turns and was not harvested before then. Try a recent week first; note the “To” time is clamped to now.

“Session cap hit — coverage not recorded for this sweep.”

A gap exceeded Max sessions per sweep (or its derived turn bound). The fetched sessions are stored, but the window is deliberately left uncovered so nothing is silently missed — narrow the window (or raise the cap) and harvest again.

“Interrupted by a server restart — please re-run the harvest.”

A service restart interrupted this harvest, so it was marked failed. Re-run it — completed windows are served from cache, so the re-run only polls what was in flight.

A session replay says no reporting turns were stored.

Real behaviour: the session ended before the first ask (immediate hang-up or error). It still counts in the end-state totals and the map’s terminal nodes.

Utterances look misspelled or odd on voice.

They are speech-to-text output — what the recogniser heard is exactly what the NLU judged. That is the point: no-match rows show the raw misheard text (“pna” for “P&A”), which is what to add as an STT-variant grammar entry.

The no-match table is empty.

Either a healthy window (every ask matched, timed out, or escalated) or a window with no asked turns. Check the tiles — Turns vs with an ask result tells you which.

“Journey render failed” / map error.

The diagram renderer is unavailable — a service problem, not a data one. Contact QVCCS support.

The tuning / narrative panel says “not configured”.

The AI features are not enabled for this service. Ask QVCCS to enable them — both features stay opt-in per click even when enabled.

Everything 401s mid-session.

The session ended after a period of inactivity. Sign in again — the warehouse, coverage and cached reports are keyed to the org, not the session, and are untouched.