Deflection Rate from Knowledge Base

Goal of the analysis:

The purpose of this analysis is to quantify how effectively the knowledge base (KB) or self-service content prevents customers from contacting support. Executives care because higher true deflection reduces service costs, improves speed-to-answer, and can raise satisfaction when issues are resolved instantly. The analysis distinguishes between “assisted containment” (customers briefly use content but still contact support) and “true deflection” (issues resolved without any subsequent contact within a defined window), enabling focused investment in content, search, and journey design that materially lowers demand on assisted channels while protecting CX.

Data required:

• Knowledge base content and metadata:
  • Article IDs, titles, categories/taxonomy, tags, publish/update dates, language/locale.
  • Article quality indicators: owner, review status, CSAT on article, “Was this helpful?” votes.
• Web and in-product analytics (self-service interaction):
  • Sessions, pageviews, time-on-page, scroll depth, exit pages.
  • Site search queries, click-through from search results, zero-result searches.
  • Events: “Mark solved,” copy code, download, link clicks to troubleshooting steps.
  • Device, browser, geo, referrer, authenticated user ID (where available).
• Assisted contact data (CRM/CCaaS):
  • Tickets/cases/chats/calls: IDs, timestamps, channel, issue category, priority/severity.
  • Customer/account ID, segment, product, region.
  • Resolution codes, handle times, transfers, reopen rates.
• Identity and journey linkage:
  • Visitor IDs/cookies, hashed user IDs, login events to map sessions to contacts.
  • UTMs or referral params that indicate support entry points.
• Cost and experience metrics:
  • Cost-per-contact by channel (phone, chat, email) and per case type.
  • CSAT/CES/NPS for KB and for assisted contacts, complaint/churn signals where available.
• Reference and benchmark data:
  • Historical volumes, seasonality, product launches/incidents.
  • Internal benchmark groups (e.g., top quartile articles, pilot markets).

Detailed step-by-step instruction on how to conduct the analysis:

1. Define eligibility and time windows:
   • Eligible self-service sessions: visits to support portal, KB, or in-product help where the customer exhibits intent (e.g., uses search, views ≥1 article).
   • Lookback/forward window to detect subsequent contact: commonly 24–72 hours; use 7 days for complex B2B issues.
2. Instrument and extract data:
   • From analytics (e.g., GA4/Adobe/Heap), pull session-level logs with user/visitor ID, timestamps, pages, queries, and key events (“Mark solved,” scroll depth ≥75%).
   • From CRM/CCaaS (e.g., Salesforce, Zendesk, Genesys), export cases/chats/calls with timestamps, channel, category, account/user IDs.
   • Pull KB metadata from your CMS/KMS (e.g., ServiceNow, Confluence, Salesforce Knowledge).
3. Link journeys:
   • Join sessions to contacts using hashed user ID where authenticated; otherwise match on cookie/visitor ID and time proximity (e.g., contact within 24–72 hours from same device/IP).
   • Exclude internal traffic and bots. Deduplicate multiple sessions from the same user within a short window.
4. Define outcome labels:
   • True deflected session: eligible session with evidence of engagement (e.g., ≥30s dwell or scroll ≥50% or “Mark solved”) and no assisted contact within the window.
   • Assisted contained: session with engagement followed by assisted contact; attribute as “not deflected.”
   • Non-engaged: bounces or quick exits; treat as “not deflected.”
5. Calculate core metrics:
   • Session-based deflection rate = true deflected sessions ÷ eligible sessions.
   • User-based deflection rate = users with at least one true deflected session ÷ users with eligible sessions.
   • Article-level deflection contribution = true deflected sessions where article X was the last engaged item ÷ eligible sessions that viewed article X.
6. Establish a counterfactual (optional but recommended):
   • Baseline contact propensity = contact rate among similar users who did not view KB (e.g., searched but clicked “contact us” directly).
   • Propensity-adjusted deflection = (baseline contact propensity − observed contact rate post-KB) × eligible sessions.
   • Run A/B tests: suppress KB for a small control or randomize search ranking to estimate causal lift.
7. Segment the results:
   • By content taxonomy, query intent, product/version, severity, channel entry (SEO vs. in-app vs. portal), device, language/region, customer segment (consumer vs. enterprise).
   • Distinguish logged-in vs. anonymous users.
8. Measure quality and risk:
   • Compute post-KB CSAT and compare to assisted CSAT; monitor subsequent contacts within 7–14 days to catch “false deflection” (silent churn or unresolved issues).
   • Track reopen rates and complaint flags for sessions marked “deflected.”
9. Translate to cost impact:
   • Estimated cost savings = true deflected contacts × weighted cost-per-contact (mix by channel avoided).
   • Include AHT avoided for time-based productivity benefits.
10. Trend and benchmark:
    • Create weekly time series to control for seasonality and releases/incidents.
    • Compare to internal benchmarks (top quartile articles/markets) and any available external references.
11. Validate and sensitivity test:
    • Vary engagement thresholds (e.g., 20s vs. 60s dwell) and windows (24 vs. 72 hours) to see stability of the metric.
    • Spot-check samples by contacting customers or reviewing transcripts where policy allows.

Format of the output of analysis:

• Executive summary page with overall deflection rate, assisted vs. true deflection, and estimated monthly cost savings.
• Funnel view: eligible sessions → engaged → true deflected → contacted.
• Time-series charts of deflection rate and volume, overlaid with releases/incidents.
• Article leaderboard: views, engagement, article-level deflection contribution, CSAT, and subsequent contact rate.
• Search analytics: top queries, zero-result searches, deflection by query intent.
• Segmented comparisons: by device, channel entry, product, region/language.
• Quality guardrails: post-KB CSAT vs. assisted CSAT, false-deflection rates, reopen rates.
• Slide appendix detailing methodology, definitions, and sensitivity analyses.

How to interpret results:

• Higher true deflection is positive if paired with stable or improving CSAT and low false-deflection; otherwise it may mask unresolved issues.
• Low deflection suggests content gaps, poor findability, or misrouted intent (customers default to contact channels).
• Large variance by segment indicates targeted opportunities: e.g., strong SEO deflection but weak in-app indicates a UI placement issue rather than content quality.
• High engagement but low deflection on specific articles implies clarity, completeness, or actionability issues; examine time-on-page and scroll depth alongside “Mark solved.”
• Benchmark comparisons should be directional; prioritize improvement where you underperform internal top quartile peers or products with similar complexity.
• Trends: steady improvement following content releases validates impact; volatility often correlates with incidents or product changes—separate “noise” from structural performance.

Steps a company can take to improve on this measure:

• Content and process (KCS-aligned):
  • Prioritize articles for top contact drivers and zero-result searches; publish within SLA after new issue emergence.
  • Improve actionability: step-by-step troubleshooting, screenshots/videos, decision trees, clear eligibility tables (e.g., for refunds/returns).
  • Localize and maintain: translation quality, region-specific policies, scheduled review cadences.
• Findability and journey design:
  • Optimize site search relevance (synonyms, typo tolerance, learning-to-rank) and facet filters.
  • Embed contextual help and surfaced articles in-product at point of friction; add smart prompts before “Contact us.”
  • Create clear escalation paths; ensure customers can still reach assisted support to avoid frustration.
• Systems and data:
  • Strengthen analytics instrumentation (events for “solved,” scroll, copy, downloads) and identity resolution across web/app/CRM.
  • Implement recommendations (“related articles”) and personalization by product, customer profile, and prior behavior.
  • Automate dashboards and alerting for sudden drops in deflection or spikes in false deflection.
• Capability building and governance:
  • Train authors and agents on KCS practices; enable agents to capture knowledge during case resolution.
  • Establish editorial standards, SEO hygiene, and ownership for each article with review SLAs.
• Product and policy fixes:
  • Address root causes behind top assisted contacts (e.g., confusing flows, unclear policies); fix the product so fewer articles are needed.
  • For “high-engagement/low-deflection” topics, simplify processes (e.g., self-serve refund requests) to enable true resolution in-channel.
• Experimentation:
  • A/B test article layouts, search ranking, and in-app prompts; measure deflection and CSAT impact.
  • Example: if mobile sessions show high bounce and low deflection, optimize page speed, shorten paragraphs, and add collapsible sections; re-test.

Benchmark comparisons:

General benchmarks:

• Overall true deflection from KB/self-service commonly ranges 10–30% of potential contacts in mixed-complexity environments.
• High-performing programs achieve 30–50% true deflection on simple/top inquiries with mature content, strong search, and in-product surfacing.

Segment- or industry-specific benchmarks:

• Simple tasks (password reset, order status): 60–90% deflection when self-serve is embedded and reliable.
• Transactional policy queries (returns, billing): 25–50% depending on policy complexity and eligibility clarity.
• Technical troubleshooting (multi-step): 10–25%, higher if integrated diagnostics are available.
• In-app contextual help typically outperforms SEO/portal by 5–15 pts due to precise intent capture.

Building internal benchmarks when externals are limited:

• Track top quartile articles by deflection contribution and set targets for laggards.
• Compare adjacent products or markets with similar complexity; aim to close the gap over defined sprints.
• Use pre/post comparisons around content releases or product changes with control groups where feasible.