E-commerce
June 28, 2026
When deploying an AI chatbot on Shopify, many brands aim to "automate everything" from day 1. The typical result: a bot that handles complex cases poorly, frustrated customers, a disabled widget, and an internal conclusion that "AI doesn't work."
The right approach: choose which questions to automate first based on volume, repetitiveness, data availability, and risk level. WISMO queries often represent 40 to 60% of support tickets on high-volume stores (Canary, e-commerce AI chatbot 2026).
This guide #120 offers a scoring method, an impact/effort matrix, a 90-day roadmap, and validation criteria by intent. Distinct from reducing AI tickets (the why/how) and chatbot limits (#124) (what not to automate): here is what to automate first, and in what order.
Summary
Why is it necessary to prioritize the questions to be automated?
Prioritizing chatbot questions determines the success or failure of the AI deployment.
Why the order matters
Early trust: the first conversations shape the bot's perception
Immature corpus: minimal reliable data at launch
Limited resources: little time to configure 50 intents
Controlled escalation: human available for the rest
Internal ROI: quick wins prove the value to management
Cost of poor prioritization
Automating a dispute, health advice, or a vague promotion worsens the conflict, generates hallucinations, or policy errors. Too many intents on day 1 = impossible maintenance and a "useless bot" narrative.
Objective of the method
Identify 5 to 10 phase 1 questions with an objective score, then expand in waves validated by KPIs. Repetitive support first (clear ROI), assisted conversion in phase 2.
What mistake should be avoided when launching the chatbot?
The error of automating everything at once kills most e-commerce chatbot projects.
Big bang symptoms
50+ intents day 1: none tested in depth
Low deflection: bot does not understand formulations
CSAT bot < 3.5: customers bypass the widget
80%+ escalation: bot passes the hot potato
Progressive 3-wave approach
Phase 1 (weeks 1-4): 5 high-confidence intents
Phase 2 (months 2-3): 5-8 medium-impact intents + PDP pre-purchase
Phase 3 (months 4+): fill-in intents and sensitive guardrails
80/20 Principle
20% of intents often account for 60 to 80% of volume. Heeya estimates that a document-grounded bot can handle 50 to 70% of incoming tickets if deployment is progressive (Heeya, customer service automation 2026).
Concrete example
Beauty brand, 40 intents day 1: CSAT 2.8 week 2, widget disabled. Restarted with 5 phased intents: CSAT 4.3 month 2. Three months of a poorly configured bot cost more than three weeks of targeted wins.
How to score each question based on 4 criteria?
The scoring method objectively ranks each candidate question.
4 criteria rated from 1 to 5
Volume: ticket frequency/month
Repetitiveness: standardizable answer without judgment
Data: information available Shopify, hub conditions, up-to-date policy
Risk: 5 = low, 1 = dispute/health/complex
Formula
Score = (Volume + Repetitiveness + Data) × Risk. Maximum 75. Target phase 1: score > 50.
WISMO example (priority #1)
Volume 5, Repetitiveness 5, Data 5 (Shopify API + carrier), Risk 5 → score 75.
Refund dispute example (human only)
Volume 4, Repetitiveness 2, Data 3, Risk 1 → score 9. Phase 3 or permanent human.
Prioritization workshop 2 h
Google Sheet: columns intent, volume, rep, data, risk, score, phase, status. Support lead + e-commerce score top 20 together. Auto formula. Alignment avoids post go-live disputes.
How to read the impact/effort matrix?
The impact/effort matrix visualizes where to invest first.
Quadrant 1: quick wins (phase 1)
High impact, low effort: WISMO, delivery times, return policy, return portal link, contact hours, unshipped cancellation.
Quadrant 2: strategic (phase 2)
High impact, medium effort: size guide, variant stock, promo conditions, shipping cost, product specs.
Quadrant 3: fill-in (phase 3)
Low impact, low effort: brand story, gift wrapping, loyalty points.
Quadrant 4: avoid early
Disputes, policy exceptions, regulated products, B2B wholesale, press, fraud. See red list section 6.
Estimate impact and effort
Impact = tickets/month × resolution time × agent cost. Example WISMO: 100 tickets × 8 min = 13 hrs/month saved. Effort = corpus setup hours + testing + maintenance. WISMO ~4 hrs; complex size guide ~20 hrs.
See tag conversations.
Which 10 questions should you automate first on Shopify?
On most DTC stores, these phase 1 intents concentrate the ROI.
Where is my order (WISMO + tracking)
Standard delivery time by zone
How to return (portal link)
Return policy timeframes and conditions
Modify address before shipping
Cancel unshipped order
Support hours and response times
Active promo code and conditions
Talk to an agent (intelligent routing)
Pending refund status
Prerequisites per intent
Validated macro: approved support lead answer
Sync data: Shopify order API or static conditions hub
20 tests: customer phrasing variations
Handoff rule: confidence < threshold or anger detected
Baseline volume: measured before automation
Vertical adjustments
Fashion: size guide phase 2. Electronics: compatibility phase 2. Food: allergens in metafield phase 2, never generative health phase 1. BotHero recommends starting with static information and then structured lookups (order number) (BotHero, prioritization sequence 2026).
Which questions should remain human-handled in phase 1?
Certain requests should not enter Phase 1, even if the volume is high.
Phase 1 Red List
Disputes and threats: chargeback, lawyer, press
Policy exceptions: late return, commercial gesture
Health advice: pregnancy cosmetics, supplements
Complex product: specifications without a solid corpus
Fraud and VIP: suspicious order, influencer
Checkout bug: immediate dev escalation
Useful partial automation
The bot can collect order number, defect photos, dispute tag, then route to a senior. Workflow automation, not resolution.
When to re-evaluate
Phase 3+ with confidence scoring, validated regulatory corpus. Some cases remain permanently human.
How to analyze your tickets to make a decision?
Analyzing your tickets replaces intuition with data-driven priorities.
90-day export procedure
Export Gorgias tags and subjects
Group by intent (manual or NLP clustering)
Count volume per intent
Score each intent (4 criteria)
Sort by descending score
Select top 5-10 for phase 1
Without ticket history
Vertical benchmark, competitor terms hub, top 20 questions the founder receives most, mystery shopping the entire journey. AeroChat recommends listing the top 20 questions before any deployment (AeroChat, Shopify automation 2026).
Complete with pilot bot logs
Widget analytics: sending pages, unmatched logs. Systematic tagging 2 weeks before automation invests in scoring quality.
Seasonal re-prioritization
Pre-BFCM: promo and shipping intents rise. Re-run scoring at D-30. See segmenting funnel (#117).
What is the 90-day roadmap for a phased deployment?
The 90-day roadmap structures the deployment without overloading the team.
Phase 1 weeks 1-4: foundations
Scoring of the top 50 questions, selection of 5 intents
Sync of the conditions + policies corpus hub
Intent configuration + 20 tests each
Gorgias handoff with transcript
Go-live on customer account + footer
KPI measurement vs. week 4 baseline
Phase 2 months 2-3: extension
Add 5-8 quadrant 2 intents. PDP pre-purchase (size, stock). Checkout shipping widget. Weekly unmatched review. Do not reduce agents until deflection is proven.
Phase 3 months 4+: optimization
Fill-in intents, proactive funnel messaging, assisted conversion, multilingual if Markets.
Gate between phases
Proceed to phase 2 if: phase 1 deflection > 40%, bot CSAT > 4.0, audit accuracy > 95%, unmatched decreasing. Widget placement: phase 1 customer account, phase 2 PDP + checkout, phase 3 proactive.
How do I validate an intent before adding another one?
Each live intent must pass a rigorous validation before the next one.
7 go-live intent criteria
20 test questions: 90%+ correct
5 documented edge cases + handoff
Linked macro source of truth
Confidence threshold configured
Escalation rule tested
Agents informed of live intent
Intent KPI tracking activated
14-day observation
Daily review W1-W2, log unmatched, CSAT segmented by intent. Correct before adding the next one.
Rollback rule
Intent CSAT < 3.5 or accuracy < 90% in week 2: deactivate, fix corpus, re-test.
Notion intent card
Trigger phrases, standard response, data source, handoff conditions, owner, live date. 5 loyal customers test 10 questions before public go-live.
Which KPIs prove good prioritization?
The prioritization KPIs validate that the correct order has been chosen.
Phase 1 KPIs
Live intents deflection: target 50-70%
Bot CSAT: target 4.0+
Unmatched rate: < 15% decreasing
Justified handoffs: 95%+ correct
Intent volume: -30% vs 30-day baseline
KPIs per intent
Volume handled, deflection %, CSAT, 7-day recontact, accuracy audit of 20 conversations/month.
90-day North Star
15 live intents, 55% deflection, 4.2+ CSAT, -25% ticket volume with constant headcount. Cost per deflected ticket = bot license / deflected tickets (unit economics).
Quarterly re-scoring
New product, promo, market: top intents evolve. Re-score every quarter.
See Chatbot KPIs (#11).
How does Qstomy help prioritize intents?
Qstomy structures the prioritization and phased rollout of chatbot intents.
Features
Ticket analysis: auto suggestion of top intents
Scorecard: template for volume, repetitiveness, data, risk
Rollout by waves: activation intent by intent
Unmatched dashboard: next intent to build
KPI per intent: deflection, CSAT, accuracy
Gate alerts: phase 2 blocked if KPI below threshold
Quantified DTC Scenario
Decor store 350 tickets/month. 90-day analysis: WISMO 28%, returns 18%, shipping 12%. Phase 1: 5 intents week 4. Deflection 52%, CSAT 4.2. Phase 2: size + stock week 8. Tickets -38% vs baseline, ROI proven to management.
7-Day Prioritization Setup
Connect Gorgias export
Intent scoring report
Phase 1 selection validated
Corpus per intent
KPI gate tests
Go-live monitoring
Explore AI support, AI sales agent, Shopify, request a demo. See DTC playbook (#118).
Which operational playbooks should be launched this week?
Playbook 1: 90-day ticket audit
Gorgias export, top 20 topics cluster, count volume. Timeframe 3 hrs. Basis of scoring.
Playbook 2: top 30 scoring sheet
Duplicate template with 4 criteria. Score in support + e-commerce workshop. Select top 5 phase 1. Timeframe 2 hrs.
Playbook 3: launch 3 quick wins
WISMO + return policy + delivery times only. 20 tests each. Tested handoff. Go-live customer account. Timeframe 1 week.
Playbook 4: week 4 gate
Measure deflection, CSAT, unmatched. If deflection > 40% and CSAT > 4.0: add 2 quadrant 2 intents. Otherwise: fix corpus, no extension.
Playbook 5: Notion intent sheet
Template: triggers, response, source, handoff, owner, KPI. One sheet per live intent. Quarterly re-scoring. Timeframe 30 min/intent.
Useful links
Automating intelligently means refusing to automate what is not ready. A perfect bot for WISMO is worth more than an average bot for everything.
Enzo
June 28, 2026
