A featured contribution from Leadership Perspectives, a curated forum for enterprise technology leaders, nominated by our subscribers and vetted by the CIOApplications Editorial Board.
Director of Product Management at Perfecto
Three Keys for Assuring Quality in Voice Chatbot-Based Apps
Recently, Google Assistant’s product manager Valerie Nygaard purchased lunch from Panera completely hands free – "Super easy, like I was talking to someone at a store," she said to an impressed demo audience. Chatbots use messaging to carry out; they respond to text and voice questions with contextual and actionable information.
Meanwhile, digital transformation in financial services and retail prompted Tangerine CIO Charaka Kithulegoda to say, “Our mobile app is the Bank.” Consumers now expect, at their convenience, interaction with a human-like attendant to achieve a meaningful goal. It’s called the new Application-to-Person (A2P) model.
Leveraging onboard (mobile device) sensors to drive innovation and user engagement, voice interfaces have matured in quality to meet end-user expectations
Chatbots- Best of Breed Touch Point
This new form of interaction is gaining tremendous traction due to its ease of use and ubiquity. Businesses leverage their own applications or use open messaging platforms such as Whatsapp, Facebook, Google Assistant, amongst other social media platforms.
Chatbots pack a new level of targeting, convenience, and “gamification” for the user. Hipmunk consolidates 50 travel vendors, “Optimizing for reduced agony in flight choices” as they succinctly put it. Cently applies retailers’ promo codes automatically on check out.
Consumers have an appetite for the seamless conversational experience – 70-80percent of users answered three chatbot surveys to get a coupon – that’s 300+percent compared to traditional email surveys.
Voice-First Interaction
Leveraging onboard (mobile device) sensors to drive innovation and user engagement, voice interfaces have matured in quality to meet end-user expectations. Significant players (Apple’s Siri, Microsoft’s Cortana, Amazon’s Alexa amongst other personal assistants ) respond to voice signals, whether stand-alone ( Google Home and Amazon Echo) or inside other applications. It’s very easy to order pizza or book travel from an app or the likes of Google Assistant and Alexa.
For the Business
Benefits for business are numerous, including the ability to learn and become more conversational with the end user. Chatbots enhance call centers by using human-like attendants. Live chat has the highest satisfaction levels for any customer service channel (73 percent), compared with 61 perceent for email and 44 percent for phone. Canada’s Rogers reported a 60 percent improvement in customer satisfaction following the deployment of a Messenger customer service bot.
Figure 1:Source: BoldChat Survey
Translated into $, this is a significant opportunity for cross-selling and cost saving (the average cost of a customer transaction via phone is around $2.50 whereas it is estimated $0.17 through a chatbot.
DBS Bank, as part of its SDG200 million investment in digital Banking, recently rolled out a conversational messaging app, enabling customers to access account balances, track expenses, and make payments without leaving the app.
Figure 2: DBS first in APAC to launch messaging banking app
Wells Fargo, Bank of America, Mastercard, AMEX and others are joining this trend.
Quality Matters as Complexity Rises and Users are Fast to Disappoint
With all these benefits, comes significant risk. User expectations for the Chatbot economy may be hard to meet. Everything new brings potential for bugs and failures:
1. Broader inputs: Chatbots will need to accommodate for broad dictionary, and in the future, add imagery inputs.
2. Languages, voice variation, accents, and stutters: Can text engines accommodate the variety of sound users will throw at them?
3. Lastly, usage of ‘millennial slang’: User interaction will inevitably use acronyms and shortcuts to accommodate fun and engaging dialog.
Ensuring Quality of Voice-First Chatbots
It will take the right set of tools and processes to ensure voice-first Chatbot quality that meets expectations:
• Extend test lab coverage to include Voice-Chatbot: The lab should enable automation test scenarios including text and speech entry (including speech imperfections), and validation of the response (both text and audio) to complete a flow. Beyond functionality, the lab should enable measurement of the Chatbot responsiveness.
• Test prioritization and scale: Growth of test cases that need to execute at the same timeframe, requires a facility to prioritize tests across resources, and parallelize the execution at scale.
• Big data reporting and fast feedback: The growth of test cases and release frequency will drive significant amounts of data to analyze. The solution need to accommodate and accelerate root cause analysis, grouping and matching of failures, and extend visibility to the entire team.
• DevOPS in mind: Agile dev teams need to extend their insight to include the app behavior in production in order to accelerate realtime issue resolution and optimize the next version of the app accordingly. Quality tools need to unify across apps pre and in-production.
Figure 3: Architecture of possible voice-chatbot test solution
Failing Fast or Getting it Right?
Voice First Chatbots are going to change the way users interact with applications. Perhaps the best thing about Chatbots is the ability to change their behavior rapidly. “A bot is only as good as the service it exposes” said Amir Shevat, head of developer relations at Slack. Brands need to experiment, plan to fail, and build the team and process such that correction is done rapidly. Agility will be the name of the game on this one.
