I used insights from users to create voice prototypes with Voiceflow, and screen prototypes with Origami Studio with which I had different kinds (pro, casual, non-) users of voice assistant technology interact.
Then I refined the interactions through co-creation suggestions and based on their feedback. What you see in these videos is a demo of those prototypes, from which the design principles have been extrapolated
Not just manage them. Users only know what you show them. There’s no hover state for interactions with voice assistants. It's up to the designers to not disappoint users by setting the bar too high
Seen as an improvement
People who didn't use VAs thought it was super normal
Frequent users thought it was a good solution and made them more curious as to what else they could do with it
“It's good that it tell you that it has trouble and lets you train it. Sometimes it doesn't work for names I say all the time"
Totally technically feasible! This is a design decision.
What Happened in the Video?
The user is guided through an onboarding process, where she is given options for voices her assistant will use, instead of a default voice chosen for her. Google Assistant currently has a similar flow
The screens show that this voice assistant improves over time and is teachable
The user gets to try out her first command which is to teach the assistant how to pronounce a name in her contacts - this shows that the assistant can actually learn, and become more adapted to the user over time
This kind of local accent training enables your assistant to work for you specfically. Since this is in a home context and the people who communicate most frequently with the assistant are the limited people who live there, the training info could theoretically stay local.
UX of AI Principles - Set the Right Expectations, Build Trust Over Time
These systems are wildly complex and it's amazing that they work at all. However, the end result is sometimes explainable, and it helps people become less frustrated to know what's happening.
Designing to make AI understandable is a challenge. But not understanding AI can pose real world challenges in much more high stakes ways. This is a start.
People felt more forgiving of the mistakes because they understood where the assistant was coming from
“It's important that it doesn't do this too often. I'm paying money for a top of the the line machine and it’s only okay if does this in the beginning”
Totally technically feasible! This is what's going on behind the scenes with NLP decisions anyway.
What Happened in the Video?
Assistants makes mistakes. It happens. But in this case, instead of going with the first most probable answer, the assistant asks for clarification. This doesn't need to happen the first or second time an error occurs, but could be useful if the assistant detects repeated queries in a short span of time with minor differences. It's more proactive.
What kind of clarification you ask? During Natural Language Processing, a confidence score between 0 and 1 is assigned to a part of your speech that has been converted to text. In theory, and in some cases, if an input has two possible interpretations within a margin of error, the assistant could present both as options and ask users to choose.
Communicating uncertainty helps users get into the mind of the assistant and helps them empathise with it in a way. In my experiments, it made them more forgiving of the assistant's mistakes, at least in moderation
UX of AI Principles - Communicate your Confidence, Explain Your Results
Technology can evolve to solve a lot of our problems, but in this case we know how to help ourselves. Virtual assistants should encourage their own training to help users, in order to build trust over time
Another insight into how the system things
Allows you to correct it and learn how it works at the same time
Users were relieved to know they can intervene
Time to prompt is important
“It needs to prompt me only when something goes wrong, not when/if it actually did understand"
Possible, but not advisable. Giving users free reign to make the voice assistant potentially say whatever they'd like could be exploited. However, there are workarounds, such as listing the top 3 guess and asking users to pick the one that's correct (or none at all).
What Happened in the Video?
The assistant misunderstood a common command and went on a rant.
The user was able to intervene and correct the system's understanding so that in future, the mistake was less like to occur.
UX of AI Principle - Keep the User in Control
Virtual assistants only need to be human enough to make them easier to interact with, but not deceive or mislead users (through their own speech or through outbound communication and marketing material) about their nature. This only leads to misunderstanding and frustration.
It can do this through eschewing defaults and giving users options that they may not realise exist.
We'll have to see. Voice assistants are surprisingly young for how ubiquitous they are. It's more of a long term thing.
“That’s what a person would say/how I would ask someone if I didn't understand“
That’s up to the makers of this tech
What Does That Even Mean?
Voice assistant technology is very young. It's understandable why we resort to anthropomorphising it - metaphors are how we process the world. Before flat design was a thing, skeuomorphism ruled the screen. What does flat design for voice assistants look and sound like?
Voice assistants are arguably the most human way we interact with AI. However, to apply human properties to voice assistants is a one-two punch to societal and technological progress: it minimises the richness of the human experience, as well as trivialises the immense potential of AI.
UX of AI Principles - Escape The Personality Cult, Help Your Users Grow