Project: Intelligent Voice Assistant
Client: Hisense
My Role: Product Interaction Designer
Overview
While working as a product designer on the interaction design team at Jamdeo, I had the opportunity of crafting and optimizing the experience of a voice assistant for Hisense's most prominent TV brand, the VIDAA TV. The goal was to create a seamless conversational UI user experience on a smart TV that was powered by back-end data from a semantic server in China. Driven by the importance of striking the fine balance between business goals and user needs, we had to design a seamless experience that delights users and adheres to an extensive PRD. Jamdeo assembled a highly skilled team of experts: interaction designers, visual designers, product managers, and software engineers. Together we kicked off the project with the intention of crafting a solid strategy.
Design Process
At the start of the project, various workshops took place with the design team and key stakeholders. The most important workshop we did was the one where we created user stories for key features and generated all the use cases we would design for the voice assistant. We analyzed tasks that the user would take to complete every journey, then applied those to the structure, navigation, and information architecture of the whole program to culminate in one holistic source, the Interaction Design Specification Document (IxD). As the Interaction designers sketched quickly to find the best options for each use case, visual and motion design team worked on their own proposals in tandem. This approach proved most efficient, because no activities were being blocked as we constantly kept each other updated on the latest discoveries we had, each in their own speciality.
In an official capacity, I was responsible for researching, creating wireframes, user flows, aligning with developers, to writing interaction behaviour definition for all components and patterns that existed in a use case. In collaboration with other interaction designers on my team we created the IxD document which included:
- Layout templates to streamline the use cases appearance between tasks.
- Wires of individual components that make up views with behaviour description.
- User flows and wireframes for each use case.
- Patterns that affected the system as a whole such as turn-taking, near and far-field voice interaction, and the transition between voice and d-pad.
System IA Map, Use case Flows and Wireframes
The voice assistant included many features that my colleagues and I worked on together, and constantly edited each other's work. However, when it came to creating the documentation deliverables, we split use cases and flows amongst each other.
Below, I am presenting some of my favourite use cases that I worked on:
- Asking for information on a celebrity.
- Asking for movies or shows starring a specific actor.
- Asking the electronic program guide for Live channels or programs.
Select an image to view details
Interaction Design Pattern: Voice to D-pad Handoff
Designing a conversational UI for a smart TV was a new challenge for me. Given the nature of the 10-foot experience, the user's mental model would prove to be variably different than that of other devices such as mobile, tablet, desktop and wearables. It was already different to start with because of the mode of interaction required to control TVs (D-pad navigation on a remote control device), and for this project voice activated commands where the primary mode of control we were designing for. However, users needed to have control over their experience and easily switch between voice navigation and traditional remote d-pad navigation. I worked on optimizing this handoff.
Select an image to view details
Pattern Rationale and Overview
Flow example of a view's transition from voice to d-pad
Flow explaining the transition using far-field voice interaction
UI variations between the two modes (voice and d-pad)
Defining the Components
The voice assistant views were made up of reusable components. A blue print of each component was created with the corresponding interaction behaviour defined. Building blocks such as the voice region, voice shortcuts, results region, tiles, cards, lists, toasts, tabs and ads to name a few. See some examples below.
Select an image to view details
Visual Design Result
The work dynamic between interaction and visual designers on the product team worked impeccably. We were all thoughtful strategists that excelled at planning together as well as executing. Most of the time visuals were being produced in tandem with wireframes, because as an interaction designer I would directly go to the visual designer assigned to the specific piece and whiteboard the intended behaviour so they can start the VD. Here are some of visual design examples from this project.
