UX Design
Speak to Pick
Inventor, Lead Interaction Designer
I developed a new method for interacting with smart home devices using voice and gestures through a smartwatch. This innovation, for which I am a co-author of four European patent applications, was showcased at the Orange Research exhibition in Paris.
Background
Voice assistants gained significant popularity, leading major companies to integrate them into their devices and services, with options like Siri, Alexa, Google Assistant, Cortana, and Bixby becoming ubiquitous. However, the accuracy of these assistants in understanding human commands proved to be far from perfect. Users frequently experienced frustration as their spoken commands were misunderstood, leading to repeated attempts and tiring follow-ups to correct errors. This persistent issue of miscommunication made the experience more frustrating than helpful for many users.
47%
questions are answered correctly by Siri
Design Implementation Leading to Four European Patent Filings
COULD YOU
THE
IN
?
302.2
302.1
302.3
User Input processing for controlling remote devices
Europe EP15306051.2 Issued June 30, 2015
European Patent Application for Human interaction with remote devices (Internet of Things)
Enriched instructions for remotely controlling electronic devices
Europe EP15306052.0 Issued June 30, 2015
European Patent Application for Human interaction with electronic devices such as laptops, touchpads,
mobile phones, Smartphones, wearable devices or smart watches.
Remote control of an electronic device with a selectable element
Europe EP15306045.4 Issued June 30, 2015
European Patent Application for a method of managing a remote electronic device using an electronic device,
a database associating argument types with alternative argument values and comprising
a compatibility mesh between the alternative argument values for a given type and contextual data.
Updating of former instructions to control a remote electronic device
Europe EP15306050.4 Issued June 30, 2015
European Patent Application for Human interaction with remote devices (Home Automation, Internet of Things)
Say what you need
Turn on the light
Your command is displayed
It is translated into pictures
Press “do it” and it’s done :)
Solution
I designed Smart assistant interface on a smartwatch that shows user commands in real life as they speak. The text is then translated into visual symbols in split seconds.
This allows user to clearly see if the command was understood correctly, and fine tune the settings for the devices they want to control.
Prototype
Fully functional apple watch app integrated with selected Smart Home devices