DRU Assist™
iOS | Android | Web
Partnering with Nuance, I designed the interface, character, animations and conversational structure for this early, yet extremely advanced chatbot.
Research
In 2016, Nuance approached Domino’s with the idea of integrating their Nina chatbot technology with our online ordering platform. At the time, Siri intents were still 18 months away and Google Assistant was nascent, so the decision was made to adapt Nina into Domino’s own, online voice ordering assistant. At this time Nina had seen some deployment at the Australian Tax Office and Virgin Airlines, but had never been integrated as a full screen native experience on mobile - and had never been used for a product as complex as fully-customisable pizza.
What would emerge from this project was a product of its time - both in its scope and, ultimately - limitations, but the ambitions and accomplishment of DRU Assist renders it beyond a toy or a gimmick - an impressive, but ultimately flawed halo product
A Feature Maximalist
The scope for this project was huge and perhaps untempered:
DRU Assist would be able to select a service method (carryout or delivery), the selected store (if carryout) the correct address (if delivery), the time for the order (ASAP or a specific time and date) and then an order of as many pizzas (theoretically limited at 20) with as many customisations of crust, sauce, ingredients as the user wanted, calling from a menu with both recipes and the ability to create a completely custom pizza - with rules around maximum numbers of toppings to prevent undercooked product, and select sides, and select drinks, and select desserts, then propose additional items, then apply any voucher the user may have, then add the order to the user’s basket and allow the user to check out.
Additionally, the experience should be able to be launched at any point in a traditional web or app user flow, pick up that user’s basket as-is, and be able to customise, change and add its content.
On top of this, DRU Assist was expected to have a personality - the marketing team was fixated on Siri’s cheeky responses to certain questions and felt these “pushing the limits” moments were critical to have in the product from day one. DRU was to be able to make jokes and comedic responses at any point in the ordering flow.
Further, DRU Assist would be able to accept typed and spoken input, with low-latency speech recognition, and TTS spoken responses.
This was the MVP scope as determined by the global leadership team, and with Nuance’s assurances, we committed to an 8 month development timeframe.
Working alongside an onsite team from Nuance and a DPE product team, we exhaustively mapped the customer journey tree, workshopped synonyms, put together a blocklist of unacceptable words while I started work on UI.
Biweekly meetings with the US team assured us that backend performance enhancements and the activation of local server infrastructure would reduce latency and improve the quality of speech recognition.
Design
DRU Assist was an extremely hush-hush project, so user testing was conducted almost entirely internally, with long afternoons of sit-and-test sessions set up in the head office break room. Early mockups were pulled together in Keynote based on Sketch files - this let us simulate the conversation with audio files generated by TTS on MacOS.
Conceptual wireframes of the design prototyped putting DRU at the top of the screen and having the conversation flow down towards the user, but this was abandoned in favour of a scrollable chat history - ostensibly to allow the user to troubleshoot if something went wrong in the speech or text recognition. The user interface was originally prototyped in Domino’s red-and-blue, this was changed to yellow and white - both red and blue deemed to emotive for a conversation flow.
As we began to user test with the Nina chatbot plugged in to the interface, we began to see substantial lag and some challenges with speech recognition accuracy. To compensate for this, I added inline buttons for every decision that DRU Assist presented - so rather than say or type “yes” or “no” the user could quickly tap to confirm. This also gave the customer a quick and clear button to hit if we got it wrong.
Where possible, we used system fonts to appear platform neutral, with limited appearances of Domino’s brand fonts, which were less legible in smaller sizes.
Putting a face to the name
The DRU character itself was designed to be sleek, modern, expressive and fun - a close sibling to an autonomous delivery robot the business had prototyped a year prior.
After playing with more organic forms I settled on a three element design, featuring the white shell, black front panel and big, expressive blue eyes. Moving the shell and front panel separately gave the 2D sprite a sense of dimensionality, and this movement combined with the ability to tilt left and right captured the expressive friendliness that brought DRU to life. We wanted to keep an element of the robotic to the design - to set clear user expectations and not over promise on interactivity. Rather than attempt lip-sync at thumbnail size, DRU’s responses - listening, thinking, speaking - would map to the illumination of its eyes.
To bring DRU beyond the app and web experience, the existing vector assets from Sketch were upscaled and imported into Adobe Character Animator (then a very early beta). These were then rigged and mapped onto a facial animation performance system, allowing an actor to drive the position of DRU, with the front and back planes of the face on a spring system, allowing one to lead the other - maintaining the three dimensional effect. The actor’s vocal performance was mapped to the illumination of the eyes through and A/B alpha channel - the relative simplicity of this allowed us to keep a very high frame rate for live performance.
Though initially planned to create bumpers for TVC, executives were so impressed with DRU’s performance that (against a new chroma green background) the digital puppet co-hosted the 2017 Domino’s Rally, the first use of Adobe Character Animator in a live stage event. The DRU digital puppet would go on to feature in TVC, social content and a Youtube video series.
Deployment
Stress Test
DRU Assist was launched with a live stage demo at Domino’s Abacus 2017 technology event.
Having been onsite for the 24 hours preceding, stress testing venue connectivity, troubleshooting the thousands of paths a demo can take that end in news-making and embarrassing mistakes - I had worked with Nuance to craft a script that (connectivity, comprehension and local weather prevailing) DRU Assist could complete successfully, repeatably.
Two hours before the demo, with the media filing in, DPE’s marketing team decided that the pizzas for the order DRU was to place would be selected at random from suggestions by the audience. This dramatically increased the opportunity for failure for a product that was - on its best day - in late beta state, and not due to be released for another month. The moment arrived, our global CEO took the stage and compared what we had accomplished to Siri and Google Assistant, and handed over to our team for the demo.
The audience was polled and came up with a Vegorama, no capsicum, on a thin crust. There was a brief pause while DRU Assist thought about the instruction, and then successfully identified and added the pizza to the order. Enormous relief for myself, the team of Nuance engineers backstage, and Domino’s.
Legacy
DRU Assist captured the imagination of the Australian news media, and was featured in QSR articles, radio, television and YouTube, in addition to appearing on full-colour print pizza boxes.
I had my brief moment of fame as the user in the recorded demo sent out in the press kit.
DRU Assist would continue to take orders (though, ironically, our analytics showed mostly typed) as a web and app integration until early 2019, when the groundwork for a new app and website meant the retirement of features that weren’t critical to the ordering flow.
It remains a strikingly feature-complete and technically impressive artefact of its moment in time, an exciting moonshot project that I am proud to have been part of.