Building bots and earning Brogrammer points

Building bots and earning Brogrammer points

Disclaimer: This article was written off the back of a presentation given to the team at W12. The eggs are real, the brogrammer sentiment was tongue in cheek.
At W12 Studios, we take two things very seriously: the first is our work, the second is breakfast. Within the prototyping team, that means only one thing.


That’s right, eggs

A few months ago I purchased the Wahl ZX642, a delightful little device with the capacity to boil 7 eggs at a time to perfection. All was well, and a small circle of us enjoyed delicious eggs every morning for breakfast. However, soon word spread throughout the land and more and more people wanted in on the action.

Thankfully, a solution was at hand. We purchased an upgrade to the little egg cooker that came in the form of the VonShef Premium. Our capacity to boil eggs increased from 7 to 16, and for a short while all was well once again.

However, as you may know, we developers are a lazy bunch and the thought of turning on and off an appliance by hand, day in and day out is just too ridiculous. There was only one thing for it.

 

eggbot 1.0

We use Slack for all of our internal communication, and we figured that being able to talk to our egg maker was the logical next step. With the addition of a Belkin Wemo switch, there would be no need to visit a website, download a new app or even visit work in order to start cooking eggs of the future. In addition, it provided us with a natural platform for user interaction and feedback.

 

+10 Brogrammer points

As all you brogrammers know, it’s very important to select the technologies you use not based upon the most suitable for the job, but based on how cool they are.

At W12 Studios we have recently started using React as our primary method for creating prototypes. My colleagues have written articles about the reasons we are moving forward with React, citing the benefits of the virtual DOM, componentisation, the ability to share code bases within the team and the fact that our clients are also starting to adopt React. Do not be fooled, as our true reason is the same as everyone else’s. React is cool and we want those points bro!

My choice of tech for the eggbot (Slack, Nodejs, SQLite, Wit.ai and Belkin Wemo)

My choice of tech for the eggbot (Slack, Nodejs, SQLite, Wit.ai and Belkin Wemo)

As you can see from above, my choices of tech are pretty cool, with the exception of SQLite, as I’m sure you all know the obvious choice was MongoDB. -5 Brogrammer points for that.

 

Connecting the dots

eggbot’s sequence of events and triggers

eggbot’s sequence of events and triggers

Setting up the slack bot was straight forward enough. I won’t go into too much detail, in fact, I won’t go into any detail whatsoever around this. I will however, recommend this tutorial to get you going. Instead, I’m going to focus on one of the more interesting components of the eggbot setup: Wit.ai.

Wit.ai is a natural speech processing API that takes an input of either audio or text. It then extracts the intent and any entities within the input. Though it might seem slightly unnecessary in the scope of this bot, remember that it’s all about using the latest and coolest tech possible.

Example from wit.ai of how to use the API

Example from wit.ai of how to use the API

The above example is taken from Wit.ai and shows the steps for a beer robot (possibly the only thing better than an eggbot, although maybe not suitable for the morning… but there was this one time). The API processes the text or voice input from a user and returns a set of insights back. In the example above the intent of the user’s input is to get an item for the user who asked. The item is a beer and it also checks if the user was polite, which in this case they were not.

The intents and entities I needed to teach my bot are very similar to the beer example above. Here are two of mine:

getting the amount of water needed based on the number of eggs

getting the amount of water needed based on the number of eggs

triggering the bot to start cooking for a set period of time, depending on the required amount of eggs.

triggering the bot to start cooking for a set period of time, depending on the required amount of eggs.

Wit.ai makes it easy to extract the correct data from the text input. From there, it was just a case of querying the database and connecting the Wemo remote plug.

 

Test run

After a little bit of tinkering and once I’d configured my Wemo remote with my network I was ready to see if it would all work.
Wemo plugged in, node server fired up, I jumped onto Slack. Eggbot was there and online, just awaiting my commands.

Success!

Success!

The bot worked! It took my command, sent it on to Wit.ai, got back a valid response, loaded the relevant information on cooking time from the database, sent me a message to say when my eggs would be done and turned on the egg cooker. Then, just as indicated, after 8 minutes it switched off the cooker and let me know they were ready.

All seemed to be good, so the next day I took the wifi plug into the office, set up a private Slack group so me and the rest of the prototyping team could give it a test run.

 

Along comes Alex, a.k.a. WolfDog

Thankfully I had built in a command to just switch off eggbot, but it was too late… Dan was already sad.

eggbot in the real world

eggbot in the real world

 

I totally underestimated the user’s love of destruction. I gave no one on the team any instructions on how to use the bot beyond what it did. For whatever reason Alex felt like the bot should have been built with an attack function. Maybe it should have been, for now, it does feel like feature bloat.

Nevertheless, this does highlight the importance of user testing. Be it guerilla testing with people in the studio, at the local coffee shop or within a lab, you cannot predict exactly how your users will interact with your designs. In addition, after spending weeks (sometimes months) working on a project, it’s easy to become too close to it and too comfortable with how you think it should be used. Just as I entered in one of the control questions and chalked that up as a win, Alex immediately tried the first thing that came to his head just to see how it might work.

So it just goes to show whether you’re picking someone up in a bar, or giving a presentation, or building a bot to cook perfect soft boiled eggs…

 

It’s all about confidence

Not the level of confidence with which eggbot delivers Slack messages, but how confident Wit.ai was that it got the intent and entities correct.

Here is the response based upon one of my training statements for Wit.ai:

Yes, this is the theme I have on my terminal

Yes, this is the theme I have on my terminal

It successfully registered my intent and extracted the number of eggs and preference for how they should be cooked. The confidence level though is still only 0.609. So it’s quite easy to see how it could register ‘eggbot -attack’ to be a valid command. One solution might be to strip out ‘eggbot’ from the start of the message, but for now, a simpler all be it hackier solution…

if (confidence < 0.6) then…

Now if someone from my team tries to send in a command that eggbot does not understand:

You might be thinking that there is no need for language like that, and of course, you are wrong. Nobody wants an overly polite robot winding them up with a ridiculous level of etiquette. If you don’t believe me, imagine living with this:

Natural speech is becoming more of a common interaction pattern and one we are seeing more frequently across all our devices and in our project work. The way in which we design for these type of interactions is becoming more and more relevant: How do we teach users what commands are available? What do we do if they try a command our system does not accept? How do we show which inputs were accepted and the effect it has had on what is displayed back to the user? This is a field we have started to explore in depth over the last few months, and examples such as this are where the rubber hits the road.

 

Eggbot V2 roadmap

Obviously, eggbot is on an iterative, sprint and development cycle accounting for user feedback whilst allowing us to integrate new features, without taking the system offline, awarding an extra +20 bonus brogrammer points.

A few things I’d like to play around with next are:
Automatically adding the water into the cooker.

An egg loading system so we can eat delicious eggs without any effort.
Integrating the HomeKit API to enable iPhone and iWatch control.

 

Key Takeaways

Be sure to earn your brogrammer points
Never underestimate the user’s desire to break things
Ensure that your confidence levels are high
Most importantly, always teach your robots to swear