Alexa AI-controlled Robot Car

Introduction

This purpose of this project is to create a robot car which can be controlled by voice. The voice control is done using the Amazon Alexa Skills Kit, and the motor control is done using Adafruit’s Raspberry Pi MotorHAT and MotorHAT python library.

Overall Architecture

There are three main sections of this project. The Raspberry Pi, AWS (Amazon Web Services), and the Motor HAT.

The Pi is used to obtain the audio input from the user as well as communicate with the motorHAT over I2C. It also can provide audio feedback, though this part is optional. The script that controls the motor’s behavior is run on the Pi and is implemented in Python.

AWS Alexa is where all the processing of voice input is done. A custom Alexa skill implemented in AWS lambda converts the voice-triggered intents into messages over the MQTT protocol. The AWS IoT message broker is used in broadcast the topic that carries the message.  The robot car Pi acts as an IoT device and subscribes to the topic and capture the message.

From an end-to-end perspective, it goes from:

  1. Human voice command to
  2. Alexa voice services on the Pi to
  3. Alexa frontend in AWS to
  4. AWS lambda where custom intents are processed to
  5. MQTT topics with intent published to
  6. MQTT broker in AWS IoT which broadcast all topics to
  7. Pi acting as a MQTT subscriber of the same topic to
  8. Pi receiving the specific messages in the topic and taking appropriate motor control actions.

Once all the processes are setup. The motorHAT will control the motors by setting direction and speed for each motor. By doing different combinations of direction and speed, the car can perform a variety of maneuvers.

alexa-bot-block-diagram-v2

Part List and Hardware setup

Part List:

  • Raspberry Pi (3 in this case)
  • Adafruit MotorHAT DC motor driver
  • Basic car chassis with two dc motors
  • Portable phone charger
  • 4x AA batteries and battery pack
  • USB microphone
  • 3.5mm jack speaker

The first step is to connect the motorHAT and the Raspberry Pi using the pin connectors and assemble to chassis. The Pi talks to the motoHAT via the standard I2C as part of the GPIO setup. Arrange the Pi, speaker, and battery pack in a way that all the ports are easily accessible. For me, the Pi went in the back, the speaker in the middle, with the battery pack in the front. The phone charger which powers the Pi is placed underneath the chassis to save space. All the wires should be able to be connected easily. Downloading the Adafruit Python library provides a DC motor test script which can be run to check that the motors are working. If the motors are spinning backwards, simply take the wires and switch them around. Also test that the microphone and speakers are working. There are plenty of resources online to help with that.

IMG_20170128_135102

Installing Alexa on Raspberry Pi 3

Before starting the installation of Alexa make sure you sign up for a Amazon developer account. A lot of authentication has to be done in this project so all the security stuff will be done in this account.

Follow the installation from below to install the Alexa on the Pi.

https://github.com/alexa/alexa-avs-sample-app/wiki/Raspberry-Pi

This site will give you the full instruction on how to install and setup the Pi to use Alexa. Near the end, it does give two options for the wake word engine (that’s the program that causes the Pi to listen when you say “Alexa”. I personally could only get kitt_ai to work.

Using AWS Lambda for Alexa skill

Using your Amazon developer account, create a new Alexa skill using the Alexa skills kit. You can use any skill name and invocation name. The invocation name is what you will use to activate your skill. Next, create some intents which are the actual tasks that the robot will perform. They should be in this format:

{
“intents”:[
{
“intent”: “intent_name”
}
]
}

Underneath will be the area where the utterances are created. Type the intent name followed by the phrase you want Alexa to recognize. For example:

intent_name use this intent

Before we can continue with the skill, we need to create a lambda function. Lambda is a service that Amazon provides to run code on their server-less architecture. Create a blank function and configure the trigger to the Alexa skills kit, selecting whatever runtime you plan on using. In my case, I will be using Python 2.7 though anything will work. You can upload your code in a zip file, edit inline, or copy and paste from your own editor. Once you have your code, select the role “lambda_basic_execution”. If you don’t have it, create a new custom role and click “allow”. Now that you have a lambda function, copy the ARN found in the top right of the function page. It should start with “arn:aws:lambda:”. Now return to your Alexa skill. Under the configuration page, select ARN, choose the closest region to you and paste in the ARN from the lambda function. You can now test whether you skill works.

Using AWS IoT as message distribution

When your lambda function runs one of your intents, it needs to trigger a message to be sent over the mqtt protocol to the AWS IoT message broker. This message must be in the JSON format and will be the message the Raspberry Pi picks up. I accomplished this with the boto3 and json formatting libraries in python. The message is sent under a topic, the name under which the message is published under, a QOS value, which I have left at 1, and a payload, which is the message itself. Make sure when creating your payload that they are unique and that there will be no chance for confusion when trying to do the motor control logic later.

Note that the boto3 library is used because it is already built into the lambda included library.  You can simply import it inside your python code and boto3 library is then available to use.  I discovered that boto3 is basically the API to the AWS management functions (configuration/etc.).  It has very limited data plane function.  For IoT and MQTT support, it can only publish but it has no capability to subscribe.  For this simply project, this is sufficient since the message flow is unidirection (from AWS lambda to IoT).   In the longer run, if I want to retrieve information from the robot/Pi, I will have to do away boto3 since it has no ability to receive MQTT messages.

On the other hand, the AWS IOT library used on the Pi is based on the AWSIoTPythonSDK. You can find it in the github link below.   This has full data plane functionality (include the subscribe function). I used this on the Pi to subscribe for the topic that boto3 published on; and uses it to receive the MQTT message to control the motor. The reason I didn’t use this in the lambda function is because this library is not built-into AWS lambda yet and cannot be readily imported.

https://github.com/aws/aws-iot-device-sdk-python

Motor Control Logic and Using Motor Control HAT

This part is pretty simple. I mentioned earlier that there was a motor testing script that was included with the installation of the Adafruit motorHAT Python library. All the code that is needed to run the motors is in that example. Just copy and paste whatever is needed.

Installation instruction:

https://cdn-learn.adafruit.com/downloads/pdf/adafruit-dc-and-stepper-motor-hat-for-raspberry-pi.pdf

Github for the HAT:

https://github.com/adafruit/Adafruit-Motor-HAT-Python-Library

The motor control instructions I implemented includes:

  • Forward
  • Backward
  • Turn right
  • Turn left
  • 3 point turn right
  • 3 point turn left
  • U turn right
  • U turn left

Future additions

Future enhancement include self balancing actions and I like to add a camera to play with computer vision.