This tutorial is the second in series on developing a Personal Assistant (Pi) using Raspberry Pi and IBM® Watson. Pi will take a picture of an object placed in front of it, classify the picture, provide classification with a score and play the results back to the user.

Note: All tutorials in this series are independent of each other, they simply use the same concepts.

Learning Objectives

At the end of this tutorial you will be able to:

  • Play audio on Raspberry Pi using Java
  • Take pictures from Raspberry Pi Camera using Java
  • Invoke Watson’s Visual Recognition service from Java
  • Invoke Watson’s Text to Speech service from Java

Hardware

Raspberry Pi 3 (RPi3) Model B, Raspberry Pi Camera, Speaker (optional)

Software

Java 8, IBM® Bluemix, JRPiCam Library

Design

This section provides a high-level design of how Pi – The Personal Assistant will recognize objects. When you run the code from a terminal a new interaction is started and following activities happen.

  • Pi greets the user
  • Pi waits for 3 seconds
  • Pi takes a picture of object placed in front of the camera
  • Pi invokes Watson Visual Recognition service, passes the PNG file as input
  • IF Watson is able to classify the image THEN
    • Pi creates a response based on results
    • Pi invokes Watson Text to Speech service
    • Pi plays the response back to user
  • ELSE Pi says “Sorry, I could not recognize the object”

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

IBM® Watson Setup

Step 1: Signup for Bluemix.

Step 2: Login to Bluemix, you will be redirected to the Console.

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

Step 3: Click on Watson.

Step 4: Click on Get started now! + to create a new service.

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

Step 5: From all Watson services, click on Visual Recognition service.

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

Step 6: Enter Service name and Credentials name (or leave as-is). You can leave the service unbound and select a Pricing Plan that suits you. Click on Create.

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

Step 7: Your Visual Recognition service is ready for use.

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

Step 8: Go back to Catalog > Watson and click on + to create Text to Speech service.

Step 9: From all Watson services, click on Text to Speech service.

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

Step 10: Enter Service name and Credentials name (or leave as-is). You can leave the service unbound and select a Pricing Plan that suits you. Click on Create.

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

Step 11: Your Text to Speech service is ready for use.

CodifyThings - Pi - Personal Assistant - Raspberry Pi - IBM Watson

 

 

Raspberry Pi Setup

Step 1: Download the code from Github repository.

Step 2: Latest versions of Raspberry Pi comes pre-installed with Java, so you only need to download required dependencies.

Step 3: Open the Constants.java file in com.codifythings.personalassistant package and copy-paste credentials from Watson services that you created.

Step 4: Run the main class com.codifythings.personalassistant.PersonalAssistant.

Final Product

Next Steps

This wraps up the tutorial. There are quite a few enhancements that you can make to this module e.g. build a robot that finds specific objects or a security camera that only records when there is a person in the vicinity.

Citations