This tutorial is the second in series on developing a Personal Assistant (Pi) using Raspberry Pi and IBM® Watson. Pi will take a picture of an object placed in front of it, classify the picture, provide classification with a score and play the results back to the user.
Note: All tutorials in this series are independent of each other, they simply use the same concepts.
Learning Objectives
At the end of this tutorial you will be able to:
- Play audio on Raspberry Pi using Java
- Take pictures from Raspberry Pi Camera using Java
- Invoke Watson’s Visual Recognition service from Java
- Invoke Watson’s Text to Speech service from Java
Hardware
Raspberry Pi 3 (RPi3) Model B, Raspberry Pi Camera, Speaker (optional)
Software
Java 8, IBM® Bluemix, JRPiCam Library
Design
This section provides a high-level design of how Pi – The Personal Assistant will recognize objects. When you run the code from a terminal a new interaction is started and following activities happen.
- Pi greets the user
- Pi waits for 3 seconds
- Pi takes a picture of object placed in front of the camera
- Pi invokes Watson Visual Recognition service, passes the PNG file as input
- IF Watson is able to classify the image THEN
-
- Pi creates a response based on results
- Pi invokes Watson Text to Speech service
- Pi plays the response back to user
- ELSE Pi says “Sorry, I could not recognize the object”
IBM® Watson Setup
Step 1: Signup for Bluemix.
Step 2: Login to Bluemix, you will be redirected to the Console.
Step 3: Click on Watson.
Step 4: Click on Get started now! + to create a new service.
Step 5: From all Watson services, click on Visual Recognition service.
Step 6: Enter Service name and Credentials name (or leave as-is). You can leave the service unbound and select a Pricing Plan that suits you. Click on Create.
Step 7: Your Visual Recognition service is ready for use.
Step 8: Go back to Catalog > Watson and click on + to create Text to Speech service.
Step 9: From all Watson services, click on Text to Speech service.
Step 10: Enter Service name and Credentials name (or leave as-is). You can leave the service unbound and select a Pricing Plan that suits you. Click on Create.
Step 11: Your Text to Speech service is ready for use.
Raspberry Pi Setup
Step 1: Download the code from Github repository.
Step 2: Latest versions of Raspberry Pi comes pre-installed with Java, so you only need to download required dependencies.
- Watson API – Java SDK
- JRPiCam (Java API for Raspberry Pi Camera)
Step 3: Open the Constants.java file in com.codifythings.personalassistant package and copy-paste credentials from Watson services that you created.
Step 4: Run the main class com.codifythings.personalassistant.PersonalAssistant.
Final Product
Next Steps
This wraps up the tutorial. There are quite a few enhancements that you can make to this module e.g. build a robot that finds specific objects or a security camera that only records when there is a person in the vicinity.