Friday, January 20, 2017

Raspberry Pi Console Matrix Screensaver

Overview




While developing on the Raspberry Pi I have long wished to have a screen saver on the console rather than the screen just going blank. One option is to use xscreensaver but I like cmatrix (Console Matrix - a "matrix the movie" like display usually run via the CLI). If you do want to try xscreensaver then you can install using:

sudo apt-get install xscreensaver


Install cmatrix


For those that want to go down the cmatrix path, this is the way to do it. Start by installing cmatrix if you don't already have it.

sudo apt-get install cmatrix

To run cmatrix just type the following in Terminal:

cmatrix

You have the following command line options:

    -a: Asynchronous scroll, more like the movie/original screensaver
    -b: Partial bold text mode
    -B: All bold text mode
    -f: Force the linux $TERM type to on
    -l: Linux support for linux console matrix.fnt font
    -n: No bold text mode
    -o: Use "old-style" scrolling - less like the movie, more like the Win/Mac screensaver.
    -s: "Screen saver" mode.  Any key aborts (default is "eye candy" mode, must abort with control C)
    -x: X window mode, use if your xterm is using mtx.pcf
    -u [update]: Update delay (0-9, default 4).
    -C [color]: Use color for matrix (default green).

While cmatrix is running you can press various keys to adjust the display.

    a: Toggle asynch mode
    b: Enable partial bold text
    B: Enable all bold text
    n: Disable bold text
    0-9: Change update delay

You can change the text colour using the following symbols :

    ! – red
    @ – green
    # – yellow
    $ – blue
    % – magenta
    ^ – cyan
    & – white
    ) – black

Making cmatrix a Screen Saver


At the moment you have to type cmatrix every time you want it to run which isn't really a screensaver. The trick is to combine cmatrix with GNU screen.  Screen enables you to run processes within a “terminal tty instance”. The install command is as you would expect:

sudo apt-get install screen

Type screen to run it and screen -list to see all the screen instances that you have running. Before you do that we need to create the screen configuration file. This will allow us to get the screen saver functionality (more or less).

When it first runs, screen will check its configuration file (.screenrc) in your home directory first. This is normally a hidden file but you can see it if you use:

ls -a

If you have an existing file then edit that, if not create a new one in your home directory.

nano .screenrc

Within the .screenrc file you want to add the following lines:

blankerprg cmatrix -BC blue -s
idle 60 blanker

You can obviously use whatever options you want for cmatrix, these are just the ones I like. Credit to K. Mandla for working this out. Note that blanker is the invocation and blankerprg is the “variable” it starts. You can replace cmatrix with something else if you get board with the matrix (e.g. tty-clock -r).

Idle is a command that is run after the specified number of seconds of inactivity is reached. This command will normally be the blanker command to create a screen blanker, but it can be any screen command. You can change this from 60 seconds if you want.

If you now type "screen" in Terminal then after 60 seconds of no activity cmatrix should run.

We almost have a fully functional screen saver. The last step is to get screen to run automatically whenever the Raspberry Pi reboots. I thought this would be easy but it took me 3 attempts to find a solution that worked. The upside is I learnt a lot about the various start up scripts and screen options.

Auto run GNU Screen on Boot - 1st Attempt


As always in linux there are lots of ways to do anything. I thought the easiest approach would be to edit the rc.local file.

The script /etc/rc.local is for use by the system administrator. It is traditionally executed after all the normal system services are started, at the end of the process of switching to a multiuser runlevel. You might use it to start a custom service, for example a server that's installed in /usr/local.

In order to have a command or program run when the Pi boots, you can add commands to the rc.local file. On your Pi, edit the file /etc/rc.local using the editor of your choice. You must edit with root, for example:

sudo nano /etc/rc.local

Add commands below the comment, but leave the line exit 0 at the end, then save the file and exit. Your file will look something like this:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
    printf "My IP address is %s\n" "$_IP"
fi 
screen -dmS "launch"
exit 0

The only problem is that rc.local runs early in the boot process, before a terminal has been created. So you can only use screen to create a detached screen (i.e. not visible). You can run processes in this detached screen but that is unhelpful for what we are trying to do. Interestingly when researching this I found that the code that should print out your IP address on boot no longer works because rc.local runs before the network is setup.

You will notice in the video that I have a Sense Hat installed on my Pi, so I do use the rc.local file to start a python program which runs on the Sense Hat as a sort of "screen saver". Actually I just like the look of it. The program is a looping version of Conway's Game of Life. Since I can't use rc.local to start an attached screen, I replaced the screen -dmS "launch" line with:

sudo python /home/pi/python_code/intro/SenseConway.py &
Notice the & after the line launching the python app?

WARNING:: If your command runs continuously (perhaps it has an infinite loop) or is likely not to exit, you must be sure to fork the process by adding an ampersand (&) to the end of the command. 

Otherwise, the script will not end and the Pi will lock up - I admit I made this mistake at one stage.  Not even CTRL C will get you out of this. In the end I fixed the issue by swapping the SD card with a new version of Raspbian and mounting the old SD card with a USB Card Reader and then editing the rc.local file. You will find the mounted SD Card under media which wasn't an obvious place and took a bit of searching. You have to use chown to change ownership of the rc.local file to the current yser, which allows you to edit it.

The ampersand allows the command to run in a separate process and continue booting with the process running.

Also, be sure to reference absolute filenames rather than relative to your home folder; for example, /home/pi/myscript.py rather than myscript.py.

Auto run GNU Screen on Boot - 2nd Attempt


An alternative to editing rc.local is to use cron. Cron is the general name for the service that runs scheduled actions. crond is the name of the daemon that runs in the background and reads crontab files. Edit your crontab file using:

sudo crontab -e
A crontab is a file containing jobs in the format:

minute hour day-of-month month day-of-week  command
You can also use cron to schedule a job to run at reboot using @reboot. Perfect I thought, except you run into the same problem as using rc.local, using @reboot you can only add detached screens.

I then tried getting cron to start screen 60 seconds after reboot, using something like:

@reboot sleep 60 && my_script.sh
But this didn't work either.

Auto run GNU Screen on Boot - 3rd Attempt


 ~/.profile (historically referred to as ./bash_profile or ~/bash_login) is another shell script run at login time. This is the script that ended up doing the trick for me. It is obviously run after boot has completed and the terminal shell is in place.

I added the following line to the end of my .profile file. "launch" is just the name of the screen session.

[ -z "$STY" ] && screen -Rd "launch"

Note - STY: Alternate socket name. If screen is invoked, and the environment variable STY is set, then it creates only a window in the running screen session rather than starting a new session.

If .profile hadn't worked, then I would have tried .bashrc - there is always another way with linux!

.bashrc is a shell script that Bash runs whenever it is started interactively. A common thing to put in .bashrc are aliases that you want to always be available.

That's it! Well not quite...

Stop Screen Blanking - 1st Attempt


You will now have cmatrix automatically running as your screen saver but after a preset time your screen will still go blank! Back to the drawing board. If you ask Dr Google you will find a number of proposed solutions.

The first thing I tried was editing the /etc/kbd/config file.

sudo nano /etc/kbd/config
I made the following two changes to the file:

# screen blanking timeout. monitor remains on, but the screen is cleared to
# range: 0-60 min (0==never) kernels I've looked at default to 10 minutes.
# (see linux/drivers/char/console.c)
BLANK_TIME=0 (Was 30)

# Powerdown time. The console will go to DPMS Off mode POWERDOWN_TIME
# minutes _after_ blanking. (POWERDOWN_TIME + BLANK_TIME after the last input)
POWERDOWN_TIME=0 (Was 30)
Then rebooted, but CLI screen blanking still occurred.

Stop Screen Blanking - 2nd Attempt


The next thing I tried was editing:

sudo nano /etc/lightdm/lightdm.conf
In that file, look for: [SeatDefaults] and insert this line:

xserver-command=X -s 0 dpms
Try rebooting again. This didn't work for me.

Stop Screen Blanking - 3rd Attempt


Then I tried adding setterm -powersave off -blank 0 to our old friend rc.local (see autorun GNU screen - 1st attempt above).

This worked!

Saturday, January 14, 2017

Mission Package 2 - Voice Recognition for AVA (Part 2)

USB Microphone


You will of course need a USB microphone in order for the Pi to hear you speak. You can buy a USB adaptor for a standard 3.5mm mic but I went with the SunFounder USB 2.0 Mini Microphone for Raspberry Pi 3, 2 Module B & RPi 1 Model B+/B (figure 1).


Figure 1. SunFounder USB 2.0 Mini Microphone

This is where a Raspberry Pi 3 may be a better choice. With built in WiFi you have an extra free USB port.

Pi AUI Suite




For the voice recognition package our first test was using Steven Hickson's Pi AUI Suite. The thing we like best about this software is the ability to create your own word lists and associated actions. The suite is up to version 3 and you can read about it on Steve's blog. The one caveat is that it appears that the suite is no longer being developed or supported. In particular text to speech (i.e. when the Raspberry Pi responds to a command) appears to be problematic. We will develop an alternative approach to get around this.

A quick summary of Pi AUI Suite. The configuration is done in the commands.conf file (more on this later).
  • The format is voice==command (where voice is the word which AUI Suite looks for)
  • You can use any character except for newlines or == 
  • If the voice starts with ~, the program looks for the keyword anywhere. Ex: ~weather would pick up on "weather" or "what's the weather" 
  • You can use ... at the end of the command to specify that everything after the given keyword should be options to the command. For example: play==playvideo ... This means that if you say "play Futurama", it will run the command playvideo Futurama 
  • You can use $# (where # is any number 1 to 9) to represent a variable. These should go in order from 1 to 9. For example: $1 season $2 episode $3==playvideo -s $2 -e $3 $1 This means if you say "game of thrones season 1 episode 2", it will run playvideo with the -s flag as 1, the -e flag as 2, and the main argument as game of thrones, i.e. playvideo -s 1 -e 2 game of thrones 
  • Because of these options, it is important that the arguments range from most strict to least strict. 
  • This means that ~ arguments should probably be at the end. 
  • You can also put comments if the line starts with # and special options if the line starts with a ! 
  • Default options are shown as follows: 
    • !keyword==pi,
    • !verify==1,
    • !continuous==1,
    • !quiet==0,
    • !ignore==0,
    • !thresh==0.7,
    • !maxResponse==-1 
    • api==BLANK,
    • !filler==FILLER FILL,
    • !response==Yes Sir?,
    • !improper==Received improper command:,
    • !duration==3,
    • !com_dur==2,
    • !hardware==plughw:1,0,
    • !language==en_us 
  • Keyword, filler, and response accept strings. verify, continuous, quiet, and ignore except 1 or 0 (true or false respectively). thresh excepts a floating point number. These allow you to set some of the flags as permanent options (If these are set, you can overwrite them with the flag options). 
  • You can set a WolframAlpha API and maxResponse (the number of branches) like !api==XXXXXX-XXXXXXXXXX amd !maxResponse==3 
  • You can now customise the language support for speech recognition and some text to speech with the language flag. Look up your country code and use that. Ex. For US: !language==en_us, for Spain !language==es, for Germany !language==de.

Installing Pi AUI Suite


The install process is fairly straight forward. Start by installing the dependencies. Open up Terminal or connect via SSH and type the following (all on one line):

sudo apt-get install -y libboost-dev libboost-regex-dev youtube-dl axel curl xterm libcurl4-gnutls-dev mpg123 flac sox

Then download the install files:

git clone https://github.com/StevenHickson/PiAUISuite.git

Change to the install directory:

cd PiAUISuite/Install

Run the install command:

sudo ./InstallAUISuite.sh

Choose y to install any dependencies required and work your way through the installation wizard. If you encounter an error during this process (I did) then install the Boost C++ library:

sudo apt-get install libboost-regex1.49.0

After this library has installed, Run sudo ./InstallAUISuite.sh again.

Alternate Text to Speech Solution


After running the install wizard you can use the setup wizard to generate your command configuration file. To run the setup wizard:

sudo voicecommand -s

As part of this process you have the option to test text to speech (TTS). This appears to be no longer working. The Pi AUI Suite is using the Google TTS API and in early 2016 Google stopped letting automated scripts access the service. You can read about this issue on the Pi AUI Suite GitHub repository.

One workaround is to install the speech synthesis software called Festival:

sudo apt-get install festival

You can test that this is working by typing the following:

echo "Test message" | festival --tts

You will obviously need to have some speakers plugged into the 3.5mm audio out plug. You can adjust the volume using:

alsamixer

Then use the up and down arrows to adjust volume. ESC to save and exit.

Configuring Pi AUI Suite


You can run voicecommand using the default configuration options detailed above, but you will probably want to tweak the available commands. The first thing we need to do is move the configuration file because we are running the program as the root user. To do this type:

sudo mv /home/pi/.commands.conf /root/.commands.conf

To edit the configuration file:

sudo voicecommand -e

or if you prefer using your choice of text editor you can type something like:

sudo nano /root/.commands.conf

You can now change your keyword or add extra commands. For example, if you want the Raspberry Pi to tell you the current time, you could add:

time==echo "The time is " | festival --tts && date | festival --tts

CTRL X to exit, select y to save and then return.

Running Pi AUI Suite


To run voice command:

sudo voicecommand -c

The program will then listen out for your keyword and attempt to respond.

In the next post we will evaluate performance of Pi AUI Suite and look at alternatives.


Tuesday, January 10, 2017

Mission Package 2 - Voice Recognition for AVA (Part 1)

Overview


The next step in AVA's evolution is to add voice recognition. There are many different approaches but I have chosen a Raspberry Pi 2 as the hardware platform because I also want to use it as the master node when implementing ROS (Robot Operating System) down the track. I will then connect the various Arduino's that AVA uses as sub nodes. This will replace the dodgy serial messaging system that I am using to communicate between microprocessors at the moment.

Figure 1. Using Disk Utility to determine SD Card Reader BSD Number.


Install Raspbian on a SD Card


Note that these instructions are for installing Raspbian directly, if you are using NOOBS then replace steps 3-6 with extract the downloaded archive and copy across onto the SD Card. Done.
  1. Download a copy of Raspbian.
  2. For macOS users, place your microSD card in a USB Card Reader. Your microSD needs to be empty and formatted in FAT32. You can format and name your card using SD Card Formatter. Select overwrite format.
  3. We now need the BSD number of our SD Card Reader. The official RPi installation instructions suggests using "About this MAC" but that didn't work for me (I am running El Capitan). So the alternate method is to fire up Disk Utility (see Figure 1). From this we can see that the Device name is disk3s1 and thus the BSD number is disk3. You need Disk Utility for the next step, so you might as well use it to determine the BSD number
  4. Unmount the partition so that you will be allowed to overwrite the disk. To do this, go back to Disk Utility and select the partition and click unmount (top of the screen); do not eject it, or you will have to reconnect it. 
  5. Extract the img file from the file you downloaded in Step 1. This can be harder than it should be. If you have problems with the standard Mac unzip then use The Unarchiver.
  6. From Terminal, run the following command:
sudo dd bs=1m if=path_of_your_image.img of=/dev/diskn

Remember to replace n with the number that you noted before (i.e. 3 in our case). Also, you need to include the complete path of your Raspbian image file. An easy way to find a file’s exact location is to drag and drop the file into the Terminal window. As this is a sudo command you will need to enter your Mac password.


Figure 2. RaspberryPi and 5" Touchscreen.


Writing to the SD Card will take a few minutes and Terminal doesn't indicate progress. You can check the progress by sending a SIGINFO signal (press Ctrl+T). That’s it! You can now eject the microSD card and plug it into your Raspberry Pi.

Adafruit 5" Display



Figure 3. Adafruit 5" Display.

As shown in Figures 2 and 3, we are using the Adafruit 5" touchscreen display as our monitor. This next bit only applies if you happen to be using the same bit of kit.

The display is a mini panel-mountable HDMI monitor with a built-in touchscreen. This backpack features the TFP401 for decoding video, and includes the attached display.

The TFP401 is a DVI/HDMI decoder from TI. It can take unencrypted video and pipe out the raw 24-bit colour pixel data - HDCP not supported! The 5" display is 800x480 resolution, which is just enough to run most software, but still small enough that it can be used in portable or embedded projects without the bulk.

You can power the entire display from a USB port. With the default 5" 800x480 display and 50mA backlight current, the current draw is 500mA total. You can reduce that down to 370mA by running the backlight at half-brightness (25mA). With the backlight off, the decoder and display itself draws 250mA. If you want more backlight control, there's a PWM input, connect that to your microcontroller or other PWM output and you can continuously dim the backlight as desired.

The touch screen shows up as a USB mouse so no special drivers are needed. You can power the driver over USB as long as your computer can supply 500mA on the USB port, and then feed it video via the HDMI port.

Note the TFP401 decoder chip does not contain a video scaler, it will not resize/shrink video! I suspect this is what causes the Raspberry Pi some trouble.

The default installation of Raspbian (and other linux variants) doesn't work with this display. The NOOBS installer works for some reason but once you boot into Raspbian the same problem as a direct install occurs (i.e. wavy lines and unreadable text).

To fix this, you need to update the config.txt file. You will either need to plug in a different display to your Raspberry Pi or do it on your laptop and copy the file onto the SD card. You can download a copy of the required config.txt file from the Reefwing Gist repository.

If you look closely at Figure 2, you will see that we are using a Raspberry Pi 2 Model B V1.1. This should give us sufficient processing power, if not I'll move to a Raspberry Pi 3.

No Sound from the Speakers?


While you are playing around with the config.txt file, it is probably worth testing that your speakers are working. You can do this by typing on your Raspberry Pi (CLI):

speaker-test

You should hear pink-noise hiss from the speakers. If not then add the following line to your config.txt file and reboot.

dtparam=audio=on

To test volume levels, it helps to have human voices for the test. To hear a female voice say "front left" and "front right", from either speaker, you can use:

speaker-test -t wav -c 2

In part 2 we will download the actually voice recognition and test to speech software.

Sunday, January 8, 2017

Amazon Echo Dot - Alexa



It may be a stretch to call the Echo Dot a robot but slap on some wheels and you are getting close. With the acquisition of the Dot there are now 4 "robots" in various state of repair running around the house. In addition to the new Dot, the robot vacuum cleaner "ZAC" from Neato is going strong, we also got a BB-8 for Christmas which likes to patrol around the house and we are working on voice recognition for AVA.

Just like Apple's "Siri" and Microsoft's "Cortana",  the Dot uses the Amazon equivalent "Alexa" as your personal digital assistant. You can ask Alexa all the usual questions "What is the weather like?" "What is the news", etc, but what I like is the ability to read you a book from your Kindle library or to play a song from any bluetooth connected device or Spotify/Pandora.

The equivalent of apps for Alexa are called skills. You can develop your own by signing up for the free Amazon Developers program, which I did. In a subsequent post I will cover how you go about whipping up a skill. It looks fairly straight forward but no doubt there will be some tricks...