Turning the GPS up to 10(Hz)!

Until today I had been using the Adafruit Ultimate GPS pretty much straight out of the box.  This meant that it was running at 9600 BAUD, with 1Hz updates, including the satellite SKY records.  That's a whole lot of data to send over a 9600 BAUD serial line.  In practice that meant that NTP wasn't very happy with the GPS output, and one update per second was nowhere near enough for logging data on a motorcycle travelling at 100+mph (on the track of course).

The GPS is capable of 1/5/10Hz updates, but defaults to 1Hz.  It's also capable of BAUD rates up to 115200, but defaults to 9600.  In order to increase the update frequency, you really have to push the BAUD much higher. A number of people also recommend dropping the SKY records to reduce the throughput.  I looked into this and, unlike some microcontrollers, the Raspberry Pi is quite happy with 115200 BAUD, so the SKY records get to stay for now!

All you need to do to change the BAUD rate is to send the GPS the appropriate command and it will respond with a confirmation string.  Same again for the 10Hz updates.  The NMEA sentences that you need to send are well documented in the PDF.  You need is a checksum generator to find the magic characters for the end of the sentence, and after that, it's a simple bit of Python code to access the serial port and send the relevant commands.

#!/usr/bin/python

from serial import Serial
import time
outStr = ''
inStr = ''

serialPort = Serial("/dev/ttyAMA0", 9600, timeout=2)
if (serialPort.isOpen() == False):
  serialPort.open()
serialPort.flushInput()
serialPort.flushOutput()
outStr = '$PMTK251,115200*1F\r\n'
print "outStr = " + outStr
serialPort.write(outStr)
time.sleep(0.1)
inStr = serialPort.read(serialPort.inWaiting())
print "inStr =  " + inStr
serialPort.close()


serialPort = Serial("/dev/ttyAMA0", 115200, timeout=2)
if (serialPort.isOpen() == False):
  serialPort.open()
serialPort.flushInput()
serialPort.flushOutput()
outStr = '$PMTK220,100*2F\r\n'
print "outStr = " + outStr
serialPort.write(outStr)
time.sleep(0.1)
inStr = serialPort.read(serialPort.inWaiting())
print "inStr =  " + inStr
serialPort.close()

Losing time

This week didn't quite go according to plan, I was planning to be test-driving the logger by now, but it hasn't happened.  Why?  Well, mainly because of the very recent change in Raspbian Wheezy to use kernel 3.18.7+.  This brings about a huge change - device tree support.  I had never even heard of device trees before and to be honest didn't know I needed to know about them.  As will all things technology there has been a huge amount of banging my head on the desk, and the odd sweary word!! Why did it affect this project?  Well, i2c is now controlled by device-tree rather than just loading modules.  This change meant that i2c no longer loaded at all in fact.

It turns out that raspi-config has been updated to use device trees, and using it to enable i2c at boot works fine.  If you go to the raspi-config advanced menu, i2c can now be enabled.  Just choose enable and load modules on boot.  This means that the RTC should again be available on the bus at boot.

This doesn't entirely solve the problem though - you actually have to get raspbian to use it, something it's not overly keen on doing. What I had been doing was this, but I wan't happy with editing the hwclock.sh script to that extent, and disabling the udev calls didn't sit well either.  Even with this, the RTC wasn't loaded until quite a way into the boot process.  There must be a better way...

(I'll spare you details of the three days I spent delving into device trees, and get straight to the point)

There is a device tree overlay provided as part of 2015-02-16-raspbian-wheezy:
/boot/overlays/ds1307-rtc-overlay.dtb

This overlay is specifically designed to load the rtc_ds1307 kernel module, which supports many of the common RTCs, including my DS3231 unit.  All one needs to do is to call it from /boot/config.txt:

dtoverlay=ds1307-rtc

Believe it or not, you will now have a running RTC straight away at boot time!  The i2c drivers will be loaded and /dev/rtc0 will be ready and waiting for you.

That's not quite the end of the story though, you'll find that the system time is still set to Jan 1st 1970 after boot. Thankfully, Sob's post pointed me to an issue with the udev helper /lib/udev/hwclock-set.  It's only designed to set the timezone, not the system time.  Thankfully that's a trivial fix; just change --systz to --hctosys.  When booting you should find your Pi now has the correct time copied from the RTC to the system time.

The last time related issue is to do with the PPS pin on the GPS.  Now that device tree is in effect, enabling PPS on a GPIO pin is no longer done with a parameter in /boot/cmdline.txt.  Instead there's an overlay for that too!.  Just add dtoverlay=pps-gpio,gpiopin=18 (or whichever pin you want to use) to /boot/config.txt and the PPS pin will be configured and /dev/pps0 will be up and running right after boot.

It's been a bit of a bumpy ride this week, but it's been well worth it!

Hardware assembly v1.0

Until now, I've been testing the components individually, and all has gone really quite well.  So, the next logical step is to put all of the hardware together in a form that is portable enough to take it out for a blast to see how the GPS/accelerometer work under real conditions (well in the warmth and safety of the car to start with).  So, I present to you, hardware build v1.0.  

I haven't even powered it up yet, and there's a lot of config to do on this RPi as it's pretty much factory defaults!

The components shown above are:

• RPi B+ in a PiBow coupe case
• Camera in an ABS plastic housing
• Wifi dongle (remote admin/GoPro control)
• Bluetooth dongle (OBD dongle)
• Adafruit Ultimate GPS (direct to Pi UART, PPS connected)
• DS3231 RTC (to help with disconnected startup) hidden behind the big bunch of wires!
• 10DOF board (lean angle, accelleration, temperature)

As soon I get my hands on one of these I can get on with the final hardware assembly.  That's assuming that I don't start adding more components like this with one or more of these attached for brake position sensor, or these for tyre temperature monitoring!

Which way is up??

Another significant component of the datalogger is an Inertial Measurement Unit (IMU).  There are many many models out there, from the really expensive to the really cheap.  I decided to start with a 10DOF (degrees of freedom) board based on these chipsets: L3GD20 LSM303D BMP180.  It has a temperature and barometric pressure sensor in addition to the usual compass, accelerometer and gyro sensors. Delivered from HK for £5.88 in a couple of weeks, not bad!  I have since ordered an MPU-9150 board for £4.90 to see how that behaves. The choice of both boards was based on the support matrix for RTIMULib, which is probably going to be the library of choice.

The board came with an unsoldered header, but I'm getting the hang of this soldering malarkey, so it was soon a soldered header!  These boards all use the I2C, so the wiring is simple: power (+3.3v), ground and the I2C lines (SDA/SCL).  I2C is on GPIO 2 (SDA) and GPIO 3 (SCL).









To check that the RPI is seeing it, use i2cdetect on I2C bus 1:

pi@pi0 ~ $ i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- UU -- 1d -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- 39 -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- UU -- -- 6b -- -- -- -- 
70: -- -- -- -- -- -- -- 77                         

There are quite a few things on the bus, most of them are the chips on this board, but there's also a RTC that I added to help out with the GPS timing.

After downloading and installing RTIMULib, it was time for a bit of python:

import sys, getopt

sys.path.append('.')
import RTIMU
import os.path
import os
import time
import math
import datetime

SETTINGS_FILE = "RTIMULib"
print("Using settings file " + SETTINGS_FILE + ".ini")
if not os.path.exists(SETTINGS_FILE + ".ini"):
  print("Settings file does not exist, will be created")

s = RTIMU.Settings(SETTINGS_FILE)
imu = RTIMU.RTIMU(s)
pressure = RTIMU.RTPressure(s)

print("IMU Name: " + imu.IMUName())
print("Pressure Name: " + pressure.pressureName())

if (not imu.IMUInit()):
    print("IMU Init Failed");
    sys.exit(1)
else:
    print("IMU Init Succeeded");

if (not pressure.pressureInit()):
    print("Pressure sensor Init Failed");
else:
    print("Pressure sensor Init Succeeded");

poll_interval = imu.IMUGetPollInterval()
print("Recommended Poll Interval: %dmS\n" % poll_interval)

raw_input("Press Enter when ready\n>")

while True:
  if imu.IMURead():
    (x, y, z) = imu.getFusionData()
    data = imu.getIMUData()
    (data["pressureValid"], data["pressure"], data["temperatureValid"], data["temperature"]) = pressure.pressureRead()
    fusionPose = data["fusionPose"]
    os.system('clear')

    print "timestamp: " + str(data["timestamp"])

    print "accelValid: " + str(data["accelValid"])
    print "accel: " + str(data["accel"])
    print "accel x: " + str(data["accel"][0])
    print "accel y: " + str(data["accel"][1])

    print "compassValid: " + str(data["compassValid"])
    print "compass: " + str(data["compass"])

    print "fusionPoseValid: " + str(data["fusionPoseValid"])
    print "fusionPose: " + str(data["fusionPose"])
    print("r: %f p: %f y: %f" % (math.degrees(fusionPose[0]), math.degrees(fusionPose[1]), math.degrees(fusionPose[2])))
    print("r: %i p: %i y: %i" % (int(math.degrees(fusionPose[0])), int(math.degrees(fusionPose[1])), int(math.degrees(fusionPose[2]))))
    print("x: %f y: %f z: %f" % (x,y,z))

    print "fusionQPoseValid: " + str(data["fusionQPoseValid"])
    print "fusionQPose: " + str(data["fusionQPose"])

    print "gyroValid: " + str(data["gyroValid"])
    print "gyro: " + str(data["gyro"])

    print "pressureValid: " + str(data["pressureValid"])
    print("Pressure: %f" % (data["pressure"]))

    print "temperatureValid: " + str(data["temperatureValid"])
    print("Temperature: %f" % (data["temperature"]))

    time.sleep(0.2)

pi@pi0 ~ $ python imu_fields.py
Using settings file RTIMULib.ini
IMU Name: LSM9DS0
Pressure Name: BMP180
IMU Init Succeeded
Pressure sensor Init Succeeded
Recommended Poll Interval: 4mS

Press Enter when ready
>
timestamp: 1423748093614330
accelValid: True
accel: (1.0306559801101685, 0.04684799909591675, -0.16396799683570862)
accel x: 1.03065598011
accel y: 0.0468479990959
compassValid: True
compass: (43.95816421508789, 25.756946563720703, -5.119741439819336)
fusionPoseValid: True
fusionPose: (2.216735363006592, -1.4253621101379395, 2.5034217834472656)
r: 127.009581 p: -81.667233 y: 143.435503
r: 127 p: -81 y: 143
x: 2.216735 y: -1.425362 z: 2.503422
fusionQPoseValid: True
fusionQPose: (-0.44976526498794556, 0.4893990159034729, 0.551458477973938, 0.5040767192840576)
gyroValid: True
gyro: (0.15618298947811127, 0.15292565524578094, -0.16467183828353882)
pressureValid: 1
Pressure: 1013.469971
temperatureValid: 1
Temperature: 21.500000

That seems to work nicely, and is easily capable of 50Hz updates, and may well do 100Hz if I decide to go to that level.

Dealing with GPS

It's been a couple of weeks since my last post, but that's not down to lack of progress, it's down to having to earn some money, fighting with a Linux virus on a handful of machines, and having far too much fun with the RPi!

As this my datalogger blog, I'm not going to cover all the fun I've been having with a PIR detector and hardware PWM for fading LEDs/LCD backlight.  It looks like this will lead to a fancy new lighting system for the kitchen cabinets that come on when you walk into the kitchen - the design and most of the code is done, just need to buy the striplight and wire it up!  Nor will I cover the ongoing work on CatCam - again using a PIR to record what my cats are actually doing when they "knock" at the flat door demanding to be let in!

OK, back to the datalogger.  One of the most important parts of the kit is the GPS unit.  The one that I bought was the Adafruit Ultimate GPS.  It's a high quality GPS unit and perfectly suited to the task.  The main delay was that I had to solder the header pins myself!  If you read the previous post then you'll know that it had been a while since I'd picked up a soldering iron.  Basically I practiced quite a bit on header pins and resistors on veroboard until I felt that I could let myself loose on a £30 component.  In the end, it actually went very well.  One pin is a bit blobby, but I'm never going to use that pin and rather than mess it all up, it's staying that way.

Of course, if Adafruit had released this hat a couple of months earlier, I wouldn't have needed to bother with the soldering at all!  That hat will almost certainly be used in the finished datalogger just because it's going to be much more robust than wires attached to the GPIO pins, and I can mount the other components on it as well.

You'll see that I have attached an external GPS aerial for now as the board lives on my desk and gets absolutely no GPS signal at all!

In terms of software configuration, I am using the built in serial port (UART) as described by Adafruit here.  There's no point using a USB to serial convertor when you have a serial port doing nothing much.  Once the UART was free, all it really took was to get GPSD up and running. To do that, run dpkg-reconfigure gpsd, and feed it /dev/ttyAMA0 for the device, and -n for the flags, which activates the GPS unit even if no clients are connected to GPSD.  Apparently NTP doesn't count as a client.  While GPSD should autodetect the BAUD rate, it's probably worth sticking init_uart_baud=9600 into /boot/config.txt and rebooting.

Now it has a fix, it's time to move on to getting NTP working.  Basically there was a lot of faffing around before I got something that works.  Loads of really good info here and here.  As I wanted time to be as accurate as possible, PPS was the way forward.  Thankfully the recent Raspbian release has PPS support built in, all you need to do is to tell the GPIO PPS kernel module which pin you're using. Do that by adding bcm2708.pps_gpio_pin=18 to /boot/cmdline.txt.  Then add pps-gpio to /etc/modules and reboot.  If the GPS unit has a fix, it should output a PPS waveform, and this can be seen using ppstest

root@raspberrypi:/home/pi# ppstest /dev/pps0
trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
source 0 - assert 1421960593.000005477, sequence: 10060 - clear  0.000000000, sequence: 0
source 0 - assert 1421960593.999991566, sequence: 10061 - clear  0.000000000, sequence: 0
source 0 - assert 1421960594.999991654, sequence: 10062 - clear  0.000000000, sequence: 0

The next step is to rebuild NTP with ATOM support.  Hopefully a future Raspbian release will fix that!  I used the dpkg method from one of the docs above so that I can install it on all future builds. Once done, ntp.conf need to be configured as follows:

# gpsd shared memory clock
server 127.127.28.0 minpoll 4 maxpoll 4 prefer
fudge 127.127.28.0 time1 0.490 refid GPSD

# pps-gpio on /dev/pps0
server 127.127.22.0 minpoll 4 maxpoll 4
fudge 127.127.22.0 refid PPS flag3 1  # enable kernel PLL/FLL clock discipline

The time1 offset of 0.490 seconds has been calculated as the average response time for the GPS over UART on this RPi, yours may be different! There are also three standard pool.ntp.org servers there for now, but they'll go at some point soon. This gives decent time sync:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+127.127.28.0    .GPSD.           0 l    9   16  377    0.000  -37.625  43.493
o127.127.22.0    .PPS.            0 l    8   16  377    0.000   -1.817   0.139
*213.171.220.65  158.43.128.66    3 u   31   64  377   25.168    1.595  36.595
+217.114.59.3    158.43.192.66    2 u   25   64  377   26.759   -2.102  37.833
+129.250.35.250  140.203.204.77   2 u   26   64  347   20.654   -2.603  63.899

So, that's GPS pretty much sorted.  Next job is to get the IMU working to measure acceleration/lean angle/heading etc.

It's been a while!

I honestly can't remember the last time I soldered anything.  We did some hardware work at Uni in 1993, but it was all breadboard based. So, it's likely it was in the late 1980's!

Having bought a few components that require soldering I thought it best to start on something easy (and cheap).  The current wiring on the LCD panel was OK (see previous post), but was definitely in need of improvement.

Individual jumper wires have a habit of coming loose and there will be plenty of vibration on the Daytona to help them along their way.  Multiple jumpers together in a header are much harder to dislodge, and it's likely that the whole lot will either be connected, or not, making troubleshooting a bit easier.

Technobotsonline sell crimp headers in various sizes, right up to 1x16 way, which fits the pins on the LCD just fine.  Removing the individual housings is a bit fiddly, but once you get the hang of it and you decide that you have enough singles as spares now so the rest don't need to survive!
Fitting the 16 way was easy:

As were the smaller ones for the GPIO, power and ground pins at the breadboard:

Much tidier, but the final problem was pin 3 on the HD44780 is contrast.  Usually one would use a pot for that, but I don't need to be able to adjust the contrast, I just want some!  As mentioned in the previous post, a 3K3 Ω resistor does the job.  A bit of youtubeing (Is that a verb yet? It is a Google brand so it should be) turned up a couple of simple enough looking techniques for an inline resistor, which is also described well here. 

Time to warm up the soldering iron.

Not bad for a first attempt!

Heatshrink fitted and ready to go.

The finished article in place and working.

I'm still not confident in my soldering abilities, so I will be doing lots more practice before I move onto the GPS header pins. Next up will be a mini power distribution rail based on vero board - lots of pins to solder there.

Overall I think we can call my first bit of soldering in 20 years a success!

LCD goodness

The datalogger is essentially going to be an embedded system, but I'd still like to have an idea what's going on.  Attaching a screen to the seat cowl of a Daytona via HDMI is a non-starter, and even this touchscreen looks a bit big (but I might revisit that once I have everything fitted).  A bit of ebaying later and a 16x2, white on blue, HD44780 LCD display arrived through the door - not bad for £3.29 including postage!

I tried various Python classes for driving it, but most of them didn't work.  It seemed to be a 4bit vs 8bit issue.  The wiring diagrams showed 4bit wiring, but the code seemed to write 8bits per character.  Lots more googling later, I stumbled upon RPLCD on github.  After a quick github clone the code was ready to rock 'n roll.  Sure enough, it seemed to work, but the display still was only readable at a very odd angle.  More googling pointed to the contrast pin.  It was tied to GND as per many many articles I'd read, but a number of others used a 10K pot.  Not having one to hand, a mid-range resistor would have to do; so a 3K3 Ω resistor was used from V0 to GND.  Much better!  I butchered one of the test programs to create an infinite loop updating the date/time using cursor positioning (just for the practice).

Finally, I decided that +5v gave a much brighter display than I wanted.  The previous post used GPIO.PWM to dim an LED. After a bit of rewiring and code merging the backlight is now dimmable.

The above layout was taking up a lot of space on the breadboard.  One strip of male to female ribbon cable later:

I think I'm going to have to look into github as these scripts are now getting a bit long, and would probably benefit from some version control as well.  For now, here's the latest code:

#!/usr/bin/env python
import sys
import RPi.GPIO as GPIO
import datetime
from time import sleep

from RPLCD import CharLCD
from RPLCD import Alignment, CursorMode, ShiftMode
from RPLCD import cursor, cleared

try:
    input = raw_input
except NameError:
    pass
try:
    unichr = unichr
except NameError:
    unichr = chr

# Set the numbering scheme to GPIO numbers
GPIO.setmode(GPIO.BCM)

# LED on GPIO 16, initiall set to off
GPIO.setup(16, GPIO.OUT, initial=GPIO.LOW)

# DOWN switch on GPIO 20
GPIO.setup(20, GPIO.IN, pull_up_down=GPIO.PUD_UP)

# UP Switch on GPIO 21
GPIO.setup(21, GPIO.IN, pull_up_down=GPIO.PUD_UP)

# Set initial dutycycle to 50
dc = 50

# Set the frequency to 200Hz.
freq = 200

# Set up the LED
led = GPIO.PWM(16,freq)
led.start(dc)

def set_pwm(gpio_id):
  global dc
  global led

  # If the down switch is pressed, decrement dc
  if gpio_id == 20:
    if dc > 0:
      dc = dc - 5
      print "Down switch pressed. Setting dc to %s" % dc
    else:
      print "Down switch pressed. dc already at min value!"
  # If the up switch is pressed, increment dc
  if gpio_id == 21:
    if dc < 100:
      dc = dc + 5
      print "Up switch pressed. Setting dc to %s" % dc
    else:
      print "Up switch pressed. dc already at max value!"
  led.ChangeDutyCycle(dc)

# Add callbacks for both switches
GPIO.add_event_detect(20, GPIO.FALLING, set_pwm, bouncetime=200)
GPIO.add_event_detect(21, GPIO.FALLING, set_pwm, bouncetime=200)

lcd = CharLCD(cols=16, rows=2, numbering_mode=GPIO.BCM, pin_rs=11, pin_rw=None, pin_e=5, pins_data=[6,13,19,26])

lcd.clear()
lcd.cursor_pos = (0, 0)
lcd.write_string('Date:')
lcd.cursor_pos = (1, 0)
lcd.write_string('Time:')
try:
  while True:
    now = datetime.datetime.now()
    lcd.cursor_pos = (0, 6)
    lcd.write_string(now.strftime('%Y-%m-%d'))
    lcd.cursor_pos = (1, 6)
    lcd.write_string(now.strftime('%H:%M:%S'))
    sleep(0.1)
except KeyboardInterrupt:
  lcd.clear()
  led.stop()
  GPIO.cleanup()