June 30, 2014 · amcharts awesome css debian graph html javascript lighttpd linux programming raspberrypi rpi rrdtool temperature WiringPi www

Temperature Log | HowTo

Background

This project was born a long time ago, when I first got my hands on Raspberry Pi and started exploring physical computing. My first attempts, back in 2013, resulted in few corrupted temperature sensors, lots of tangled cable and few lines of Python code. Now I reminded myself of the dust covered project and decided to take it further.

RRD day graph

My initial idea was simply to log temperatures into Round Robin Database Tool (RRDTool) and create simple graphs. While developing the web front-end for my project, I've decided to make more of it, added some extra features to graphs, polished website appearance and optimized data collection.

Alongside, I've created a sister project based on MySQL database for data collection and AmCharts for visualization.

Both work well, as you can see here: Temperatue Log.

Hardware

One of the easiest sensors to connect to a Raspberry Pi is the DS18B20 digital thermometer. It's optput is read through a multi-device 1-wire bus that is directly supported by a driver in the Linux kernel. Several sensors can be connected in parallel to the same data-wire and read individually over the bus interface by their hard-coded IDs. An extensive tutorial on connecting the sensor to Raspberry Pi was written by Matthew Kirk.

For my project I've used 5 sensors on ~20m of cable. While lots of tutorials advice of using 4k7Ω resistor (only one!), because of the length of my network (or "network weight"), I've ended up using 2kΩ resistor for better readings.

I've soldered pin rows onto Humble Pi prototyping board for convenience of (re)attaching sensors.

Reading the data

The 1-Wire drivers are not loaded by default - they need to be activated manually. To load them permanently, edit /etc/modules on Raspberry Pi

raspberry@pi ~$ sudo vim /etc/modules 
# /etc/modules: kernel modules to load at boot time. 
# 1-Wire devices 
w1-gpio 
# 1-Wire thermometer devices 
w1-therm

and reboot the device.

Once connected, to check whether a sensor is detected type:

raspberry@pi ~$ cat /sys/bus/w1/devices/w1_bus_master1/w1_master_slaves 
28-000005502198 
28-00000550283e 
28-000005502a61 
28-000005501105 
28-000005515d97

The output is a list of sensor addresses on the 1-Wire network.

A quick check if the sensor operates as it should:

raspberry@pi ~$ cat /sys/bus/w1/devices/28-*/w1_slave 
91 01 4b 46 7f ff 0f 10 25 : crc=25 YES 
91 01 4b 46 7f ff 0f 10 25 t=25062 
5b 01 4b 46 7f ff 05 10 b5 : crc=b5 YES 
5b 01 4b 46 7f ff 05 10 b5 t=21687 
8c 01 4b 46 7f ff 04 10 2e : crc=2e YES 
8c 01 4b 46 7f ff 04 10 2e t=24750 
fa 01 4b 46 7f ff 06 10 99 : crc=99 YES 
fa 01 4b 46 7f ff 06 10 99 t=31625 
71 01 4b 46 7f ff 0f 10 56 : crc=56 YES 
71 01 4b 46 7f ff 0f 10 56 t=23062

The temperature is the last readout after 't=' in milidegrees Celsius.

To make the temperature outputs more human-readable, I've decided to use Python:

def read_temperature(file): 
    tfile = open(file) 
    text = tfile.read()
    tfile.close() 
    lines = text.split("\n") 
    if lines[0].find("YES") > 0: 
        temp = float((lines[1].split(" ")[9])[2:])
        temp /= 1000 
    return temp

Since I don't need Fahrenheit readouts, I didn't include the conversion in my code. However, if you need the temperature in F instead of Celsius (or both), there are numerous tutorials on the wide web on this subject.

RRDTool

Now that I could read the temperature from the sensors I needed a way to store the information. Round Robin Database is perfect for this task. It's a circular database that lets you store a predefined amount of data. After initial creation it is as big as it will ever get and just contains "unknown" data.
The 'rrdtool create' command is used to setup the database:

rrdtool create temperature.rrd \ 
  --start now --step 60 \ 
  DS:a:GAUGE:120:-50:50 \ 
  DS:b:GAUGE:120:-50:50 \ 
  DS:c:GAUGE:120:-50:50 \ 
  DS:d:GAUGE:120:-50:50 \ 
  DS:e:GAUGE:120:-50:50 \ 
  RRA:AVERAGE:0.5:1:12 \ 
  RRA:AVERAGE:0.5:1:288 \ 
  RRA:AVERAGE:0.5:12:168 \ 
  RRA:AVERAGE:0.5:12:720 \ 
  RRA:AVERAGE:0.5:288:365

This creates a database with a base data interval of 1m (-step 60), with a data range of -50 to +50 (degrees C), and some calculated averages for 6hours, day, week, month and year.

To get more of RRDTool, read an extensive tutorial.

Now, to pass sensor readings to the database, I used Python again.

Full Python code.

Creating Graphs

Since rrdtool can also graph, a shell script running once every 10 minutes create a graph image:

#!/bin/bash 
RRDPATH="/path/to/database" 
IMGPATH="/path/to/img/dir" 
RAWCOLOUR="#FF0000" 
RAWCOLOUR2="#CC3366" 
RAWCOLOUR3="#336699" 
RAWCOLOUR4="#006600" 
RAWCOLOUR5="#000000" 
TRENDCOLOUR="#FFFF00" 
RRDFILE="database_file.rrd" 

#hour 
rrdtool graph $IMGPATH/hour.png --start -6h \ 
--full-size-mode \ 
--width=700 --height=400 \ 
--slope-mode \ 
--color=SHADEB#9999CC \ 
--watermark="Bart Bania" \ 
DEF:temp1=$RRDPATH/$RRDFILE:a:AVERAGE \ 
DEF:temp2=$RRDPATH/$RRDFILE:b:AVERAGE \ 
DEF:temp3=$RRDPATH/$RRDFILE:c:AVERAGE \ 
DEF:temp4=$RRDPATH/$RRDFILE:d:AVERAGE \ 
DEF:temp5=$RRDPATH/$RRDFILE:e:AVERAGE \ 
LINE2:temp1$RAWCOLOUR:"Water" \ 
LINE2:temp2$RAWCOLOUR2:"NaN1" \ 
LINE2:temp3$RAWCOLOUR3:"Server Fan" \ 
LINE2:temp4$RAWCOLOUR4:"Main Room" \ 
LINE2:temp5$RAWCOLOUR5:"Hall\l" \ 
HRULE:0#0000FF:"freezing" 
...

At the top of the script I've got pre-defined changeable variables, such as paths and colour codes for my convenience. This particular example creates a graph for the last 6 hours for all five sensors.

Day and night time on the graph

Apart from a plain graph, I wanted to add a bit of flavour to it by marking day and night times, as well as dusk based on Civil twilight calculations. Instead of writing my own code, I've decided to make it simpler and use an external tool. A great small C program for calculating sunrise and sunset came in hand.
sunwait gives a whole bunch of information, including civil, nautical, and astronomical twilights, so I had to extract the data that was of interest to me:

# Geographical position 
LAT="" 
LON="" 

# Get the relevant data in an ugly manner 
# Calculate twilight 
DUSKHR=`/usr/bin/sunwait sun up $LAT $LON -p | sed -n '/Sun rises/{:a;n;/Nautical twilight/b;p;ba}' | cut -c 45-46` 
DUSKMIN=`/usr/bin/sunwait sun up $LAT $LON -p | sed -n '/Sun rises/{:a;n;/Nautical twilight/b;p;ba}' | cut -c 47-48` 
DAWNHR=`/usr/bin/sunwait sun up $LAT $LON -p | sed -n '/Sun rises/{:a;n;/Nautical twilight/b;p;ba}' | cut -c 30-31` 
DAWNMIN=`/usr/bin/sunwait sun up $LAT $LON -p | sed -n '/Sun rises/{:a;n;/Nautical twilight/b;p;ba}' | cut -c 32-33` 

# Calculate night time 
SUNRISEHR=`/usr/bin/sunwait sun up $LAT $LON -p | sed -n '/Sun transits/{:a;n;/Civil twilight/b;p;ba}' | cut -c 30-31` 
SUNRISEMIN=`/usr/bin/sunwait sun up $LAT $LON -p | sed -n '/Sun transits/{:a;n;/Civil twilight/b;p;ba}' | cut -c 32-33` 
SUNSETHR=`/usr/bin/sunwait sun up $LAT $LON -p | sed -n '/Sun transits/{:a;n;/Civil twilight/b;p;ba}' | cut -c 45-46` 
SUNSETMIN=`/usr/bin/sunwait sun up $LAT $LON -p | sed -n '/Sun transits/{:a;n;/Civil twilight/b;p;ba}' | cut -c 47-48` 

# Convert hours/minutes to seconds 
SUNR=$(($SUNRISEHR * 3600 + $SUNRISEMIN * 60)) 
SUNS=$(($SUNSETHR * 3600 + $SUNSETMIN * 60)) 
DUSK=$(($DUSKHR * 3600 + $DUSKMIN * 60)) 
DAWN=$(($DAWNHR * 3600 + $DAWNMIN * 60))

and add it to graphs as shaded area representing night times where I live:

CDEF:nightplus=LTIME,86400,%,$SUNR,LT,INF,LTIME,86400,%,$SUNS,GT,INF,UNKN,temp1,*,IF,IF \ 
CDEF:nightminus=LTIME,86400,%,$SUNR,LT,NEGINF,LTIME,86400,%,$SUNS,GT,NEGINF,UNKN,temp1,*,IF,IF \ 
AREA:nightplus#E0E0E0 \ 
AREA:nightminus#E0E0E0 \ 
CDEF:dusktilldawn=LTIME,86400,%,$DAWN,LT,INF,LTIME,86400,%,$DUSK,GT,INF,UNKN,temp1,*,IF,IF \ 
CDEF:dawntilldusk=LTIME,86400,%,$DAWN,LT,NEGINF,LTIME,86400,%,$DUSK,GT,NEGINF,UNKN,temp1,*,IF,IF \ 
AREA:dusktilldawn#CCCCCC \ 
AREA:dawntilldusk#CCCCCC

Full BASH code.

Scheduling

Using crontab, I can schedule automatic data collection and graph creation. Since the graphs are created mainly for this website, I've decided to use my web user's crontab:

root@raspberrypi ~# crontab -u www-data -e 

* * * * * /path/to/python/poller.py > /dev/null 2>&1 
*/10 * * * * /path/to/bash/script.sh > /dev/null 2>&1

The above invoke the tasks:

  • Python code addind data to RRD - every minute
  • Bash script creating graphs - every 10 minutes

The Website

The website itself is hosted on my main sever. All the files are dynamically transferred from Raspberry Pi. This ensures me the SD card would last longer and no data would be lost. Apart from that, it ensures the website runs smooth without any interrupts.

The website was created with HTML, CSS and JavaScript.

External tools used here are:

the source for the website is available through Git repository.

  • LinkedIn
  • Tumblr
  • Reddit
  • Google+
  • Pinterest
  • Pocket

Contact