2012/02/22

Graphing Vivarium Humidity and Temperature Using RRDTool

The temperature and humidity monitoring of my vivarium is really coming together.  I'm recording a timestamp, humidity, and temperature to an SD card every minute using Adafruit Industries' SD card breakout board:
Components on breadboard

The SD card is sneaker netted to a computer running RRDtool and used to produce graphs to give a sense of how the environment looks across the day.  The temperature nicely lines up with what I'd expect given the way the thermostat on the house heating is set up:
24 Hours of Temperature Data



On the other hand, the humidity seems to wander quite a bit more than I'd have expected.  I'll be tuning my RRD settings and generally playing around with the set up to make sure this isn't being introduced by how I'm collecting data:
24 Hours of Humidity Data
And all together:
24 Hours of Humidity and Temperature Data
Wiring
I did shuffle the Boarduino to the center section of my breadboard to get some space on the side for the SD card breakout board.  The design of this breakout is really nice for adding to an Arduino or Boarduino as the pins line up 1-to-1.  I haven't played around with the Card Detect pin yet, but at some point I plan to try it out.
Zoomed View of the Connection Between the Boarduino and Adafruit SD Card Breakout Board






Components with Logical Connections


Arduino Code

At startup, configure the SD card pins and set up related variables
Once a minute, log the timestamp and the current sensor readings
Log the readings and timestamp to a file on the SD card

Full source code is available on github:
https://github.com/dmalec/Project-Masdev/tree/master/masdev_sdcard 

RRDtool
It's worth noting that there are much more sophisticated ways to use RRDtool than what follows and that the setup below only tracks one day worth of data (E.G. it's very likely that during import later data from the SD card will overwrite earlier data from the SD card, since it's unlikely one will be meticulous about dumping the card data to RRD after exactly 24 hours). All that said, this is a first iteration, and the future of the vivarium will hopefully include a more advanced setup.

Creating the RRDtool Database
rrdtool create vivarium.rrd \
Create the database in a file named vivarium.rrd
--start 1329597420 \
Specify that the first time point in the database is 1329597420. This is in unix time which, broadly, is the number of seconds which have elapsed between midnight on January 1st, 1970 and the time you want to specify as the earliest potential point in the database.
--step 60 \
Specify the number of seconds expected between individual data readings. In this case, I'm expecting to take a reading every minute from the two sensors.
DS:h0:GAUGE:300:0:100 \
Set up the humidity data source. It will be called h0 and is of type GAUGE, which is good for values that vary over time. If 300 seconds pass without receiving a new value, RRD will consider the value to be unknown (if less time passes, RRD will generate interpolated data to cover the gap). 0 is the minimum expected value. 100 is the maximum expected value.
DS:t0:GAUGE:300:0:100 \
Set up the temperature data source. It will be called t0 and is otherwise a straight copy of the humidity sensor. This could cause issues if the reading goes below 0 or above 100, but in those cases I'll probably have bigger concerns :).
RRA:AVERAGE:0.5:1:1440
Set up the archive for the above datasources to track the average value. 0.5 specifies that half of the recorded data points may be marked as unknown before the stored value is marked unknown (I'm pretty sure this is currently irrelevant since I'm inputting data at the same rate that RRD is storing it - if I start reading at a higher rate than once a minute, and tweak the 300 above to be something less than 60, it should start to be relevant -- but take this with a grain of salt as it's pure speculation). Use 1 recorded data point to generate 1 stored data point. Store 1440 seconds (1 day) worth of data.


Importing the Sensor Data into the RRDtool Database
My thinking when writing the code for the Arduino was that I could walk the file line by line and append it to an rrdtool update command. This kept producing graphs which were off by 5 hours... it took an embarrassingly amount of time for me to realize that was suspiciously the same as my UTC offset. The Arduino has no concept of local vs. UTC time - so unixtime on my Arduino and on my computer are not the same thing. There's some magic going on below to split the line up, add the appropriate offset, and then join the line. This relies on the computer which originally set the time on the Arduino and the computer running RRD to have the same UTC offset. It also suggests I should rethink my approach.

Creating an RRDtool Graph
rrdtool graph vivarium-humidity.png \
Specify the filename of the image to create.
--width 1024 --height 512 \
Set the area in pixels of the graphing area in the image.
--start $START --end start+24h --step 60 \
The first time point in the graph is set to the earliest timestamp in the RRD file.
The end time point in the graph is set to the first time point plus 24 hours.
Specify 60 seconds between datapoints.
--title "Vivarium Humidity"
Set the title at the top of the image.
--vertical-label "Relative Humidity" \
Set the text to draw along the left of the image
--watermark "Generated on `date`" \
Set text at the bottom of the image and include the current system time
--color CANVAS#FFFCED \
Set the background color of the graphing area to #FFFCED.
--color BACK#E8E1CC \
Set the background color of the border around the graphing area to #E8E1CC.
--color ARROW#000000 \
Set the color of the arrows at the ends of the axes to #000000
DEF:my_humid=vivarium.rrd:h0:AVERAGE:step=60 \
Define a variable, which varies over time, my_humid from the file name 'vivarium.rrd' and template name 'h0'. 'AVERAGE' specifies that data should be averaged if needed to match the graph. The granularity of the data is specified as 60 seconds.
VDEF:max_humid=my_humid,MAXIMUM \
Define a single value variable max_humid based on the maximum value of my_humid.
VDEF:min_humid=my_humid,MINIMUM \
Define a single value variable min_humid based on the minimum value of my_humid.
CDEF:smooth_humid=my_humid,300,TREND \
Define a variable, which varies over time, smooth_humid based on a sliding window average of my_humid across a 5 minute window.
LINE6:smooth_humid#1A1A40:"Avg" \
LINE4:smooth_humid#39398C \
LINE2:smooth_humid#5353CC \
Draw three lines, one on top the other, to produce a "single" line with a gradient effect. The first 6 pixel line is drawn with the darkest color #1A1A40 to get a strong outline and appears in the legend as "Avg". The later lines are, respectively, 4 and 2 pixels thick.
LINE4:max_humid#66281A::dashes \
LINE2:max_humid#B2462E:"Max":dashes \
Draw two lines, one on top the other, to produce a "single" line with an outline effect. The first 4 pixel line is drawn with the darkest color #66281A to get a strong outline. Both lines are drawn as dashed lines to differentiate from the variable line.
LINE4:min_humid#2A616B::dashes \
LINE2:min_humid#48A5B8:"Min":dashes \
Very similar to the maximum line, but tracking the min_humid.
COMMENT:"\n" \
Force a line break after the color labels in the legend.
GPRINT:my_humid:AVERAGE:"Avg\: \t%3.2lf%s\n" \
GPRINT:my_humid:MAX:"Max\: \t%3.2lf%s\n" \
GPRINT:my_humid:MIN:"Min\: \t%3.2lf%s\n"
Print min/max/avg for my_humid in the legend. The format string escapes : with \: so it's not interpreted as a field delimiter. \t forces the right hand side to line up at the next tab stop. %3.2lf specifies that the floating values should be printed with 3 digits, a decimal point, and then 2 digits. \n forces a line break after each field.

Bill of Materials
Description Cost
Adafruit MicroSD Card Breakout Board $15.00
Kingston 2GB MicroSD Flash Card $8.99
Adafruit AM2302 (Wired DHT22) Temperature-Humidity Sensor $15.00
Adafruit Boarduino $17.50
Adafruit FTDI To USB Adapter $14.75
16x2 LCD $9.95
Adafruit LCD Backpack $10.00
Adafruit Real Time Clock (DS1307) Breakout Board $9.00
12V 6A Power Supply $8.62
Total $108.81


No comments:

Post a Comment