Tuesday, August 20, 2013

The Surveillance System


this year I planned to travel with my family to China for a few weeks, so we had to leave alone our house. While being on vacation, I wanted to check sometimes if everything was alright at home. So I decided to install two more cameras and connect them to the webserver of my home automation system. I had one camera attached already before and I wrote about it here. Both cameras I placed inside the house so I did not have to bother about rain. One was directed to our garden through a window, the other was directed to our living room. I could have done more thoughts about the camera placement, but I considered this more as an experiment. Also I was constrained with the cables. I wanted to use the raspberry connected to the cameras without wireless, so there was only one place left to put the cameras without drilling holes: close to my dsl modem.  

The webcams and the hub

I bought two webcam vx-800 mostly because they are cheap, and I had good experience with them with the my first camera. Unfortunately this type of camera has a fixed shutter, so it controls the light passed through with the shutter speed. The problem is that the pictures are very often overexposed. But for this time I did not want to spend more money on the them.
The usb ports of the raspberry pi can only deliver about 100mA, and with two cameras, it will be over the limit. So I also bought a 4 port hub from logilink. A power supply came with it, and it was able to provide more than 1 A. It never came up to my mind before, but when you have a power supply for the hub, you dont need an additional one for the raspberry. It draws its power from the hub via usb.

The motion detection

I think it is quite useless to have a pure webcam, which constantly delivers life pictures. It is better that the cameras detect motions and store the motions on the sd card as a video. For this I installed the application motion. I created a howto guide here, so I will only describe a few settings here.
In the motion config file motion.conf, I had to add (or take out the comments) the following two lines:
  
thread /etc/motion/thread1.conf
thread /etc/motion/thread2.conf


So since I had two cameras, there were two config files. In the files thread1.conf and thread2.conf I adjusted the parameters target_dir and webcam_port. It worked almost instantly (I had few problems, but honestly I forgot about them).
You can setup motion to generate shockwave files each time the number of pixels exceeds a threshold between two sequential pictures. I assume it is the parameter ffmpeg_video_codec.
And this is exactly what I want. If there is some movement, the raspberry pi is storing this movement as a short video. Now the webserver just needs to pick up the videos and integrates them into the webpage generation process.

The video retrieval

my webserver runs on a different raspiberry pi, than the one which runs the motion detection with the cameras. So I implemented a getter program which downloads the swf files from the camera raspberry. I pasted the code which is doing this below. The first step, it logs into the camera raspberry with ssh and it uses a public key to overcome the need for a password, see the -i option. Then it executes the find command to get all swf files which are not older than 3 hours. In the second step, it searches for all swf files in an directory of the webserver. In the foreach loop, the program checks if the swf files already have been loaded, if not, it loads them with a scp command. Note the -i option, which is needed to omit the password entry.


my @args1 = ("ssh", "-i /home/pi/cam23key", "pi\@192.168.2.103 ", "\"find /data/webcam/cam2/ -mmin -180 -type f  | grep swf; \"");
my @args2 = ("find /var/www/cgi-bin/pics/cam3 -type f | grep swf");

my line;
my @lines1;
my @lines2;

open(IN1, "@args1 |") or die "Can't open pipe: !";
while(<IN1>) {
    line = _;
    chomp(line);
    push(@lines1, line);
}
close(IN1);

open(IN2, "@args2 |") or die "Can't open pipe: !";
while(<IN2>) {
    line = _;
    chomp(line);
    push(@lines2, basename(line));
}
close(IN2);


foreach(@lines1) {

    line = _;
    if (grep {_ eq basename(line)} @lines2) {
    print "Found line\n";
    } else {
    print "No line, copying ...\n";
    my ret = system("scp -q -i /home/pi/cam23key pi\@192.168.2.103:line /var/www/cgi-bin/pics/cam3");


    }
}


I apologize for taking out the perl dollar characters. It seems blogger cannot handle them.

I added this program in the webservers crontab, and it executes it about every hour once.

Integration into the webserver

swf files can be integrated into html with the following html::template code:

<embed width=30% height=30% fullscreen=yes src="<!-- TMPL_VAR NAME=FIRSTPATH -->">  

Key is the embed tag, which makes inclusion of swf files possible. TMPL_VAR NAME is a perl html::template module parameter, which is one of the perl modules I make use of.  FIRSTPATH is a html::template variable which gets assigned in the perl program.  It is the path and file name of the swf file to be displayed. Below you can see an example of the webpage provided by my webserver:



I added in the template code a selection form, where the user can select a swf file from a list. The template code looks like this:

<select name="selected_picture">
   <option value="<!-- TMPL_VAR NAME=FIRSTPATH -->" selected="selected"><!-- TMPL_VAR_NAME=FIRSTPIC -->
   </option>
   <TMPL_LOOP NAME="PICTURES">
   <option value="<!-- TMPL_VAR NAME="PATH" -->"><!-- TMPL_VAR NAME="PIC"--> 

   </option>
   </TMPL_LOOP>

</select>
<p><input type="submit" name="Submit" value="Submit" /></p>



FIRSTPATH is a html::template variable containing the selected swf video file, so the selection form is displaying the swf name which is actually running as a video.

 PICTURES is a html::template variable. The perl program assigns a list of swf files into it. The list is gathered from the perl program by reading the directory, where the swf files are located (this is where the getter perl program puts them, see above):

opendir(D, "path") || die "Can't opendir\n";
my @filearr = grep(/\.swf/,readdir(D));


I apologize for taking out the perl dollar characters. It seems blogger cannot handle them.

TMPL_LOOP is a html::template instruction, which is looping through PICTURES and adding its entries into the selection form.
 
 Below you can see how this all looks like when accessing the website. You can see how a swf video is selected from the list. After this the user needs to press a submit button and the new swf video is loaded.



Final words

I had the camera surveillance system running for more than two weeks while I was in China. It just worked fine. The swf video had been often overexposed, but this was expected due to the low quality of the cameras I use.
I thought that I would have troubles with accessing the webpage because of China's internet restriction policies. But I was completely wrong and I learned my lessons not to be too prejudiced.



Saturday, May 18, 2013

The Ventilator Controller

here I like to spend few words about the ventilator controller I have created. I bought two ventilators which need 220V to drive its motors. Probably I could have used something commercial to switch and drive them, but I wanted to use CAN. Therefore I decided to create my own circuit with mikrocontroller, CAN interface and 220V drivers for the ventilators. 

 The mikrocontroller

I used the same type of mikrocontroller as for the one-wire controller: AT90CAN128. It has a CAN interface, which I knew how to use. The PCA82C250 I use for the CAN transceiver.  

The power supply

The power supply contains the same chips as for the one-wire controller. The step down contoller
LM2575 converts 12V DC to 5V DC and drives the digital chips. The 12V DC comes along with the CAN bus cable, which I have installed inside my basement.

The 220V drivers

To drive the ventilators, I use two S202 S12 solid state relais. Each relais contains a digital input, which can be directly connected to the mikrocontroller. I also added varistors and fuses for circuit protection.

Below a picture of the circuit I created. The red lines indicate the chip positions.


Saturday, May 4, 2013

The One Wire Controller

In this section I want to describe the one wire controller in more detail. Before I started the home automation project I have completed few projects with atmel 8 bit controllers, so I was familiar with this type of controller. But I still made few thoughts about alternatives such as the logo device, or other programmable logic controllers from Siemens. Logo is a simple device, but I did not like that I can only attach peripherals with a proprietary bus system. And the perpherials and the bus system was expensive. I knew from the beginning, that I wanted to add a humidity sensor, which needs data conversion. It could have been an analog/digital converter or a signal converter, depending on the humidity sensor chip.
It became clear to me, that I will be most flexible, if I design my own controller. But I had to decide which components I wanted to include to the controller; components such as power supply, sensors, memory etc. So I created a list of sensors, which I present here:
  • Temperature Sensor
  • Humidity Sensor
  • Real Time Clock (is this a sensor? Let's say yes)
  • Rain Sensor
The temperature sensors had to be distributed around the basement, because I wanted to sense condensation at certain locations, and this can be done, if I measure humidity at certain spots together with temperature.
Due to this distribution of sensors around the basement, it was necessary to think about a bus system, or alternatively wireless. The decission came in favor of a bus system. The rationals are listed in one of the posts below, so I will not elaborate. After doing some research and discussions with colleagues, I found a bus system called one wire from Dallas instruments (now Maxim). The one wire bus looked very promissing because:
  •  the wire needs only one strand (in theory)
  • sensors are available, such as the temperature sensor (DS18B20)
  • drivers are available, it converts I2C into one wire signals (DS2482S-100)
Since I know the CAN bus very well, it could have been an alternative for accessing sensors. Unfornuately, there are not many inexpensive sensors available which I can connect directly to CAN. This is different with one wire. So I decided to use one wire.

The one wire driver

The one wire driver DS2482S-100 is the device interface between a master processor and the devices attached on the one wire bus. Unfortunately I realized pretty late that this chip actually implements only the physical layer of the one wire protocol, so there was a lot work left for me to implement the driver software. However this presentation has helped me to implement the driver: one wire overview.

The real time clock

I tried to find a real time clock which has a I2C or SPI interface, so it can be connected to an atmel processor. I chose the RV 3029-C2, because it was available from the distributor from where I usually order. This real time clock is convenient, because it has a crystal included and that it can be powered with 5V and 3,3V.

The power supply

I created a couple of circuits before and I always used the LM2575 from National Semiconductors (or Texas Instruments) to generate regulatated 5V. It is a step down regulator and it is supposed to use less power than a linear voltage regulator.

The CAN drivers

CAN pins from the processor cannot be connected directly to the CAN bus. The signals need to be leveled and there are chips available for this. They are called CAN tranceivers. One widely available chip is the PCA82C250 from NXP. The drawback of this chip is that it can only be powered with 5V. But there are also other chips with the same packaging and pinout which can be powever by 3,3V (e.g. the SN65HVD230D).

The EEPROM

There is persistent memory needed for the data the processor collects, so I tried to find an eeprom with large memory capacity.  My distributor had the 24AA1025 available, so I decided to use this one. It can store up to 125kBytes of data, but it needs an extra pin to select the two separate devices on the same chip. The protocol for accessing the chip is I2C.

The USB driver

I had to add an interface to access the data from the processor and the eeprom from my laptop. I used a FT232R from FTDI for this purpose. This chip can be connected to the processor via the RS232 interface and it converts the signals to USB. A special driver needs to installed on the laptop to access the chip. I used windows terminal for this purpose.

The processor

As mentioned above, I used to work with atmel processors before and it came natural that I choose an atmel processor for this project again. There is one atmel processor which has a CAN interface which is the AT90CAN128. It has 128kByte flash size. The gcc compiler for this processor is available here, and jtag devices for program download is widely available.

 The one wire controller

Below a picture of my one wire controller. I used eagle from CADSoft to create the schematics and the layout. A PCB manufacturer created the board for me. Like always I created layout mistakes, which I had to correct somehow, such as I underestimated the package size of the eeprom. Somehow I was able to solder it on the board anyway.

 Final Words

Before I would create such a layout again, I would do some research if I could use one of the Arduino boards. I am not sure if these boards were available during the time I created my cicuit, however they became pretty popular by now.
I would probably completely take out the USB port, because it became quite useless, since I access the one wire controller from the raspberry pi through CAN. For debugging it was very useful, but I could do the same now through CAN as well.













Saturday, April 27, 2013

The Webserver on Raspberry Pi

in this blog I will describe what I have done to implement a webserver on my raspberry pi. I was making thoughts about the user interface I want to have for my control system. I actually started with a command line interface in C++ to access the sensor data and to control the actuators (ventilator). But I realized very soon that I can never remember the syntax of command lines and it almost always leads me to look into the source code to understand how the syntax was. Also, a graphical user interface is much nicer and is something to show off. In order to save time I decided to use the command line commands, and to call them from the graphical interface. Next I had to decide how program the graphical interface. I am a C++ programmer, but I am not familiar with screen programming at all. But it would be really nice, if I could look from my work place into the basement data, or from any other place such with a smart phone. So the next decision came pretty fast: I need a webserver. Here the list of the requirements I had:
  • Make use of the available command line commands
  • Need a webserver
  • Need a website to access the basement data and to control the actuators
 So in the following chapters I will go more into detail:

 The Webserver

I am not programmer who has done with webservers a lot and I have little knowledge about php, python, css and all these other knicks knacks available on the market. But I know perl, and I remember that perl was used (and is still used) for dynamic websites. To save time again, I decided that I want to use perl, because I do not want to learn yet another language, which is doing all the same anyway. And perl has system and open comamnds to call my C++ command line commands and it can also read out the return values easily. So I need a webserver, which can do perl (probably most webserver can do anyway).
Next I had to decide which webserver I use. After a little research I found these:
 I was reading pro and cons, and I got as many opinions about webservers as websites I have visited, such as: lighttpd has memory leaks, nginx performs so well, apache is overloaded, cherokee does not perform, fhem has too small community etc etc. I made my decision upon ease of use or ease of install on the raspberry pi. I found this website, and I was seeing that lighttpd is very easy to install (probably the others too, but first comes first) and it can do perl (probably the others too). Installation of lighttp was no problem at all.
The next decision was, what gateway I use to generate HTML code with perl. Again I made a little research, but much less than before. Pretty fast I decided to use standard CGI, just because I used it in the past once, because lighttpd supports it and because I found the book Web, Graphics & Perl/Tk from O'Reilly in my bookshelf where it is described (I have to admit I did not read much). I do not remember that I had to do something special in /etc/lighttpd/lighttpd.conf, I left the setup as is after installation. I would just recommend to everyone to take a look into this file to see if perl is enabled and to check the documents root. What is important though, is that you set the permission of the document root directly /var/www appropriately and that you set the group and owner to www-data (which is the owner's name of the webserver). I also created a directory cgi-bin in /var/www for the perl programs, see also this website.

Webpage Programming

 Again I had to do some research because during my first trials, I created HTML content directly with perl print statements and the perl code did not really look good (perl code never looks good in my opinion). But I found some good perl tutorials here, which helped me to make a choice on using perl's HTML::Template. Here you can separate your perl program from your HTML code, and you dont have to use perl's print statements to generate HTML.

The Login Page

 First I did not want to spent too much time in all this, but I wanted a minimum in security. So a password entry is needed, before I can use my website. Considering that I do not have the raspberry pi running all the time, I think this is sufficient at the moment. I am not a security specialist, but I know I should not store the password in plain text for lookup on the server. I need to encrypt the typed in password and and I need to encrypt the lookup password and then compare both encrypted password. Another question was, where to store the encrypted password? I can use a file, but what if there are more than one user who want to have access. I thought that storing encrypted passwords in a simple file is not a good idea, so I chose mysql database to store them. The good part about perl is, that it has password encryption support Crypt::PasswdMD5 and mysql support DBI. So I installed the packages with the cpan command. I had to setup mysql first after installation. I basically did these commands on the raspberry pi:

mysql -u root -p #starts my sql with root as user, and new command prompt mysql should show up

mysql> create database yourdatabase;
mysql> grant all privileges on yourdatabase.* to pi@localhost;
mysql> use yourdatabase
mysql> create table users(login varchar(20), password varchar(80));
mysql>exit

I found this program to generate a md5 encrypted password from plain text.  Also I created a perl program to add a user into the database with the encrypted password as paramter (actually I pipe it). An extract of my perl program looks like this:

use DBI;
my dbname="yourdatabase"; #see above database creation
my username="pi"

my pw=....; 
my data1 = "DBI:mysql:dbname";
my data2 = DBI->connect(data1, username, undef, { RaiseError => 1 }) || die( "cant open DB!");
my sql="insert users values('ARGV[0]', 'pw');";
my abfr1=data2->prepare(sql);

abfr1->execute;

I apologize for not showing the perl dollar signs. But blogger seems to have a problem with it.

Now I can add users with their encrypted passwords into the database. Back to the login page. Perl has another convenient library is CGI::Session. This is useful to pass on parameters from one page to another. Also it has the feature to expire, which gives another security element into my webpages. Below you find an extract of the perl code to generate the login page from the login HTML template. The template just consists of a form with user entry, password entry and a submit button, so I will not explain any further. If the user is entering the user name and password into the login webpage, then data is sent to the webserver and the login.pl program below is called again. This time it will call the function user_authenticated, which is just comparing the entered user name/password with the user name/password in the database. If they match, the webserver redirects to start.pl, which is the perl code to generate my website.

use CGI::Session;
use HTML::Template;
use Crypt::PasswdMD5;
use DBI;

my session   = CGI::Session->new or die CGI::Session->errstr;
my template  = HTML::Template->new(
                      filename          => './login.tmpl',
                      associate         => [session],
                      die_on_bad_params => 0,
                      global_vars       => 1,
                      cache             => 0 );
if ( cgi->param('Login') ) {
    my home = './start.pl';
    print cgi->redirect(home) if user_authenticated();
}
print session->header;
warningsToBrowser(1);
print template->output;


I apologize for not showing the perl dollar signs. But blogger seems to have a problem with it.

 The login page looks like this:










     


 

The Curve Tab

The control system runs for two years with only few interruption. I was interested how the sensor data has developed during the past two years, so I think this could be best represented as a curve.
Each time the control system gathers data from the sensors, it is stored in the 1-wire controller's eeprom. I implemented a couple of command line commands to query the sensor data from the 1-wire controller into the raspberry pi. The question was now, where to put the data. The answer came fast, since I started with a database while creating the login page. So it came natural to store the sensor data into the same database. So I created a new table with the followin mysql command:

mysql> use yourdatabase;
mysql> create table sensorvalues(id int not null primary key auto_increment, datestamp date, ...  ,mot1 int); 


Then I implemented a perl program which calls one of my command line commands to gather the sensor data from the 1-wire controller eeprom and stores them into the mysql database. Meanwhile the 1-wire controller has gathered more than 2500 entries, while each entry contains data and time stamps, two relative humidity values, two temperature humidity sensor values, two absolute humidity values, and five temperature sensor values. So I grouped the relative humidity data in one graph, the temperature humidity sensor values in one graph, the absolute humidity values in one graph and the five temperature sensor values in one graph. Therefore I will have four graphs with several curves on each. I have to admit that I did not read much in the book Web, Graphics & Perl/Tk from O'Reilly, but one interesting thing I have seen is, that graphs can be easily created with gnuglot. The installation for gnuplot is described here. The good thing about gnuplot is, that you can run it as a command line and it creates a picture file.
So what I need is a mysql command which retrieves all data within a time range (start date and end date) and stores it into a file, then a gnuplot command is executed with the file as parameter and with additional view settings. The perl commands to access the mysql database look like this:

my data2 = DBI->connect(data1, username, undef, { RaiseError => 1 }) || die("cant open DB!");
my @ergf;

startdate = ARGV[0];
enddate = ARGV[1];
sql = "select * from sensorvalues where datestamp between \"startdate\" and \"enddate\";";
my abfr1=data2->prepare(sql);
abfr1->execute;

I apologize for not showing the perl dollar signs. But blogger seems to have a problem with it.

 Next the perl program stores all the data into a file called "data". Then it executes the gnuplot command line which looks like this for relative humidity picture:

open(GP, "| /usr/bin/gnuplot") or die "couldnt open gnuplot: !";
print GP "set terminal png size 1000,250\n";
print GP "set xdata time\n";
print GP "set timefmt \"%Y-%m-%d\"\n";
print GP "set xrange [\"startdate\":\"enddate\"]\n";
print GP "set yrange [0:100]\n";
print GP "set format x \"%Y-%m-%d\"\n";
print GP "set timefmt \"%Y-%m-%d %H:%M:%S\"\n";
print GP "set linestyle  2 lt 2\n";
print GP "set output \"relhum\_startdate\_enddate.png\"\n";
print GP "set xtics tics\n";
print GP "set style line 1 lt 2 lc rgb \"green\" lw 2\n";
print GP "set style line 2 lt 2 lc rgb \"red\" lw 2\n";
print GP "plot \"data\" using 1:3 ls 1 title 'relative humidity 0' with lines, \"data\" using 1:4 ls 2 title 'relative humidity 1' with lines\n";
close GP;


Note the line set xtics tics. I had trouble with the x-axis labels, which did overlap. By setting xtics to an appriopriate number, the overlapping problem disappeared. The code for the other graphs is similar, so I will skip this.
I added in the curve tab HTML template a HTML select forms, where I can setup the start date and the end date. As with the login page, I use HTML::Templates to split the perl program from the HTML code. I figured very fast that handling of dates are not easy. When calling the webpage I wanted that the browser shows the current date as the end date, and one week before that as a start date. That means I need to calculate seven days from todays date. In some cases it is easy to calculate, but in other cases not. Here gets perl convenient again, because there are libraries which are already doing this. I installed Date::Calc with cpan and I included the code below:

(sec,min,hour,mday,mon,year,wday,yday,isdst) = localtime(time - 7 * 86400);

 Below you see a picture of the curve tab webpage:















 

In the picture above you see the select forms to setup a start date and a end date, and submit button. The default setup is todays date as end date, and 7 days before that as start date. And you see two graphs with relative humidity and temperature humidity sensor curves.
  

The Sensor Tab

It is good to see the sensor values on curves, but it is not really easy to read out the exact value from them. So I came to the conclusion that I need another tab where I can see the exact values from a specific date and time. 
























I created an HTML select form so I can choose the date and the time from when I want to show the data. The time select form offers predefined time values, which are the times when the 1-wire controller stores the sensor values into its eeprom. After submiting with the button the webserver alls the sensor tab perl program, which is accessing the mysql database with the command:

sql = "select * from sensorvalues where timestamp like \"time0[0]:%\" and datestamp = \"sensorvalues{'sample_year'}-mon0-day0\";";

I apologize for not showing the perl dollar signs. But blogger seems to have a problem with it.

Then the perl program is storing the values into HTML::Template parameters and it is generating the HTML code from the template.

The Camera Tab

The camera tab shows the last feature I have implemented, but it is not finalized yet. When choosing the camera tab, the perl program generates HTML with a picture from a webcam, which I have installed in my backyard. I have described the implementation of the webcam in this forum (I used a raspberry pi for this too!).
The problem which I was facing here is how to get the picture from my raspberry pi driving the webcam to my raspberry pi running the webserver. I decided to use scp in combination with secure/private keys. When things go automatically, I cannot use something where I need to type in a password manually. So I have to get around it. Fortunately it can be done without passwords, if I use the scp command with the -i option and with a private key generated by ssh-keygen. I stored the public key on the server side, which is in my case the raspberry pi driving the webcam. I created a perl script which is executing the following command:

my @args = ("ssh", "-i /somedir/privkey", "pi\@xxx.xxx.xxx.xxx ", "\"find /data/webcam/motion/* -type f -name \\*.jpg | sort -n | tail -1;\"");
open(IN, "@args |") or die "Can't open pipe: !";

I apologize for not showing the perl dollar signs. But blogger seems to have a problem with it.

The command above is first accessing the webcam raspberry via ssh. Then it executes the find command to create a list of jpg files (in this case the list has size one) the webcam has created. Then the perl program is executing the following:

my ret = system("scp -q -i /somedir/privkey pi\@xxx.xxx.xxx.xxx:line /var/www/cgi-b
in/pics");


I apologize for not showing the perl dollar signs. But blogger seems to have a problem with it.

It accesses with scp the raspberry pi driving the webcam and copies the jpg file directly into the cgi-bin. From there the HTML code can reference the picture.
A cron job is executing both commands above and it is starting them once an hour.This means that the picture is only updated once an hours as well.
Below you can see a picture of the camera tab.



















Probably I will combine this in future with a HTML select form, where I can choose the picture I want to see.

 Final Words.

 I am pretty happy with the result of my project, though I cannot tell that it is finalized yet. The camera feature needs to be improved. I am still struggling about what to do to make it more usable. I thought about using swf files instead of jpg files, so I can see some movements.
The webserver's performance is ok. It is definitively not fast, but it is sufficient for me. Especially the curve creation tab perl program takes some time (about 5-10 seccond), so I need to be patient there.
Another few words about perl. I used to dislike perl very much before I have done this project. The reason was that I cannot create good looking perl programs. But this completely has changed now after I realized how large the cpan library for perl is. I had the impression that for each problem I ran into, there was a perl library already available, such as Date::Calc, CGI::Session or Crypt::PasswdMD5. Meanwhile I really like perl!



Saturday, April 20, 2013

The Network

To have internet access to the control system, I decided to use the Raspberry Pi with the EW-7811UN wireless USB adapter which connects it to the internet router. My router can setup a dynamic DNS, so I am able to access the raspberry pi, without knowing the IP address. Having domain name is also convenient for the webserver programming, so the Raspberry Pi's webserver does not need to figure out the router's IP address. On the SPI port I added the CAN controller MCP2515, so the Raspberry can communicate with the other devices via CAN protocol, see Picture below:

There are two more controllers attached to the CAN bus: the 1 wire controller and the ventilator controller.

The 1 Wire Controller

The 1 wire controller contains interfaces for two 1 wire buses. I decided to connect the temperature sensors on the first 1 wire bus, and the humidity sensors on the second. The reason why I have a 1 wire controler is that 1 wire sensors are widely available. In theory they can even drive their power from the signal line. But I am not using the 1 wire sensors in that way (I could though). Sensors with CAN interfaces are less available, they are larger, they need power supply, and they are more expensive.

The Ventilator Controller

The ventilator controller uses two S202 S12 solid state relais to apply 220V to the ventilators to turn them on and off.

Final Words

Why did I choose CAN and 1 wire instead of using zigbee, FS20, or something similar with wireless? There are several reasons:
  • Zigbee, FS20 or anything else with wireless were not very popular or available during the time I designed the control system. I am not aware that there are sensors having zigbee (maybe there are, but I did not look), and if yes, I can imagine they are expensive.
  • I am a little suspicious about the reliability of radio controlled systems. E.g. the operating range of these radio controlled systems is rather limited. Probably I would have needed repeaters all over the house. And what if the neighbor uses the same FS20 system. So when he opens his garage door with FS20, he triggers my ventilators as well? 
  • All the devices need power supply anyway, so there must be a cable which supplies power somehow. In my system the devices draw power from the CAN cable.
So I can have radio controlled devices, which need power and probably repeaters (which also need power), or I install in a one time effort (I know it is not a little effort) a cable in the house with bus signals strands and power strands.





Sunday, April 14, 2013

The Humidity Sensor

Commercial humidity sensors usually measure only the relative humidity, because it is easy to realize technically. The relative humidity however does not describe how much water is contained in the air. It only describes the relation between the partial pressure of water vapor and the saturated vapor pressure. Alternatively you can interprete the relative humidity as the relation between absolute humidity and the saturated vapor density. In the NASA note D-8401 , you can find all formulas describing the relationships between absolute humidity and relative humidity, see formula (5). The absolute humidity comes down to a formula having temperature, relative humidity and the saturated vapor pressure as variables, see formula (2) combined with (5). The saturated vapor pressure is approximately a logarithmic function with the temperature as variable. I implemented it in the code as a lookup table. Below you find the saturated vapor pressure curve I created:


The humidity sensor

On the humidity sensor which I designed, I use the sht11 from Sensirion to measure temperature and relative humidity. It contains a proprietary 2 wire interface, so connecting to I2C will not work. The processor on the humidity sensor is a atmega16 from Atmel. The code running on the processor implements Sensirion's 2 wire protocol. Also I connected the addressable switch DS2408 to the processor. This converts bit signals applied from the processor to the switch into 1-wire signals. This is how my control system communicates with the humidity sensors.

Code to calculate absolute humidity from relative humidity and temperature

here I want to show how I have implemented a C function to calculate the absolute humidity from the relative humidity and the temperature. Above I was telling that I implemented the saturated vapor pressure as a lookup table. This is how I have done it:

   static int es[] =
{63,70,76,83,90,103,117,130,144,158,198,212,226,240,255,283,297,326,354,383,401,435,474,516,
560,610,657,710,764,818,872,939,1007,1076,1145,1227,1310,1407,1504,1602,1701,1813,1940,
2068,2196,2339,2482,2641,2814,2988,3163,3366,3570,3778,4000,4236,4520,4848,5099,5399,5622,
5862,6199,6538,6879,7382,7889,8399,8750,9102,9604};

The first entry corresponds to the temperature -25 degree Celsius. The last entry correspons to 45 degree Celsius. So I can just reference the array with a Celsius value (added with 25), and I get the corresponding pressure value. The piece of code below just protects the processor from overrunning memory:

    if(T < -25.0) {
      T = -25.0;
    } else if(T > 45.0) {
    
      T = 45.0;
    }

Last but not least, the absolute humidity is calculated with the code below:

    H = es[(int)(T+ 25)]; // saturated vapor pressure, need 25 degrees for offset
    H *= U; Multiply relative humidity with saturated vapor pressure
    H /= 461.5; // divide by gas constant
    H /= (T+ 273.0); // divide by temperature in Kelvin


U is the relative humidity, T the temperature. Both values come directly from the humidity sensor sht11. 461,5 is the value of the gas constant.

Final words

There are other examples to calculate the absolute humidity, e.g. here. The method I use is simple. The sensors I have in my basement run constantly with this code for more than 2 years. The circuit I desgined with DS2408, atmega16  and sht11 runs stable and I have not noticed any problems until now.

Wednesday, April 10, 2013

Home Automation with Raspberry Pi

In this blog I want to describe how I reduce humidity in my basement without using an energy consuming air dryer. So the control system I implemented is blowing air into the basement, when the air outside of the house is dryer, then inside the basement.

The control system

I have installed two ventilators, five temperature sensors and one humidity sensor in my basement. I installed one humidity sensor outside of my house, as well. The humidity sensors measure the relative humidity, but the sensors translate the values into absolute humidity by doing some math (see here).  Below a picture of the controllers. The right circuit (red) is a controller for the ventillators. It contains two S202 S12 solid state relais to control both ventillators. The controller has a CAN interface to communicate with the left controller. The left controller (green) gathers the temperatures, humidity values from all sensors installed in the basement. It has a CAN interface and 1 wire bus interfaces. Also the circuit has a 24LC1025 eeprom to store the gathered data persistently. I used DS18B20 temperature sensors and connected them directly to the 1 wire bus coming from the controller. Both controllers use at90can128 mikrocontrollers.




The controllers have USB ports. When I wanted to get all measured values onto my laptop, I had to go down to the basement and dump out the eeprom values from USB. In the picture below you can see the humidity values of both humidity sensors from the eeprom dump. The data was gathered from mid 2011 to beginning 2013. In the middle of the curves you can see two lengthy interruptions. I switched off the control system, because I was renovating the basement.



This data gathering was pretty time consuming, so I prefered to have wireless and get the data from the living room or from the office. I needed something that works with wireless on the one hand, and retrieve the sensor data from the controller's eeprom on the other hand. So I came up with the raspberry pi.

Bringing in the raspberry pi

The raspberry has SPI and I2C ports, but no CAN. So I need something which translates SPI (or I2C) to CAN. There are chips available for doing this. Such as the MCP2515 . The chip is widely available, and runs with 3.3V which can be powered from the raspberry pi. Also I needed another chip which is a CAN transceiver. It connects the MCP2515 chip to the CAN bus. I used SN 65HVD230D, which is also widely available and which operates also on 3.3V.

This is the schematic:


I soldered this all together with silver wire and now it looks like this from top, see picture below. On the left upper corner in the picture below you find the raspberry pi. On the buttom right corner you find the components from the schematic above:



On the raspberry runs the wheezy operating system which can be downloaded here. In its forum I described what I have done to install the necessary software to get the raspberry working with SPI, so see CAN for home automation, in case you are interested. The latest wheezy versions have a straightforward way to get the WLAN USB stick running (WIFI config on the wheezy screen). I used the EW-7811UN from EDIMAX.

I created a couple of C functions to retrieve the eeprom data from the basement's controller via CAN. CAN for home automation forum entry describes what I have used to create the functions. The problem with CAN is, that it is rather slow. I cannot dump each time the eeeprom if I want to see the whole curve. This would take more than 5 minutes. So I installed mysql on raspberry and dump all eeprom values into a database. I implemented another script which queries the controller's eeprom once a day for new data and the data is stored into the database. 

The User Interface

As a last step, I needed a user interface. I decided to use a web server and implement a web page to display the data. After little research I found ligtttp and installed it on the raspberry. The decision came mainly because I found somewhere a website which helped me to get this installed easily. lighttpd has perl support, which is convenient for me, because I did not have to learn yet another language like php or python. Also perl has excellent support concerning date calculation needed to setup a time period. So I configured lighttpd to support CGI and wrote the dynamic webpage in perl. I created a login page and a curve display page. The curves I created with gnuplot. It needs to be installed on the raspberry and it works fine, some more information you can find here. Below you can see a snap shot from the webpage:


In the snap shot above you see menu boxes to setup the time period and three more curves. The upper curve above shows the relative humidity values during the time period mid 2011 to beginning 2013. The middle curve shows the temperatures values from the humidity sensors. And the lower curve shows the calculated absolute humidity from both curves. So each time the absolute humdity values of the one curve is above the other, the ventillators are turned on automatically for a short period of time (around 20 min.).

Outlook

For now, I am pretty happy with my implementations. I can go into my office and take a look into my basement sensor values, and make my thoughts about them. But I was thinking, it would be good to have rain information added to the curves above. Rain information could give me a indications if the humidity values are correlated with the rainy weather. They probably are, but it would be nice if that graphic shows this as well. So it seems that there is more to come.