Recording weather with Arduino, Elasticsearch and Kibana

What do half a Bunnings weather station, an Arduino, a NodeMCU (or 3), Elasticsearch, a fishtank and swimming pool have to do with each other?

Nothing unless you like to monitor and measure pretty much everything around the house.  I do, and ever since I learned to fly, I have had a fascination with the weather (I am told boaties suffer from a related affliction).

I could just buy a weather station, but where's the fun in that when for the same amount of money (OK, maybe a little more) I can make my own and bore people senseless when I go on and on about it?

Actually, there is a practical purpose to this as well.  I wanted to teach myself about Elastic and Kibana.  Now you can wade through dry tutorials, but nothing in my opinion beats working with real live data.  Especially if the data is of interest to you.

The design

What do you want to measure?

This is actually pretty important (not to mention obvious), but before embarking too far; take a good hard look at what you want to achieve, because you are going to hit limitations along the way.  My list came down to:

• Air temperature (measured and "feels like")
• Relative humidity
• Dew point
• Barometric pressure
• UV level
• Light intensity
• Wind speed
• Wind direction
• Rainfall

I also wanted to be able to measure a subset of these values (chiefly temperature and humidity) from remote locations and collate this information somehow.

How am I going to measure all of this?

There are so many low cost modules out there now which make the job much easier than a few years ago.  There's no doubt I would end up with a bunch of sensors, but at only a couple of dollars each (mostly) I get a lot of data for my money.  Let's look at the solution:

Temperature, Humidity, Pressure

There's a wealth of choices here.  For my aquarium monitor, I use the Dallas Semi DS18B20 attached to a Raspberry Pi.  They work well, and when supplied encased in stainless steel are perfect for immersion.

The DHT/MHT series (DHT11, DHT22, etc) give me relative humidity as well.  They are cheap, but rather slow and fiddly to drive.

I settled in the Bosch BME280.  It is supplied on a 10mm x 10mm PCB, gives me temperature, barometric pressure, humidity and even altitude all over an I2C connection.

Dew Point

The Dew point is used as an indication of general comfort, as well as other useful things in the petrochemical industry.  But mostly it's about the relative comfort or otherwise in the present environment.  We don't need a sensor for this if we have temperature and relative humidity.  All we need is a bit of math to work it out. Wikipedia explains it better than me.

Frost Point

In doing my research I found a little-quoted figure called the Frost point. If we know the temperature and dew point we can work out the temperature at which frost would form in the current climate.  As we get only a handful of frosts per year where we live it's not much use, but as my wife's plants don't like frost, and I can get it for the price of a little math, well why not?

UV Level

If you live in Australia, sooner or later you're going to worry about UV exposure.  I'm sure other countries are not immune either, but it's becoming common conversation these days/

Electro-Optical Components make a cheap analogue sensor called the SS12SD.  These can be obtained for around $A2.00 each on a breakout board.

Light intensity

This is related to, but not the same as the UV sensor.  Primarily it is an input to the "feels like" calculation.

In this case I am using a BH1750 sensor, which is commonly used in phones, laptops and tablets to adjust screen lighting to suit ambient conditions.

It connects over the I2C buss and handily gives a reading in Lumens so you can avoid any math.

Wind Speed

Now, you can't have a weather station without one of those turns ping-pong ball things (it's an anemometer you clown), because it gives you scientific credibility as well as the current wind speed.he bulk of them are and electro-optical or magnetic arrangement which gives you 1 or 2 pulses per revolution.  If you buy a dedicated unit (such as from Davis Instruments), there will be calibration data included.  However if you're recycling one like yours truly, you will need to work out how many metres per revolution.

Wind Direction

If you're using an off-the-shelf unit, they generally fall into one of two categories.  Either a potentiometer (variable resistor) or hall effect switches.

The potentiometer devices may have a "dead spot" in their output, and are becoming less common.  

The hall effect models use a set of hall-effect switches to switch fixed resistors into circuit, and give you an effective variable resistance without the dead spot.  However, the resolution is generally limited to 45 degrees or so.  This is quite sufficient for what I need.

Rainfall

Simplest choices are "tipping bucket" sensors or an optical device such as the Hydreon RG-11.  Both have strengths and weaknesses, and most commercial stations use the tipping bucket types, but you have to mount them securely to prevent false readings.

The final design

The controller

While all this is going on, I have to decide what controller I am going to use.  The choice is driven from a couple of perspectives, chiefly functionality and price.

I had used a Raspberry Pi in the past, but it doesn't have analogue I/O.  I could add an I2C ADC module, but the Pi is not that cheap to begin with.

The NodeMCU is another interesting option.  It has inbuilt WiFi and is very cheap, but it only gives me a single analogue input without add-on converters.

I happened to have a Freetronics "Ether10", which is and Arduino Uno compatible board with a Wiznet Ethernet interface included and it supports PoE (Power Over Ethernet), which meant one less cable I needed to run.

Sensor Channels

My final set of inputs is as follows:

• Analogue x 2 (Wind direction and UV level)
• I2C x 2 (Temp/RH/Pressure and Light intensity)
• Pulse input x 2 (Rainfall and Wind speed)
• Timer interrupt (more on this later)

Some of the sensor solution presented itself on a weekend trip to Bunnings when I found a damaged Holman iWeather unit in the bargain bin.  Display was knackered, but the sensors seem to be in good order, and for the princely sum of $25 instead of $98, it was mine.  Actually, $98 didn't seem to bad in the first place, even if I did have to ditch the display unit.

This appears to be the same design sensor unit as used by Oregon Scientific and a few other manufacturers as well.  As best as I can tell it is originally manufactured by Mi*Sol out of China.

Parts List
The following is the list of components in my original station:

Freetronics Ether10 Board x 1 (You could substitute an Arduino Uno + Ethernet Shield)
Freetronics "Screw Shield" x 1 (Optional, but recommended)
Holman iWeather outdoor assembly, or similar
Bosch BME280 on a breakout board
BH1750 Light intensity sensor on a breakout board
SS12SD UV sensor on a breakout board
Power supply to suit (or use PoE injector)
Legrand weather-proof enclosure x 1
12mm Weather-proof cable glands x 2
100V/0.25W Zener diodes x 6
Blue LED x 1
Orange LED x 1
1X/0.25W Resistors x 6
220-ohm Resistors x 2
10-way terminal strip x 2
Hookup wire
Screws, spaces, nuts all M3

Next:  Assembling the Station

Later: System Software