Smarthome data logging with RPi/domoticz/influxdb/grafana

By Atomstar on Sunday 2 December 2018 16:01 - Comments (1)
Categories: Linux, Smarthome, Views: 363

When we got a smart meter and solar panels I wanted to record this data. Along the way I added more sensors and I've extended the software

Goals

  1. Record (smart) meter data and graph it (now including: electricity smart meter via USB, PV via bluetooth, heat meter via machine vision, water meter via line sensor, temperature via EvoHome)
  2. Log data in generic, long-lasting format locally (i.e. no externally hosted solution)
  3. Robust
  4. Interactive graphs on smartphone (domoticz, plot.ly, Highcharts, dygraphs)
  5. Customisable graphs (like grafana)
  6. Cheap/low-power solution (i.e. no Intel NUC)
  7. Versatile wrt interfaces (e.g. bluetooth/wifi/GPIO/USB)
  8. Not used to control or actuate. For that I use COTS products that work and don't crash :p

Architecture

Data model
Units
I use only SI units for storing meter data so I can combine e.g. heat meter (native unit MJ) with smart meter (native unit Wh) with PV (native unit kWh).
  1. Power: W, J/s
  2. Energy: J
  3. Temperature: deg C
  4. Wind: m/s
  5. Wind direction: deg
  6. Voltage: V
  7. Water: liter
  8. Rain: mm
Database scheme
Influxdb can only combine / add / subtract data from the same measurement. Keep this in mind when designing your scheme.

Generic format: <unit>, type=<quantity>, device=<datasource>, subtype=<device qualifier> value1=X value2=Y value3=Z <timestamp>.

Fully worked out, this looks like:
Power,type=elec,device=<kaifa/sma/smartplug> energy=X power=Y
Power,type=heat,device=<landisgyr> energy=X
Power,type=radiation,device=solar energy=X
temperature,type=heating,device=livingroom actual=X setpoint=Y
temperature,type=device,device=sma actual=X
temperature,type=weather,device=XXweatherloc actual=X
wind,type=weather,device=XXweatherloc wind=X
winddirection,type=weather,device= XXweatherloc winddir=X
voltage,type=elec,device=sma, DC=X AC=Y
water,type=potable,device=sensus water=X
rain,type=weather,device=XXweatherloc rain=X

Optionally this could all be in one table (called 'measurement' in influxdb parlance)
Polling cadence
  • Temperature rooms: 1x/5 minutes, resolution: 0.5 deg C
  • Power: 1x/min
  • PV: 1x/min
  • Water: on-demand incremental
  • Heat: 1x/minute
Data usage at 1 measurement/minute is: 1440min/day * 2byte/measurement 3kb/day/metric = 1 MB/yr/metric.
Software architecture
  • Easy access visualization: domoticz
  • Database: influxdb
  • Full visualization only accessibly locally: grafana
  • Selected visualizations remotely accessible on smartphone: Dygraphs
  • Data collection: domoticz with pushers to influxdb, as well as custom scripts to push directly to influxdb.

Implementation

Implementation occurred more or less chronologically in this order. I've written the text including hindsight to fix initial mistakes:
  1. Installing domoticz on Rpi
  2. Extending domoticz with influxdb/grafana
  3. Connecting sensors to Rpi/domoticz

Alternatives

Before settling on the above solution I've considered (and rejected) the following alternative solutions for various reasons:
  • hass.io + influxdb: NOK hass.io has no influxdb export, and appears too heavy (never actually tried)
  • OpenHab: NOK extremely slow on RPi0, briefly tried on RPi3B but still slow.
  • prometheus: not as widely supported as influxdb. Might actually be better because of more powerful query language
  • rpi-monitor / pimatic: appears not mature enough
  • vis.js / Dygraphs / Highcharts / Rickshaw / Plotly: too complicated

Sources

https://blog.jokielowie.c...f-latwe-i-piekne-wykresy/
https://community.openhab.org/t/oh-webserver/36756/10
https://www.circuits.dk/install-grafana-influxdb-raspberry/
https://prometheus.io/doc.../#prometheus-vs.-influxdb
http://dirkgroenen.nl/pro...-endpoints-documentation/
http://blog.videgro.net/2017/09/smart-meter/
https://community.openhab...her-chart-meteogram/39324
https://www.thethingsnetw...d-push-your-iot-data/1788

Volgende: Installing domoticz on raspberry pi (1/3) 02-12 Installing domoticz on raspberry pi (1/3)
Volgende: Reading out TP-Link HS110 on Linux/Raspberry Pi 25-11 Reading out TP-Link HS110 on Linux/Raspberry Pi

Comments


By Tweakers user S0epkip, Monday 3 December 2018 12:55

Dat doe je met DSMR-Reader, niet met Grafana etc. :+

Comment form
(required)
(required, but will not be displayed)
(optional)