Geek Answers Handbook

Choosing TSDB for a one-time installation for a smart home

I am creating a one-time data collection unit for a smart home. It is expected that it will be launched on a machine with a raspberry peak (~ 1 GB), processing about 200 thousand data points per day (each of 64-bit ints). We work with vanilla MySQL, but performance starts to crash, especially for queries on the number of records for a certain period of time.

As I understand it, this is basically exactly what the time series databases are for. In any case, the unusual thing in my situation is that the amount is relatively low, as well as the amount of available RAM.

A quick look at Wikipedia is offered by OpenTSDB, InfluxDB, and possibly BlueFlood. OpenTSDB offers 4G of RAM, although this may be for high-volume settings. InfluxDB actually mentions the readings of the sensors, but I can not find much information about what resources are needed.

So, here is my real question: are there any obvious red flags that will make any of these systems unacceptable for the project being described?

I understand that this is an invitation to the flame, so I hope that people will keep it on the bright and useful side. Thank you very much in advance!

+8
mysql raspberry-pi influxdb raspberry-pi2 opentsdb
source share
4 answers

InfluxDB must be in order with 1 GB of RAM on this volume. Built-in sensors and low-power devices such as the Raspberry Pi are definitely the primary use case, although we haven't done much testing with the latest beta versions other than compiling on ARM.

InfluxDB 0.9.0 has just been released, and 0.9.x should be available in our Hosted environment in a few weeks. Younger instances have 1 GB of RAM and 1 CPU equivalent, so they are a reasonable proxy for your Pi performance, and the free trial lasts two weeks.

If you have more specific questions, please contact us at infuxdb@googlegroups.com or support@influxdb.com and we will see that we can help.

+3
source share

You tried with OpenTSDB. We use OpenTSDB for nearly 150 homes to collect smart meter data, where data is collected every 10 minutes. that is, many data points in one day. But we did not test it in raspberry pi. For Raspberry pi, OpenTSDB can be quite difficult, since it needs to run a web server, HBase, and Java. For suggestions only. You can use Raspberry pi as a data center for smart home and send data from the Raspberry server to the server and store all points on the server. Later on the server, you can do whatever you want, for example, aggregation, or perform statistical analysis, etc. You can then send the results back to the smart hub.

+1
source share

ATSD supports the ARM architecture and can be installed on the raspberry Pi 2 to store sensor data. Ubuntu or Debian is currently required. Make sure that the device has at least 1 GB of RAM and an SD card with a high write speed (60 MB / s or more). The size of the SD card depends on how much data you want to save and how long we recommend at least 16 GB, you should plan ahead. It is also recommended to use a backup battery to protect against failures and disrespectful stops.

Here you can find a detailed guide to setting up the temperature / humidity sensor paired with your Arduino. Using the manual, you can transfer sensor data to the ATSD using the MQTT or TCP protocol. Open source thumbnails included.

+1
source share

Try VictoriaMetrics . It should work on systems with low RAM, such as the Raspberry Pi. Check out these instructions on how to create one for ARM .

VictoriaMetrics has the following additional benefits for small systems:

  • It is easy to configure and maintain, since it has no external dependencies, and all configuration is performed using several command line flags.
  • It is optimized for low CPU utilization and low I / O load of persistent storage.
  • It compresses data well, so it uses a small amount of persistent storage space compared to other solutions.
0
source share

More articles:


All Articles