Geek Answers Handbook

NoSQL and weather data

So, here is this new cool thing, these NoSQL databases. And so there is my data: lines of lines of lines of meteorological data: values ​​representing certain measurements at a particular station (indicated by the WMO number, not coordinates), at a specific time.

Not every station measures every parameter, and not every parameter is measured all the time.

I store this data (30 year value of hourly values, resulting in ~ 1 billion values) currently in MySQL. The constant growth and unsurpassed addition of even more data give me a little headache.

Reading about NoSQL-based systems that seem to scale quite easily, I was wondering if NoSQL is a viable data storage concept for meteorological data. Do you have any experience?

Update: I forgot about typical queries: most queries require data on the time axis: Ie give me the temperature of the station 066310 from 01/01/2010 00:00 to 01/01/2010 00:00.

Or: give me the latest values ​​of all the parameters of a particular station.

+5
source share
3 answers

NoSQL can come in handy when your data structure is pretty simple (like a simple keystore) / predictable, and you don't need relational integrity or the need for special and / or advanced queries.

, .

, . , NoSQL.

PostgreSQL MySQL ( ) .

+2

, .

  • nosql, Cassandra (, , )
  • "nosql" ( ( MySQL) )
  • Cassandra ( TimeUUID)
  • , , MySQL, , .
  • MySQL, , -? ( ).
  • Things like indexes and search are what you would need to implement manually in many nosql repositories if that scares you, possibly with sql.

Thank you for listening;)

+1
source

More articles:


All Articles