Simple structure implementation for efficient storage of tuple list

Question

I need to create a list of the following type

[(latitude, longitude, date), ...]

where latitude and longitude are floats, and date is an integer. I'm running out of memory on my local machine because I need to store about 60 million of these tuples. What is the most efficient memory (and at the same time simple to implement) way of representing these tuples in python?

The precision of latitude and longitude should not be so large (sufficient to represent values ​​such as -65.100234), and integers should be large enough to handle UNIX timestamps.

I used swig before to define "c-structs" that are generally much more memory efficient than their python, but this is hard to implement ... maybe there is some scipy or numpy way to declare such tuples that use less memory .. Any ideas?

Answer 1

NumPy, numpy.recarray. 8 , , , , 32- , , 60 1,2 . , NumPy , .

:

# Create an uninitialised array with 100 records
a = numpy.recarray(100,
                   formats=["f8", "f8", "i4"],
                   names=["latitude", "longitude", "date"])
# initialise to 0
a[:] = (0.0, 0.0, 0)
# assign a single record
a[0] = (-65.100234, -38.32432, 1309351408)
# access the date of the first record
a[0].date
# access the whole date column
a.date

NumPy, ctypes ctypes, , NumPy, , SWIG.