How to convert a QuerySet django to an array of numpy records?

Question

How to convert a QuerySet django to an array of numpy records?

PS: I know that you can repeat and construct it, but is there any other solution for cleaning?

Answer 1

What I was looking for:

• From QuerySet qs get vlqs (django.db.models.query.ValuesListQuerySet)

  vlqs = qs.values_list ()

• Hidden vlqs for list

  mylist = list (vlqs)

• Create an array of numpy records

  r = np.core.records.array (mylist, names = 'field1, field2, field3') // Names are model fields

Answer 2

import numpy as np qs = MyModel.objects.all() vlqs = qs.values_list() r = np.core.records.fromrecords(vlqs, names=[f.name for f in MyModel._meta.fields]) 

It uses the QuerySet iterator directly and avoids the time step and garbage to create a python list. It also uses MyModel._meta.fields to get the actual field names from the model, as described in Get Model Fields in Django

If you just want one field (for example, the “votes” field of the model) to be extracted into a one-dimensional array, you can do:

 vlqs = qs.values_list('votes', flat=True) votes = np.fromiter(vlqs, numpy.dtype('int_')) 

Answer 3

This is similar to the question: "How do I convert the contents of my refrigerator to lunch?" It depends on what you have in the refrigerator and what you want to eat. The short answer (the equivalent of saying “culinary”) is to iterate over the query, build objects of any composite data types that you want to create as an instance of the array (usually iterable and a dictionary). The long answer depends on what you really would like to accomplish.

Answer 4

what can you do:

 [index[0] for index in qs.values_list('votes')] 

and ready ... xd

Answer 5

If you want to get all your objects and create a numpy array with objects as elements of the array:

 import numpy as np qs = MyModel.objects.all() numpy_array = np.array(list(qs)) 

According to my work, I am using something as shown below:

 import numpy as np qs = MyModel.objects.values_list('id','first_name','last_name').filter(gender='male').order_by('id') numpy_array = np.array(list(qs)) 

The rows of the array correspond to the records and columns of the array correspond to the values ​​that I defined above (id, first name, last name).

Answer 6

And to put it in a neat little function that you just pass any Queryset Django to:

 import pandas as pd import numpy as np def qs_to_df(qs): """ QuerySet to DataFrame """ Model = qs.model np_array = np.core.records.fromrecords(qs.values_list(), names=[f.name for f in Model._meta.fields]) return pd.DataFrame(np_array) 

Answer 7

Disabling the @CpILL response allows you to turn most requests into an array of numpy entries as follows:

 def qs_to_ra(qs, *args): """ Turn most querysets directly into a numpy record array :param qs: django queryset :param args: takes a list of field names to specify :return: numpy.recarray """ model = qs.model if args: return np.core.records.fromrecords(qs.values_list(*args), names=args) return np.core.records.fromrecords(qs.values_list(), names=[f.name for f in model._meta.fields]) 

You can also turn them directly into a pandas framework, for example:

 def qs_to_df(qs, *args): """ Turn most querysets directly into a pandas dataframe. :param qs: django queryset :param args: takes a list of field names to specify :return: pandas.DataFrame """ model = qs.model if args: return pd.DataFrame.from_records(list(qs.values_list(*args)), columns=args) return pd.DataFrame.from_records(list(qs.values_list()), columns=[f.name for f in model._meta.fields])