Convert Pandas dataframe to Dask dataframe

Question

Suppose I have a pandas dataframe like:

df=pd.DataFrame({'a':[1,2,3],'b':[4,5,6]}) 

When I convert it to a dask dataframe, then what should the name and divisions parameter consist of:

 from dask import dataframe as dd sd=dd.DataFrame(df.to_dict(),divisions=1,meta=pd.DataFrame(columns=df.columns,index=df.index)) 

TypeError: init () missing 1 required positional argument: 'name'

Edit : Suppose I created a pandas framework, for example:

 pd.DataFrame({'a':[1,2,3],'b':[4,5,6]}) 

Similarly, how to create a dask dataframe, since it needs three additional arguments like name,divisions and meta .

 sd=dd.Dataframe({'a':[1,2,3],'b':[4,5,6]},name=,meta=,divisions=) 

Thanks for your reply.

Answer 1

I think you can use dask.dataframe.from_pandas :

 from dask import dataframe as dd sd = dd.from_pandas(df, npartitions=3) print (sd) dd.DataFrame<from_pa..., npartitions=2, divisions=(0, 1, 2)> 

EDIT:

I find a solution :

 import pandas as pd import dask.dataframe as dd from dask.dataframe.utils import make_meta df=pd.DataFrame({'a':[1,2,3],'b':[4,5,6]}) dsk = {('x', 0): df} meta = make_meta({'a': 'i8', 'b': 'i8'}, index=pd.Index([], 'i8')) d = dd.DataFrame(dsk, name='x', meta=meta, divisions=[0, 1, 2]) print (d) dd.DataFrame<x, npartitions=2, divisions=(0, 1, 2)>