Geek Answers Handbook

Delete rows with empty lists from pandas data frame

I have a data frame with some columns with empty lists and others with row lists:

donation_orgs donation_context 0 [] [] 1 [the research of Dr. ...] [In lieu of flowers , memorial donations ...]

I am trying to return a dataset without any rows where there are empty lists.

I tried just checking for null values:

 dfnotnull = df[df.donation_orgs != []] dfnotnull

and

 dfnotnull = df[df.notnull().any(axis=1)] pd.options.display.max_rows=500 dfnotnull

And I tried iterating over and checking existing values, but I think the lists do not return Null or None, as I thought they were:

 dfnotnull = pd.DataFrame(columns=('donation_orgs', 'donation_context')) for i in range(0,len(df)): if df['donation_orgs'].iloc(i): dfnotnull.loc[i] = df.iloc[i]

All three of the above methods simply return each row in the original data frame. =

+6
source share
4 answers

You can try slicing as if the data frame were rows instead of lists:

 import pandas as pd df = pd.DataFrame({ 'donation_orgs' : [[], ['the research of Dr.']], 'donation_context': [[], ['In lieu of flowers , memorial donations']]}) df[df.astype(str)['donation_orgs'] != '[]'] Out[9]: donation_context donation_orgs 1 [In lieu of flowers , memorial donations] [the research of Dr.]
+5
source

I know this is not the best solution (and not even nice), but I think it works:

 import numpy as np df = pd.read_csv('yoursamplefile.csv') df = df.replace('[]', np.nan) df.dropna(how='any', inplace=True)

Basically, I would replace empty list lists with NaN , and then reset according to your rule. This works by reading a sample file from a file; in fact, I don't know if the dataframe contains empty lists like "[]" or just empty values.

+1
source

You can use the following single line:

 df[(df['donation_orgs'].str.len() != 0) | (df['donation_context'].str.len() != 0)]
0
source

To avoid converting to str and actually use list s, you can do this:

 df[df['donation_orgs'].map(lambda d: len(d)) > 0]

It displays the donation_orgs column in the length of the lists of each row and saves only those that have at least one element , filtering out empty lists.

He returns

 Out[1]: donation_context donation_orgs 1 [In lieu of flowers , memorial donations] [the research of Dr.]

as was expected.

0
source

More articles:


All Articles