Getting all rows with a value of NaN

Question

I have a table with a column with some NaN values ​​in it:

ABCD 2 3 2 Nan 3 4 5 5 2 3 1 Nan 

I would like to get all the rows where D = NaN. How can i do this?

Answer 1

Creating df for illustration (containing Nan)

 In [86]: df =pd.DataFrame({'a':[1,2,3],'b':[3,4,5],'c':[np.nan, 4,5]}) In [87]: df Out[87]: abc 0 1 3 NaN 1 2 4 4 2 3 5 5 

Checking which indexes are null for column c

 In [88]: pd.isnull(df['c']) Out[88]: 0 True 1 False 2 False Name: c, dtype: bool 

Checking which indexes are not null for column c

 In [90]: pd.notnull(df['c']) Out[90]: 0 False 1 True 2 True Name: c, dtype: bool 

Choosing df strings where c is not null

 In [91]: df[pd.notnull(df['c'])] Out[91]: abc 1 2 4 4 2 3 5 5 

Select df strings, where c is null

 In [93]: df[pd.isnull(df['c'])] Out[93]: abc 0 1 3 NaN 

Choosing rows of a column c df, where c is not null

 In [94]: df['c'][pd.notnull(df['c'])] Out[94]: 1 4 2 5 Name: c, dtype: float64 

Answer 2

For a solution that does not include pandas, you can do something like:

 goodind=np.where(np.sum(np.isnan(y),axis=1)==0)[0] #indices of rows non containing nans 

(or negation if you want rows with nan) and use indexes to process the data. I'm not sure that sum is the best way to combine np.any , but np.any and np.all don't seem to have an axis parameter, so this is the best way I've found.