Pandas.isnull () not working on decimal type?

Question

Am I missing something or do we have a problem with pandas.isnull ()?

>>> import pandas as pd
>>> import decimal
>>> d = decimal.Decimal('NaN')
>>> d
Decimal('NaN')
>>> pd.isnull(d)
False
>>> f = float('NaN')
>>> f
nan
>>> pd.isnull(f)
True
>>> pd.isnull(float(d))
True

The problem is that I have a dataframe with decimal.Decimal values ​​in it, and df.dropna () does not remove NaN for this reason ...

Answer 1

Yes, this is not supported, you can use a property that is NaNnot equal to the one itself, which still works for types Decimal:

In [20]:
import pandas as pd
import decimal
d = decimal.Decimal('NaN')
df = pd.DataFrame({'a':[d]})
df

Out[20]:
     a
0  NaN

In [21]:    
df['a'].apply(lambda x: x != x)

Out[21]:
0    True
Name: a, dtype: bool

So you can do:

In [26]:
df = pd.DataFrame({'a':[d,1,2,3]})
df[df['a'].apply(lambda x: x == x)]

Out[26]:
   a
1  1
2  2
3  3