Geek Answers Handbook

Pandas still gets SettingWithCopyWarning even after using .loc

At first I tried to write code that would look like this:

import numpy as np import pandas as pd np.random.seed(2016) train = pd.DataFrame(np.random.choice([np.nan, 1, 2], size=(10, 3)), columns=['Age', 'SibSp', 'Parch']) complete = train.dropna() complete['AgeGt15'] = complete['Age'] > 15

Having got SettingWithCopyWarning, I tried using .loc:

 complete.loc[:, 'AgeGt15'] = complete['Age'] > 15 complete.loc[:, 'WithFamily'] = complete['SibSp'] + complete['Parch'] > 0

However, I still get the same warning. What gives?

+15
source share
2 answers

Note. Starting from version 0.24 for pandas is_copy deprecated and will be removed in the next version. Although the private _is_copy attribute exists, an underscore indicates that this attribute is not part of the public API and therefore should not depend on it. Therefore, in the future, it seems that the only right way to silence SettingWithCopyWarning is to do it globally:

 pd.options.mode.chained_assignment = None

When complete = train.dropna() are executed, dropna can return a copy, so due to excessive care, Pandas set complete.is_copy to Truthy values:

 In [220]: complete.is_copy Out[220]: <weakref at 0x7f7f0b295b38; to 'DataFrame' at 0x7f7eee6fe668>

This allows Pandas to warn you later when it is complete['AgeGt15'] = complete['Age'] > 15 that you can change the copy that will not affect the train . For starters, this can be a useful warning. In your case, it seems you have no intention of modifying train indirectly by modifying complete . Therefore, a warning is just a pointless annoyance in your case.

You can turn off the warning by setting

 complete.is_copy = False # deprecated as of version 0.24

This is faster than creating the actual copy, and SettingWithCopyWarning in the bud (at the point where _check_setitem_copy ):

 def _check_setitem_copy(self, stacklevel=4, t='setting', force=False): if force or self.is_copy: ...

If you are really sure that you know what you are doing, you can disable SettingWithCopyWarning globally with

 pd.options.mode.chained_assignment = None # None|'warn'|'raise'

An alternative way to silence a warning is to make a new copy:

 complete = complete.copy()

However, you may not want to do this if the size of the DataFrame is large, since copying can take up a considerable amount of time and memory, and it is completely pointless (except to silence the warning) if you know that complete is already copy,

+17
source

I resolve this by creating a copy of the dataframe:

 complete = train.copy()
+2
source

More articles:


All Articles