Geek Answers Handbook

How to filter shift +/- 1 day in Pandas?

Hi guys, suppose I have timers data. How to filter data that occurs only after 1 day?

Suppose the data

date name 2015-04-03 A 2015-04-04 A 2015-04-05 A 2015-04-03 B

I want to do something like

 df[df.shift(1).contains(df.name) or df.shift(-1).contains(df.name)]

which would give me

 date name 2015-04-03 A 2015-04-04 A 2015-04-05 A

How to do it in pandas?

+5
source share
3 answers

You want to enclose your conditions in parentheses and use bitwise | instead of or :

 In [83]: df[(df['name'].shift(1) == df['name']) | (df['name'].shift(-1) == df['name']) ] Out[83]: date name 0 2015-04-03 A 1 2015-04-04 A 2 2015-04-05 A
0
source

I would use value_counts for the date:

 vc = df.date.value_counts()

Then I would, if the data set is small, I would use .isin :

 df[df.date.isin(vc[vc == 1].index.tolist())] date name 1 2015-04-04 A 2 2015-04-05 A

If the dataset is larger, I would use the merge operation:

 df_singles = df.merge(left_on='date',right=pd.DataFrame(vc[vc == 1]), right_index=True) del df_singles[0] date name 1 2015-04-04 A 2 2015-04-05 A
+1
source

Convert the date column to a datetime object. Then you can calculate the difference between the days. After that, you can filter the number of days differences you want to keep.

 data['date'] = pd.to_datetime(data['date']) data['Diff'] = data['date'].diff() print data date name Diff 0 2015-04-03 A NaT 1 2015-04-04 A 1 days 2 2015-04-05 A 1 days 3 2015-04-03 B -2 days data_filtered = data[data['Diff'] == '1 days'] del data_filtered['Diff'] print data_filtered date name 1 2015-04-04 A 2 2015-04-05 A
0
source

More articles:


All Articles