Geek Answers Handbook

Datetime in pandas dataframe will not subtract from each other

I am trying to find the time difference between two columns in a pandas frame as in datetime format.

Below is some data in my data frame and the code I used. I checked three times that these two dtypes columns are datetime64.

My details:

date_updated                  date_scored 
2016-03-30 08:00:00.000       2016-03-30 08:00:57.416  
2016-04-07 23:50:00.000       2016-04-07 23:50:12.036

My code is:

data['date_updated'] = pd.to_datetime(data['date_updated'], 
format='%Y-%m-%d %H:%M:%S')
data['date_scored'] = pd.to_datetime(data['date_scored'], 
format='%Y-%m-%d %H:%M:%S')
data['Diff'] =  data['date_updated'] - data['date_scored']

The error message I get is:

TypeError: data type "datetime" not understood

Any help would be appreciated, thanks!

My work around the solution:

for i in raw_data[:10]:
scored = i.date_scored
scored_date =  pd.to_datetime(scored, format='%Y-%m-%d %H:%M:%S')
if type(scored_date) == "NoneType":
    pass
elif scored_date.year >= 2016:
    extracted = i.date_extracted
    extracted =  pd.to_datetime(extracted, format='%Y-%m-%d %H:%M:%S')
    bank = i.bank.name
    diff = scored - extracted
    datum = [str(bank), str(extracted), str(scored), str(diff)]
    data.append(datum)
else:
    pass
+6
source share
3 answers

I ran into the same error using the above syntax (worked on another machine):

data['Diff'] =  data['date_updated'] - data['date_scored']

It worked on my new machine with:

data['Diff'] =  data['date_updated'].subtract(data['date_scored'])
+8
source

. , to_datetime , .

import io
import pandas as pd
# Paste the text by using of triple-quotes to span String literals on multiple lines
zz = """date_updated,date_scored
2016-03-30 08:00:00.000,       2016-03-30 08:00:57.416  
2016-04-07 23:50:00.000,       2016-04-07 23:50:12.036"""

data = pd.read_table(io.StringIO(zz), delim_whitespace=False, delimiter=',')

data['date_updated'] = pd.to_datetime(data['date_updated'])
data['date_scored'] = pd.to_datetime(data['date_scored'])
data['Diff'] =  data['date_updated'] - data['date_scored']

print(data)
#          date_updated             date_scored                     Diff
# 0 2016-03-30 08:00:00 2016-03-30 08:00:57.416 -1 days +23:59:02.584000
# 1 2016-04-07 23:50:00 2016-04-07 23:50:12.036 -1 days +23:59:47.964000
+1

You need to update pandas. I just ran into the same problem with the old code that was used to run without problems. After updating pandas (0.18.1-np111py35_0) to a newer version (0.20.2-np113py35_0), the problem was resolved.

+1
source

More articles:


All Articles