Working with Eastern Standard Time (EST) and Eastern Summer Savings (EDT) in R

Question

I have a series of data sets from a continuous water quality monitoring probe with the date and time set at Eastern Standard Time (EST), so there is no Daylight Saving Time (EDT). In R, the fields are recognized as factors when importing the data table from the MS access database, however, when converting using as.POSIXct () the dates and times from 02:00 (24 hours) to 2016-03-13 become NS. This is due to the transition from EST to EDT ... therefore technically 2016-03-13 02:00 does not exist.

Some created data as an example

test<-data.frame(Date=rep(as.Date("2016-03-13"),120),Hour=rep(seq(0,23,1),5),Min=rep(seq(0,60,15),24))

Is there a way to convert a factor or symbol field to a POSIXct field while keeping the time zone designation EST? Alternatively, is there a way to identify and convert the correct date and time to EST and EDT?

I walked around and can't make it work. I tried to convert to GMT (or UTC) and then convert back to EST (tz = "America / New_York"). I understand that this is a constant problem, and people who work with date and time data, especially in R, would like to move away from EDT.

Any help is appreciated ... I am on my way on this.

Answer 1

The problem with using POSIX tz = "America/New_York" is that daylight saving time is taken into account (UTC-4 or UTC-5), even if the base timestamps are stored in US Eastern Standard Time (UTC-5).

You must specify tz as Etc/GMT+5 . From there, it is easy to convert EST, Eastern Local Time and GMT. Note that in R, time zones west of UTC are indicated by a positive offset (see the documentation on time zone names in ?timezone ).

Here are some examples of the data (Daylight Saving Time entered into force at 2:00 Eastern time on 3/16/16):

 StartTime=as.numeric(as.POSIXct("2016-03-11 0:00:00",format="%Y-%m-%d %H:%M",origin="1970-01-01",tz="Etc/GMT+5")) EndTime=as.numeric(as.POSIXct("2016-03-15 0:00:00",format="%Y-%m-%d %H:%M",origin="1970-01-01",tz="Etc/GMT+5")) Interval=15*60 #15-min data.EST=as.POSIXct(seq(from = StartTime,to = EndTime, by=Interval),origin="1970-01-01",tz="Etc/GMT+5") #generate date stamps # convert Eastern Standard Time (in R: GMT+5) to local time (accounts for daylight savings): data.EastCoast<- format(data.EST, tz="America/New_York") # convert Eastern Standard Time (in R: GMT+5) to UTC/GMT: data.UTC<- format(data.EST, tz="GMT") compare.times<-data.frame(data.EST,data.EastCoast,data.UTC) compare.times[(198:203),] data.EST data.EastCoast data.UTC 198 2016-03-13 01:15:00 2016-03-13 01:15:00 2016-03-13 06:15:00 199 2016-03-13 01:30:00 2016-03-13 01:30:00 2016-03-13 06:30:00 200 2016-03-13 01:45:00 2016-03-13 01:45:00 2016-03-13 06:45:00 201 2016-03-13 02:00:00 2016-03-13 03:00:00 2016-03-13 07:00:00 202 2016-03-13 02:15:00 2016-03-13 03:15:00 2016-03-13 07:15:00 203 2016-03-13 02:30:00 2016-03-13 03:30:00 2016-03-13 07:30:00 

Good luck

Answer 2

When converting to POSIX, you need to specify the time zone. See this example:

 test<-data.frame(Date=rep(as.Date("2016-03-13"),96),Hour=rep(seq(0,23,1), each=4),Min=rep(seq(0,45,15))) wrong<-as.POSIXct(paste(test$Date, test$Hour, test$Min), format="%Y-%m-%d %H %M") ans<-as.POSIXct(paste(test$Date, test$Hour, test$Min), format="%Y-%m-%d %H %M", tz="EST") compare<-cbind(test, wrong, ans) 

In the "wrong" vector, the time zone was not specified, therefore NA, but in the second case the "Eastern standard" was specified and the desired result was set.