Receive online data every hour in R

Question

I would like to receive observatory data every hour when they update the forecast.

Below is my one-time data retrieval code.

library(RCurl) web <- getURL("http://www.hko.gov.hk/contente.htm") web <- unlist(strsplit(web, "\r\n")) head(web) temp <- unlist(strsplit(web[1245], "</span>")) MINtemp <- vector() MAXtemp <- vector() for (i in 1:9){ mintemp <- substr(temp[2*i-1], nchar(temp[2*i-1])-1, nchar(temp[2*i-1])) mintemp <- as.numeric(mintemp) MINtemp <- append(MINtemp, mintemp) maxtemp <- substr(temp[2*i], nchar(temp[2*i])-1, nchar(temp[2*i])) maxtemp <- as.numeric(maxtemp) MAXtemp <- append(MAXtemp, maxtemp) } status <- strsplit( substring(web[1242],12), "</a></td><td align") status <- substring(unlist(status), 178) weather <- vector() for (i in 1:9){ status[i] <- unlist(strsplit(status[i], "width"))[1] weather <- append(weather, substr(status[i], 1, nchar(status[i])-3 ) ) } RH <- unlist(strsplit(web[1248], "</span>")) MINRH <- vector() MAXRH <- vector() for (i in 1:9){ minRH <- substr(RH[2*i-1], nchar(RH[2*i-1])-1, nchar(RH[2*i-1])) minRH <- as.numeric(minRH) MINRH <- append(MINRH, minRH) maxRH <- substr(RH[2*i], nchar(RH[2*i])-1, nchar(RH[2*i])) maxRH <- as.numeric(maxRH) MAXRH <- append(MAXRH, maxRH) } forecast <- paste("+", 1:9, "day(s)", sep=" ") current <- as.character(rep(Sys.time(),9)) DATA <- data.frame(cbind(current,forecast,MINtemp, MAXtemp, MINRH, MAXRH, weather)) DATA 

Received Data

 > DATA current forecast MINtemp MAXtemp MINRH MAXRH weather 1 2014-05-04 08:37:55 + 1 day(s) 21 25 80 95 Cloudy with a few showers and thunderstorms. Showers will be more frequent later 2 2014-05-04 08:37:55 + 2 day(s) 22 25 75 90 Cloudy with showers. A few squally thunderstorms at first 3 2014-05-04 08:37:55 + 3 day(s) 21 24 75 95 Cloudy with a few showers 4 2014-05-04 08:37:55 + 4 day(s) 22 25 80 95 Cloudy with a few showers 5 2014-05-04 08:37:55 + 5 day(s) 23 26 80 95 Cloudy with showers and a few squally thunderstorms 6 2014-05-04 08:37:55 + 6 day(s) 23 26 80 95 Cloudy with showers. Showers will be heavy at times with squally thunderstorms 7 2014-05-04 08:37:55 + 7 day(s) 22 25 80 95 Cloudy with showers and squally thunderstorms 8 2014-05-04 08:37:55 + 8 day(s) 22 25 70 95 Mainly cloudy with a few showers 9 2014-05-04 08:37:55 + 9 day(s) 22 26 70 90 Mainly cloudy 

I would like the R script to execute on its own every hour. Then use rbind(DATA, data) to accumulate the data set. I am looking for a similar topic using CMD R BATCH. Although I can do it inside R using Sys.sleep() and while(substr(Sys.time(), 15,16)=="00") ?

I was looking for a similar task for planning this link

What I found Rscript.exe in the directory C:\Program Files\R\R-3.0.2\bin\Rscript.exe

and I saved my Rscipt to D:\mydocument\test.r So far I have not figured out how to complete the task.

Answer 1

I think a reasonable course of action is to write a script that does everything you want to do (data loading, cleaning, storing, ...) and using something like:

• Task Scheduler for Windows link
• Cronjob under Linux link
• ICal, crontab, CronniX, Maintidget, Macaroni and MacJanitor, automatic scheduled task, Launchd Editor, launchctl on Mac link

... planning means starting it at regular intervals instead of having one R session running and waiting all the time, for example. Sys.sleep(60*60) .

Answer 2

You can Sys.sleep() use Sys.sleep() , but it smells like bad code .

Instead, configure the cron job to update the code every time. Then your script is simple and more reliable.