Run program (R) in Python to perform the operation (execute the script)

Question

I am linking the execution of an R script from python, perfectly displaying and saving the results. Using rpy2 was a bit difficult, so I thought I would just call R directly. I have the feeling that I will need to use something like "os.system" or "subprocess.call", but I am having difficulty deciphering the manuals of the modules.

Here's the R script "MantelScript", which uses a special stat test to compare two distance matrices at a time (distmatA1 and distmatB1). This works in R, although I haven't added iteration bits yet to read and compare a bunch of files in pairs ( I really need help, also btw! ):

library(ade4) M1<-read.table("C:\\pythonscripts\\distmatA1.csv", header = FALSE, sep = ",") M2<-read.table("C:\\pythonscripts\\distmatB1.csv", header = FALSE, sep = ",") mantel.rtest(dist(matrix(M1, 14, 14)), dist(matrix(M2, 14, 14)), nrepet = 999) 

Here is the corresponding bit of my python script that reads some previously formulated lists and draws matrices to compare them using this Mantel test (it should pull the first matrix from identityA and sequentially compare it with each matrix in identifier B, then repeat with the second matrix from identifier B, etc.). I want to save these files and then call the R program to compare them:

 # windownA and windownB are lists containing ascending sequences of integers # identityA and identityB are lists where each field is a distance matrix. z = 0 v = 0 import subprocess import os for i in windownA: M1 = identityA[i] z += 1 filename = "C:/pythonscripts/distmatA"+str(z)+".csv" file = csv.writer(open(filename, 'w')) file.writerow(M1) for j in windownB: M2 = identityB[j] v += 1 filename2 = "C:/pythonscripts/distmatB"+str(v)+".csv" file = csv.writer(open(filename2, 'w')) file.writerow(M2) ## result = os.system('R CMD BATCH C:/R/library/MantelScript.R') - maybe something like this?? ## result = subprocess.call(['C:/R/library/MantelScript.txt']) - or maybe this?? print result print ' ' 

Answer 1

If your R script has only side effects, but if you want to continue processing the results using Python, you would still be better off using rpy2.

 import rpy2.robjects f = file("C:/R/library/MantelScript.R") code = ''.join(f.readlines()) result = rpy2.robjects.r(code) # assume that MantelScript creates a variable "X" in the R GlobalEnv workspace X = rpy2.rojects.globalenv['X'] 

Answer 2

Stick to it.

 process = subprocess.Popen(['R', 'CMD', 'BATCH', 'C:/R/library/MantelScript.R']) process.wait() 

When the wait() function returns, the .R file is complete.

Note that you must write your .R script to create a file that your Python program can read.

 with open( 'the_output_from_mantelscript', 'r' ) as result: for line in result: print( line ) 

Do not spend a lot of time connecting the pipeline.

Invert time when creating the base process "Python spawns R".

You can add to this later.

Answer 3

If you are interested in invoking the R subprocess from Python at all.

 #!/usr/bin/env python3 from io import StringIO from subprocess import PIPE, Popen def rnorm(n): rscript = Popen(["Rscript", "-"], stdin=PIPE, stdout=PIPE, stderr=PIPE) with StringIO() as s: s.write("x <- rnorm({})\n".format(n)) s.write("cat(x, \"\\n\")\n") return rscript.communicate(s.getvalue().encode()) if __name__ == '__main__': output, errmsg = rnorm(5) print("stdout:") print(output.decode('utf-8').strip()) print("stderr:") print(errmsg.decode('utf-8').strip()) 

Better to do it through rscript.

Answer 4

Given what you are trying to do, a pure R-solution might be more accurate:

 file.pairs <- combn(dir(pattern="*.csv"), 2) # get every pair of csv files in the current dir 

A pair is a column in a 2xN matrix:

 file.pairs[,1] [1] "distmatrix1.csv" "distmatrix2.csv" 

You can run the function in these columns with apply (with the option "2", which means "act on the columns"):

 my.func <- function(v) paste(v[1], v[2], sep="::") apply(file.pairs, 2, my.func) 

In this example, my.func simply glues the two file names together; you can replace this with a function that performs the Mantel Test, something like (untested):

 my.func <- function(v){ M1<-read.table(v[1], header = FALSE, sep = ",") M2<-read.table(v[2], header = FALSE, sep = ",") mantel.rtest(dist(matrix(M1, 14, 14)), dist(matrix(M2, 14, 14)), nrepet = 999) }