Python redirecting popen stream to python function

Question

I am new to python programming. I have this problem: I have a list of text files (compressed and not), and I need: - connect to the server and open them - after opening the file I need to take its contents and pass it to another python function that I wrote

def readLogs (fileName): f = open (fileName, 'r') inStream = f.read() counter = 0 inStream = re.split('\n', inStream) # Create a 'list of lines' out = "" # Will contain the output logInConst = "" # log In Construction curLine = "" # Line that I am working on for nextLine in inStream: logInConst += curLine curLine = nextLine # check if it is a start of a new log && check if the previous log is 'ready' if newLogRegExp.match(curLine) and logInConst != "": counter = counter + 1 out = logInConst logInConst = "" yield out yield logInConst + curLine def checkFile (regExp, fileName): generatore = readLogs(fileName) listOfMatches=[] for i in generatore: #I'm now cycling through the logs # regExp must be a COMPILE regular expression if regExp.search(i): listOfMatches.append(i) return listOfMatches 

to clarify the information contained in these files. The function is designed to write only 1 line of logs that are stored in these files using 3 lines ... The function works fine with files read from my local machine, but I can’t figure out how to connect to the remote server and create these single-line logs without saving the contents of each file to a string, and then working with the string ... The command I use to connect to a remote computer is:

 connection_out = Popen(['ssh', retList[0], 'cd '+retList[2]+'; cat'+fileName], stdout=PIPE).communicate()[0] 

retList [0] and retList [2] are user @remote and the name of the folder I should be accessing

Thank you all in advance!

UPDATE:

My problem is that I have to establish an ssh connection first:

 pr1=Popen(['ssh', ' siatc@lgssp101 ', '*~/XYZ/AAAAA/log_archive/00/MSG_090308_162648.gz*' ], stdout=PIPE).communicate()[0] 

All the files that I need to open are stored in a list, fileList [], some of them are compressed (.gz), and some are only text files! I tried all the procedures that you showed before the bot didn’t work ... I think I changed the third argument of the Popen function, but I can’t figure out how to do it! Is there anyone who can help me?

Answer 1

You do not need to split the stream / file on the line yourself. Just repeat:

 for ln in f: # work on line in ln 

This should work equally well for files (using open () for file ()) and pipe (using Popen). Use the stdout property for the popup to access the channel connected to the subprocess stage

Example

 from subprocess import Popen, PIPE pp = Popen('dir', shell=True, stdout=PIPE) for ln in pp.stdout: print '#',ln 

Answer 2

Remove InStream and just use the file file.

For your code to read:

 for nextLine in f.readlines(): . . . 

Ber has this right.

To clarify, the default iteration behavior for a file object is to return the next line. therefore, " for nextLine in f " will give you the same results as " for nextLine in f.readlines () ".

For more information, see the documentation for the file object: http://docs.python.org/library/stdtypes.html#bltin-file-objects

Answer 3

If you want to do something through ssh, why not use the Python SSH module ?

Answer 4

Try this page, the best popen information I have found so far ....

http://jimmyg.org/blog/2009/working-with-python-subprocess.html