Avoid buffered read "for string in ..."

I read line by line from a named pipe that provides lines in a second-rate rhythm. I tried simple simple

for line in file:
    processLine(line)

but processLine()never called. (EDIT: it is called in the end after many lines have been read, which take several minutes.) A study with the help straceshowed that the process actually makes the final system call read()every second, and is also expected to receive a full line every time.

I can simply assume that the idiom for line inbuffers the input and is called processLine()later with each line of input, probably when the buffer is full or in case the input completes (which in my case will never be).

Is it possible to explicitly point the buffer used here to something less?

Or is there another way to tune things so that each line is also processed in the second or second rhythm?

EDIT:

I am currently using this workaround:

for line in lineByLine(namedPipe):

And this lineByLine():

def lineByLine(openFile):
    line = ''
    while True:
        char = os.read(openFile.fileno(), 1)
        if not char:
            if line:
                yield line
            break
        line += char
        if line.endswith('\n'):
            yield line
            line = ''

But this ugly workaround, of course, has no solution.

+4
source share
2 answers

As you say in your question, file.next()internal buffers. Usually this behavior is correct and undetectable.

file.readline()not buffered in the same way. Your bulky example program creates a generator that allows you to use it file.readline()as iterable in a loop for.

iter:

import sys
for line  in iter(namedPipe.readline, ''):
  print line
+2

( -u), . readline() while True: . :

def lineByLine(openFile):
    while True:
        line = openFile.readline()
        if not line:
            break
        yield line

, , ( ), , . : -/

0

All Articles