Problems loading images using python and gobject

Question

I have a script with a GTK interface (GObject) that I use to post to my photo blog.

I am trying to improve its responsiveness by loading images into a background thread.

I was not lucky that I was trying to populate the GdkPixbuf objects from the background thread, all I tried was just jumbled.

So, as an alternative, I thought I would read the files in the background thread and then push them into GdkPixbuf on demand. This approach gave some unexpected and rather depressing results that make me wonder if I am doing something very bad.

I play with slightly compressed jpegs from my camera, they are usually around 3.8mb.

Here is the initial download of the uploaded image:

pb = GdkPixbuf.Pixbuf.new_from_file(image_file) 

This is an average of about 550 ms, but not huge, but rather tedious if you want to view dozens of images.

Then I laid it out, here I read the file:

 data = bytearray(open(self.image_file).read()) 

These are average values ​​of 15 ms, which is really nice, but also a curious concern, if we can read the file in 15 ms, what is the rest 535 ms spent on?

By the way, a bytearray call exists because PixBufLoader does not accept data otherwise.

And then loading Pixbuf:

 pbl = GdkPixbuf.PixbufLoader() pbl.write(data, len(data)) pbl.close() pb = pbl.get_pixbuf() 

This is about 1400 ms, which is almost 3 times more than what Gtk can do.

Am I doing something wrong here?

Answer 1

My guess: you are doing something wrong. I just compared libjpeg-turbo with gdk.PixbufLoader and almost did not find any differences in speed. The code I used is below.

For libjpeg-turbo (jpegload.c):

 #include <assert.h> #include <stdio.h> #include <stdlib.h> #include <sys/time.h> #include <jpeglib.h> void decompress(FILE* fd) { JSAMPARRAY buffer; int row_stride; struct jpeg_decompress_struct cinfo; struct jpeg_error_mgr jerr; cinfo.err = jpeg_std_error(&jerr); jpeg_create_decompress(&cinfo); jpeg_stdio_src(&cinfo, fd); jpeg_read_header(&cinfo, TRUE); jpeg_start_decompress(&cinfo); row_stride = cinfo.output_width * cinfo.output_components; buffer = (*cinfo.mem->alloc_sarray) ((j_common_ptr) &cinfo, JPOOL_IMAGE, row_stride, 1); while (cinfo.output_scanline < cinfo.output_height) { (void) jpeg_read_scanlines(&cinfo, buffer, 1); } jpeg_finish_decompress(&cinfo); jpeg_destroy_decompress(&cinfo); } int main(int argc, char** argv) { long len; FILE *fd; unsigned char *buf; struct timeval start, end; int i; const int N = 100; int delta; /* read file to cache it in memory */ assert(argc == 2); fd = fopen(argv[1], "rb"); fseek(fd, 0, SEEK_END); len = ftell(fd); rewind(fd); buf = malloc(len); assert(buf != NULL); assert(fread(buf, 1, len, fd) == len); gettimeofday(&start, NULL); for(i = 0; i < N; i++) { rewind(fd); decompress(fd); } gettimeofday(&end, NULL); if(end.tv_sec > start.tv_sec) { delta = (end.tv_sec - start.tv_sec - 1) * 1000; end.tv_usec += 1000000; } delta += (end.tv_usec - start.tv_usec) / 1000; printf("time spent in decompression: %d msec\n", delta/N); } 

For python gdk (gdk_load.py):

 import sys import gtk import time def decompress(data): pbl = gtk.gdk.PixbufLoader() pbl.write(data, len(data)) pbl.close() return pbl.get_pixbuf() data = open(sys.argv[1]).read() N = 100 start = time.time() for i in xrange(N): decompress(data) end = time.time() print "time spent in decompression: %d msec" % int((end - start) * 1000 / N) 

Test run results:

 $ gcc jpegload.c -ljpeg $ ./a.out DSC_8450.JPG time spent in decompression: 75 msec $ python gdk_load.py DSC_8450.JPG time spent in decompression: 75 msec $ identify DSC_8450.JPG DSC_8450.JPG JPEG 3008x2000 3008x2000+0+0 8-bit DirectClass 2.626MB 0.000u 0:00.019 

EDIT: and another test using gi.repostiroy this time:

 import sys import time from gi.repository import GdkPixbuf def decompress(filename): pb = GdkPixbuf.Pixbuf.new_from_file(filename) return pb N = 100 start = time.time() for i in xrange(N): decompress(sys.argv[1]) end = time.time() print "time spent in decompression: %d msec" % int((end - start) * 1000 / N) 

And the results:

 $ python gi_load.py DSC_8450.JPG time spent in decompression: 74 msec 

GdkPixbuf.PixbufLoader using gi.repository is really much MUCH slower than the "clean" gtk.gdk . The code:

 import sys import time from gi.repository import GdkPixbuf def decompress(data): pbl = GdkPixbuf.PixbufLoader() pbl.write(data, len(data)) pbl.close() return pbl.get_pixbuf() data = bytearray(open(sys.argv[1]).read()) N = 100 start = time.time() for i in xrange(N): decompress(data) end = time.time() print "time spent in decompression: %d msec" % int((end - start) * 1000 / N) 

Results:

 $ python gi_load.py DSC_8450.JPG time spent in decompression: 412 msec 

But GdkPixbuf.Pixbuf.new_from_file is as fast as a clean version of C, even using gi.repository , so you either do something wrong or expect too much.

Answer 2

I developed a small image viewer with pygtk. I use PixbufLoader, but I only feed N bytes per write (). In combination with idle_add (), I can upload an image in the background while the application is still responding to user input.

Here is the source: http://guettli.sourceforge.net/gthumpy/src/ImageCache.py