How to implement panning / scaling of gigapixel bitmaps?

Question

In my project, I use (uncompressed 16-bit shades of gray) gigapixel images that come from a high-resolution scanner for measurement purposes. Since these bitmaps cannot be loaded into memory (mainly due to memory fragmentation), I use tiles (and tiled TIFF on disk). (see https://stackoverflow.com/a/167289/ )

I need to implement pan / zoom in a way like Google Maps or DeepZoom. I have to apply image processing on the fly before being presented on the screen, so I cannot use a pre-prepared library that directly accesses the image file. For scaling, I intend to save the image with several resolutions in my file (pyramid storage). The most useful steps, apparently, are + 200%, 50% and all show.

My code base is currently C # and .NET 3.5. I am currently suggesting a type of form if WPF does not give me much advantage in this area. I have a method that can return any (processed) part of the main image.

Specific Issues:

• hints or links on how to implement this panning / zooming by generating parts of the image on demand.
• any code that can be used as a basis (preferably commercial or LGPL / BSD as a license).
• DeepZoom can be used for this (i.e. is there a way that I can provide a function to provide tiles when resized correctly for the current zoom level?) (I need to have an exact pixel address)

Answer 1

I decided to try something myself. I came up with a simple GDI + code that uses tiles that I already have. I'm just filtering out parts that are relevant to the current clipping area. It works like magic! Please find my code below. (Double buffered form settings for best results)

protected override void OnPaint(PaintEventArgs e) { base.OnPaint(e); Graphics dc = e.Graphics; dc.ScaleTransform(1.0F, 1.0F); Size scrollOffset = new Size(AutoScrollPosition); int start_x = Math.Min(matrix_x_size, (e.ClipRectangle.Left - scrollOffset.Width) / 256); int start_y = Math.Min(matrix_y_size, (e.ClipRectangle.Top - scrollOffset.Height) / 256); int end_x = Math.Min(matrix_x_size, (e.ClipRectangle.Right - scrollOffset.Width + 255) / 256); int end_y = Math.Min(matrix_y_size, (e.ClipRectangle.Bottom - scrollOffset.Height + 255) / 256); // start * contain the first and last tile x/y which are on screen // and which need to be redrawn. // now iterate trough all tiles which need an update for (int y = start_y; y < end_y; y++) for (int x = start_x; x < end_x; x++) { // draw bitmap with gdi+ at calculated position. dc.DrawImage(BmpMatrix[y, x], new Point(x * 256 + scrollOffset.Width, y * 256 + scrollOffset.Height)); } } 

To test this, I created an 80x80 matrix of 256 fragments (420 MPixel). Of course, I will have to add some delayed loading in real life. I can leave tiles (empty) if they are not already loaded. In fact, I asked my client to insert 8 GByte into his machine, so I don’t have to worry too much about performance. Once downloaded tiles can remain in memory.

 public partial class Form1 : Form { bool dragging = false; float Zoom = 1.0F; Point lastMouse; PointF viewPortCenter; private readonly Brush solidYellowBrush = new SolidBrush(Color.Yellow); private readonly Brush solidBlueBrush = new SolidBrush(Color.LightBlue); const int matrix_x_size = 80; const int matrix_y_size = 80; private Bitmap[,] BmpMatrix = new Bitmap[matrix_x_size, matrix_y_size]; public Form1() { InitializeComponent(); Font font = new Font("Times New Roman", 10, FontStyle.Regular); StringFormat strFormat = new StringFormat(); strFormat.Alignment = StringAlignment.Center; strFormat.LineAlignment = StringAlignment.Center; for (int y = 0; y < matrix_y_size; y++) for (int x = 0; x < matrix_x_size; x++) { BmpMatrix[y, x] = new Bitmap(256, 256, PixelFormat.Format24bppRgb); // BmpMatrix[y, x].Palette.Entries[0] = (x+y)%1==0?Color.Blue:Color.White; using (Graphics g = Graphics.FromImage(BmpMatrix[y, x])) { g.FillRectangle(((x + y) % 2 == 0) ? solidBlueBrush : solidYellowBrush, new Rectangle(new Point(0, 0), new Size(256, 256))); g.DrawString("hello world\n[" + x.ToString() + "," + y.ToString() + "]", new Font("Tahoma", 8), Brushes.Black, new RectangleF(0, 0, 256, 256), strFormat); g.DrawImage(BmpMatrix[y, x], Point.Empty); } } BackColor = Color.White; Size = new Size(300, 300); Text = "Scroll Shapes Correct"; AutoScrollMinSize = new Size(256 * matrix_x_size, 256 * matrix_y_size); } 

It turned out to be the easy part. Getting asynchronous multi-threaded I / O in the background was much more difficult. However, it works for me as described here. The issues that need to be addressed are related to most of the threading of .NET / Form than to this topic.

In pseudocode, it works as follows:

 after onPaint (and on Tick) check if tiles on display need to be retrieved from disc if so: post them to an async io queue if not: check if tiles close to display area are already loaded if not: post them to an async io/queue check if bitmaps have arrived from io thread if so: updat them on screen, and force repaint if visible 

Result: I now have my own user control that uses approximately 50 megabytes for very quick access to tiled files of arbitrary size (tiled).

Answer 2

This CodeProject article : Create ... A DeepZoom image collection may be useful for reading, as it talks about generating a DeepZoom image source.

This MSDN article contains a section called Dynamic Deep Scaling: providing pixels at runtime and linking to this Mandelbrot Explorer , which 'kinda' sounds just like you are trying to do (that is, it generates specific parts of the mandelbrot, ask if you want specific parts of your gigapixel image upon request).

I think the answer to the question: "Can DeepZoom be used for this?" probably "Yes", however, since it is only available in Silverlight, you will need to do some tricks with built-in web browser control if you need the WinForms / WPF client application.

Sorry, I cannot provide more specific answers - I hope these links help.

ps I'm not sure if Silverlight supports TIFF images - this can be a problem if you do not convert to another format.

Answer 3

I think you can solve this problem as follows:

• Image Generation:

  • segment your image in several subimages (tiles) of small resolution, for instace, 500x500. These images are depth 0
  • combine a series of tiles with a depth of 0 (4x4 or 6x6), resize the combination, creating a new tile with a depth of 500 × 500 pixels.
  • continue this approach until you get the full image using just a few fragments.

• Image visualization

  • Start from the highest depth
  • When the user drags the image, dynamically load the tiles.
  • When the user enlarges the image area, reduce the depth by loading fragments for this region with a higher resolution.

The end result is similar to Google Maps.