I have large images that I need to view in my application (6000x8000 24-bit JPEGs for example). We have large images of diagrams because the diagrams are highly detailed. However, users like to browse through the images before selecting one to view in detail (think thumbnail browser). Now the ideal solution would be to pre-generate thumbnails. For the sake of argument that's not an option. So our only option is to resize the images on the fly. So like a good developer I hit Google for the answer. In previous posts I've said speed is paramount and you don't have much to work with on a P2 400Mhz with 128MB RAM. Now its very possible that I didn't look hard enough but all of the solutions I found were some variation on the following code:
Image big = null;
using( Stream stream = File.OpenRead(@"C:\big.jpg") )
{
big = Image.FromStream(stream, false, false);
}
Bitmap small = new Bitmap(big, 600, 800);
big.Dispose();
So what's wrong with this picture? I'll tell you what's wrong, we load the entire big image into memory, then create the small image, then downsample the big image. Now holding a 6000x8000 24-bit image in memory takes 6000x8000x3 bytes and that comes to...carry the 1...144 MB of RAM. Not that big a deal when you have a Gig of RAM or so, but our poor customers can draw the diagram by hand in the time it takes to do all that in virtual memory. I even evaluated out-of-process solutions like ImageMagick. However their command-line "convert" appeared to do the same thing based on the time it took to the downsampling and info I got from Process Explorer. I was in the process of evaluating Raster Imaging Pro for .Net from LEADTOOLS and was hopeful that they had a solution. Once again however I was disappointed. The toolkit loads the entire image into memory (albeit much faster than Image.FromStream) before performing the downsample.
I nearly gave up but finally stumbled upon some building blocks offered by LEADTOOLS that allowed me to assemble an acceptable solution. The toolkit offers some image loading events that allow you to tap into the load "buffer" that contains one or more rows of image data that have been read and decoded from the file. The solution I put together resizes the buffer as it is read and incrementally builds the downsampled image. So at any one time you have memory allocated for:
- The entire downsampled image - 1.44 MB (600x800x3)
- One row of the source image for our processing - 18 KB (6000x3)
- An additional 16 rows of the source image for the buffer provided by the toolkit - 288 KB (16x6000x3)*
- An additional 10 rows for the downsampling buffer - 180 KB (10x6000x3)**
* The toolkit provided me some multiple of 8 rows of image data at a time. A JPEG expert could probably explain the significance. For smaller images I got 8 rows at a time. I can only presume for extremely large images it would read a larger number of rows at a time. Ideally I would only get one to reduce the memory requirements.
** The number of rows required for the downsampling buffer is directly related to the scale factor and the sampling method. I only mostly know what I'm talking about here but a rough rule of thumb is 1/<Scale Factor>. In our case 1/600/6000.
So we are able to downsample the large image using ~2 MB of RAM as apposed to 145 MB of RAM required with the previous solution. Not bad if you ask me. Now here's the caveats:
The downsampled image would be consided "poor" by most standards because I believe the sampling method used by the RasterBufferResize class is "nearest neighbor." The resulting speed and resource requirements however make it good enough for our users. My next step will be to explore writing a box averager buffer resizer in hopes that I can achieve greater quality without affecting speed too much.
The solution implemented uses the LEADTOOLS product which is not free. However, if you have access to a free/open source library that incrementally loads and decodes image data (please tell me about it :), you should be able to apply the same process.
Here's the code for the StreamDownsample class written and tested using Raster Imaging Pro for .Net v14.5.0.13 (only tested with jpegs):
public class StreamDownsample
{
private readonly RasterBufferResize _resizer = new RasterBufferResize();
private readonly RasterCodecs _codecs;
private readonly string _filePath;
private readonly Stream _stream;
private DownsampleResult _result;
private int _maxWidth;
private int _maxHeight;
private int _destinationRow;
private byte[] _line;
private bool _shutdown;
private StreamDownsample(Stream stream, int maxWidth, int maxHeight, RasterCodecs codecs)
: this(maxWidth, maxHeight, codecs)
{
if (stream == null) throw new ArgumentNullException("stream");
_stream = stream;
}
private StreamDownsample(string filePath, int maxWidth, int maxHeight, RasterCodecs codecs)
: this(maxWidth, maxHeight, codecs)
{
if (filePath == null) throw new ArgumentNullException("filePath");
_filePath = filePath;
}
private StreamDownsample(int maxWidth, int maxHeight, RasterCodecs codecs)
{
if (maxWidth <= 0) throw new ArgumentException("maxWidth must be greater than 0");
if (maxHeight <= 0) throw new ArgumentException("maxHeight must be greater than 0");
if (codecs == null) throw new ArgumentNullException("codecs");
_maxWidth = maxWidth;
_maxHeight = maxHeight;
_codecs = codecs;
}
public DownsampleResult Downsample()
{
_codecs.Options.Load.AllocateImage = false;
_codecs.Options.Load.StoreDataInImage = false;
_codecs.LoadImage += new CodecsLoadImageEventHandler(CodecsLoadImage);
try
{
if (_filePath == null)
{
_codecs.Load(_stream);
}
else
{
_codecs.Load(_filePath);
}
}
finally
{
if (!_shutdown)
{
Shutdown();
}
}
return _result;
}
private void CodecsLoadImage(object sender, CodecsLoadImageEventArgs e)
{
if ((e.Flags & CodecsLoadImageFlags.FirstRow) == CodecsLoadImageFlags.FirstRow)
{
Startup(e.Info);
}
for (int i = 0; i < e.Lines; i++)
{
Array.Copy(e.Buffer, i * e.Info.BytesPerLine, _line, 0, e.Info.BytesPerLine);
_resizer.ResizeBuffer(_line, i + e.Row, e.Info.BitsPerPixel);
int bufferCount = _resizer.LineWidth * 3;
for (int j = 0; j < _resizer.CopyRepetitions; j++)
{
_result.Image.SetRow(_destinationRow++, _line, 0, bufferCount);
}
}
if ((e.Flags & CodecsLoadImageFlags.LastRow) == CodecsLoadImageFlags.LastRow)
{
Shutdown();
}
}
private void Startup(CodecsImageInfo imageInfo)
{
int downsampledWidth = imageInfo.Width;
int downsampledHeight = imageInfo.Height;
if (imageInfo.Width >= imageInfo.Height)
{
if (imageInfo.Width >= _maxWidth)
{
downsampledWidth = _maxWidth;
downsampledHeight = downsampledWidth * imageInfo.Height / imageInfo.Width;
}
}
else if (imageInfo.Height >= _maxHeight)
{
downsampledHeight = _maxHeight;
downsampledWidth = downsampledHeight * imageInfo.Width / imageInfo.Height;
}
IRasterImage downsampledImage = new RasterImage(RasterMemoryFlags.Unmanaged, downsampledWidth, downsampledHeight, imageInfo.BitsPerPixel, imageInfo.Order, RasterViewPerspective.TopLeft, imageInfo.Palette, null);
double scaleFactor = downsampledWidth / (double)imageInfo.Width;
_result = new DownsampleResult(downsampledImage, scaleFactor);
_destinationRow = 0;
_line = new byte[imageInfo.BytesPerLine];
_resizer.Start(imageInfo.Width, imageInfo.Height, downsampledWidth, downsampledHeight);
_shutdown = false;
}
private void Shutdown()
{
_shutdown = true;
_codecs.LoadImage -= new CodecsLoadImageEventHandler(CodecsLoadImage);
_line = null;
_resizer.Stop();
}
public static DownsampleResult Downsample(string filePath, int maxWidth, int maxHeight)
{
return Downsample(filePath, maxWidth, maxHeight, new RasterCodecs());
}
public static DownsampleResult Downsample(string filePath, int maxWidth, int maxHeight, RasterCodecs codecs)
{
StreamDownsample downsampler = new StreamDownsample(filePath, maxWidth, maxHeight, codecs);
return downsampler.Downsample();
}
public static DownsampleResult Downsample(Stream stream, int maxWidth, int maxHeight)
{
return Downsample(stream, maxWidth, maxHeight, new RasterCodecs());
}
public static DownsampleResult Downsample(Stream stream, int maxWidth, int maxHeight, RasterCodecs codecs)
{
StreamDownsample downsampler = new StreamDownsample(stream, maxWidth, maxHeight, codecs);
return downsampler.Downsample();
}
}
public struct DownsampleResult
{
public DownsampleResult(IRasterImage image, double scaleFactor)
{
if (image == null) throw new ArgumentNullException("image");
_image = image;
_scaleFactor = scaleFactor;
}
private readonly IRasterImage _image;
public IRasterImage Image
{
get { return _image; }
}
private double _scaleFactor;
public double ScaleFactor
{
get { return _scaleFactor; }
}
}
posted @ Friday, February 23, 2007 5:33 PM