Busting the alignment myth
This will be a special feature for hard core falconeye followers ;)
It all started with the exciting new feature of the Pentax K-7 camera which can produce true HDR images in-camera (it takes 3 shots, 3 exposure values apart from each other, and creates a single, tone-mapped or blended image from the sequence). It does it so well, esp. in the medium setting (as opposed to strong) that a limitation of the feature does actually hurt: you require a tripod to use the function.
The general opinion why Pentax left out the alignment step from their HDR function is that it is a processing power-hungry computation not easily done in-camera.
Not my opinion, though. I felt that Pentax has left out 10% of the implementation which reduces 90% of the functionality. Shouldn't it be the other way round?
Now, opinion isn't proof.
So, I implemented my own alignment operator, benchmarked it and compared its quality against the state-of-the-art in HDR.
The algorithm
The implementation is based on an algorithm whose computing time increases linearly with image size and is constant with amount of distortion. Note that I did not attempt to cope with the alignment problem known in panorama stitching, i.e., I don't do SIFT key extraction, I don't correct for lens distortion, I don't project onto a sphere before alignment, etc. pp. However, what I do is correct for shift and rotations which is a mathematical first order approximation to the latter and rather exact for small shifts and rotations.
The implementation uses a monochrome quad tree and high bit depth. The coding is in Java. It isn't based on anybody else's intellectual property.
The benchmark
I benchmarked the algorithm with a 5 image HDR from Munich Siegestor. It had distortions of up to 20 pixels and needed rotation as well. The source images haven't been ideally sharp (e.g., 1/25s and wide open with 15mm). And no program created a perfect alignment. So, I considered this example be adequate. The images were 14.6 MPixel JPG images.
The processing times on a Mac Mini (Early 2009), one processor used, are as follows:
- Single alignment: 175 ms
- Alignment of all HDR images: 700 ms
- Reading of all images from disk into Java heap: 16 s
- Shift-turn and write images to disk: 34 s
- Total, disk to disk (70 MB transferred), 5 images: 50 s
The K-7 has a dedicated image DSP and can certainly rival this. E.g., it can JPG-compress a 14.6 MPixel image in about 100 ms only!
Because the HDR function within K-7 already has created the required image structures in its 2 GB memory, it would be conclusive to assume that addition of an auto-alignment feature to the HDR function would add less than an extra 1/2 second to overall operation time.
If processing speed isn't the reason, so maybe it is quality? This must be looked at. In particular as some of the alignment operators I compared with took several to nearly 10 minutes to complete.
The quality
I've run several alignment operators and come up with the following ranking:
- Photoshop CS3; PhotoAcute ("normal alignment")
- Falk Lumo Operator (as of this blog article); PhotoMatix 3 ("by matching features")
- PhotoMatix 3 ("by matching features"; "reduce ghosting")
- PhotoMatix 3 ("by correcting horizontal and vertical shifts")
- No alignment
All pre-aligned images have been read into Adobe Photoshop CS3, either by HDR merge (without further alignment) or by opening the 32 Bit .hdr-file written by PhotoMatix. Tonemapping was done in Photoshop CS3, using the "Equalize Histogram" method. The latter isn't best but as it has no parameters, I thought it would make for a good option in a comparative study. Finally, the result was studied to derive the above ranking. Which is entirely subjective!
Below are the two samples from category #2. Please, visit the gallery to view the other samples, too. Note that all sample images have been resized to 6 MPixel.
Fig.1 No alignment
Fig.2 PhotoMatix
Fig.3 Falk Lumo Alignment Operator (as decribed here)
Conclusion
The myth is busted. There is no reason to not include auto-alignment into the next firmware release of the K-7 HDR feature. In less than an extra second of processing time, about PhotoMatix quality can be provided. Make it so!
View or post comments: click here