Lens Docs

Appearance

Pixel Color Lookup ​

There are still a few problems with the Reverse Pixel Mapping technique. First of all is that when mapping a pixel from a fixed integer position in the destination, you can end up with a non-integer position in the source image. That is a location that falls somewhere between the individual pixels on the source image. To determine what color should be returned a process called Interpolation is used to determine the final color for that real position by mixing the colors of the surrounding pixels.

The Interpolation setting will also handle the case when a part of a distorted image becomes 'stretched out' so that a single source pixel becomes smeared over a large area of the destination image. However, the opposite is not handled very well by a simple interpolation method. And that requires other techniques which we will look at below.

Supersampling ​

Interpolation works well for simple image distortions. But if part of the source image gets compressed into a much smaller area, each destination pixel could actually require a merging of a much larger area of the source image. Remember pixels are not really points, but represent a rectangular area of a real image.

This means in some cases we really should be trying to compress a large area of the source image into a single destination pixel. When this happens a simple Pixel Lookup will fail, as it only looks up the color at a single 'point' in the source image (using the surrounding pixel neighbourhood), and does not merge and combine all the colors of the input image that may have to be compressed into that single pixel.

The result of this is that a destination pixel could end up with an essentially random color from the source image, rather than an average of all the colors involved. This is not in itself bad, but when all the pixels in a area doe this you get images with seemingly random, isolated pixels, Moire effects, and aliased 'stair-casing' effects. Thin lines also start to look more like dotted and dashed lines, or could disappear entirely. All these effects are known collectively as Aliasing Artifacts.

One solution to this to do more color lookups from the source image, for each and every pixel in the destination to try to determine a more correct color for each pixel in the destination image. The simplest solution is generally known as supersampling, or oversampling. See the Wikipedia Entry on Supersampling.

By taking more samples from the source image, over the area that will map onto each destination pixel, the final color of that pixel will become a more accurate representation of distorted image at that point. The more color samples you make, the more accurate the final color will be, and a smoother more realistic look will be generated, though the slower the distortion becomes.

Remember this technique only really improves the general look of the destination in areas where the source image becomes compressed by more than 50%. In areas where the distortion magnifies the source image, or keeps it about the same scale, a single Interpolated Look Up of the source image look up will generally produce a good result with just one single lookup.

One final word of warning. Supersampling is limited by the number of samples that was used for each pixel in the final image, and thus the amount of scaling used in the final resize. This determines the final 'quality' of the distorted image. But by using larger scaling factors, the distorted image will of course be much much slower to generate. But have even higher quality, has its limits.

In summary, supersampling can improve the look of images with only minor distortions, such as rotations, shears, affine, and simple perspective. But it has limits to the types of distortions that it can improve.

TIP

To use supersampling in Lens, you can set outputScaling option to some value, larger than 1:

ts
const result = await distort(image, distortionName, args, {
  outputScaling: 2,
});

Area resampling ​

One of the best alternatives to supersampling methods is Area Resampling.

Rather than distorting a larger image and averaging the results by resizing, which is just taking and averaging more samples from the image, we actually determine exactly how many pixels from the source image should be merged together (based on the 'scale' of the distortion at that point) to generate each specific output pixel. That is figure out a rough 'area' within the source image, each output pixel represents, and merge (filter) all the pixels in that area according to a resampling filter.

When area resampling a distorted image, the area of pixel (window) to get samples from will not only change position, but also will change size. As such one pixel in the destination may only need to merge a few source image colors, or even just one single interpolated color lookup (such as in enlargements). While another pixel elsewhere in the destination image, may need to sample a very large number of pixels to generate the correct final color. Close to infinities it may even have to include all the pixels in the source image as part of the sampling process.

Also, the area that a destination pixel represents in the source image, may not be a simple square, circle, or even ellipse but may actually be a highly distorted shape, according to the distortion being used. Calculating and handling such awkward shapes can be very time-consuming, or near impossible to achieve.

Elliptical Weighted Average (EWA) ​

Using an elliptical area of the source image to calculate colors for each destination pixel, is a method known as Elliptical Weighted Average (EWA) Resampling, and was outlined in the PDF research paper "Fundamentals of Texture Mapping and Image Warping" by Paul Heckbert (who also wrote the 'zoom' program from which practically all image resize algorithms are derived).

An ellipse is the perfect shape for either Affine Distortions or Perspective Distortions. It is especially good for extreme scale reductions. And while not perfect for other distortions, it is generally a reasonable fit for many other distortions, such as Arc and Polar Distortions (but not their reverse), as well a radial distortions like the Barrel Distortion.

But it is a poor fit for non-linear distortion mappings such as De-Polar and Shepard's Distortion, as such it is not used for these distortions.

WARNING

Polar, De-Polar and Shepard's distortions are not implemented in Lens at the moment.

Supersampling does not have this shape problem as each 'sample' is reverse mapped onto the destination. So it becomes the better sampling method in such cases. But as mentioned it may not sample all the pixels needed, or even sample too many pixels.

Summary ​

Using a simple ellipse (EWA resampling) to do 'area resampling' does produce good results, as all the source pixels involved in scaled, affine or perspective distortions, will be merged to produce the final color of an individual destination pixel.

In cases of very non-linear distortions, such as in De-Polar Distorts, or indeterminate distortions, such as Shepard's Distortion or even ray-tracing, finding the correct 'Area' to resample all the source pixels needed, becomes prohibitive, and supersampling is the best method to improve results.

But for straight tiling, enlargements, and unscaled rotations, a very fast single 'point' interpolated lookup is probably all that is required.

Remember however all resampling techniques are just methods for determining the color of each individual pixel. It is not actually part of how an image is distorted, except with regard to the mapping of locations between destination and source (or visa-versa if possible).