I finally got around to editing some photos from my recent San Francisco & Maui trip. Enjoy! 🤙

Compared to DSLR cameras, smartphone cameras have smaller sensors, which limits their spatial resolution; smaller apertures, which limits their light gathering ability; and smaller pixels, which reduces their signal-to-noise ratio. The use of color filter arrays (CFAs) requires demosaicing, which further degrades resolution. In this paper, we supplant the use of traditional demosaicing in single-frame and burst photography pipelines with a multi-frame super-resolution algorithm that creates a complete RGB image directly from a burst of CFA raw images. We harness natural hand tremor, typical in handheld photography, to acquire a burst of raw frames with small offsets. These frames are then aligned and merged to form a single image with red, green, and blue values at every pixel site. This approach, which includes no explicit demosaicing step, serves to both increase image resolution and boost signal to noise ratio.

Further indications that mobile photography is disrupting the digital imaging industry thanks to a tighter integration of hardware and software.

The future of photography is computational, not optical. This is a massive shift in paradigm and one that every company that makes or uses cameras is currently grappling with. There will be repercussions in traditional cameras like SLRs (rapidly giving way to mirrorless systems), in phones, in embedded devices and everywhere that light is captured and turned into images.

Sometimes this means that the cameras we hear about will be much the same as last year’s, as far as megapixel counts, ISO ranges, f-numbers and so on. That’s okay. With some exceptions these have gotten as good as we can reasonably expect them to be: Glass isn’t getting any clearer, and our vision isn’t getting any more acute. The way light moves through our devices and eyeballs isn’t likely to change much.

What those devices do with that light, however, is changing at an incredible rate. This will produce features that sound ridiculous, or pseudoscience babble on stage, or drained batteries. That’s okay, too. Just as we have experimented with other parts of the camera for the last century and brought them to varying levels of perfection, we have moved onto a new, non-physical “part” which nonetheless has a very important effect on the quality and even possibility of the images we take.

The present of photography is already computational, it’s always been, since the advent of the digital camera: from the very moment an imaging sensor’s output signal is digitized, billions of operations are performed to turn a stream of electrical levels into a colorless bitmap, then reconstructing colors by means of interpolation, correcting gamma, white balance, lens aberrations, reducing noise, compressing the image by discarding information not visible by the human eye, etc., just to name a few basic operations performed by a typical image processing pipeline. There is no such thing as #nofilter.

What we are seeing now is an unprecedented rate of innovation in image processing enabled by huge advancements in computing power and integration, bound to marginalize the traditional photographic industry.