Computational Photography, Artificial Intelligence and Alice

The Four Rules of Computational Photography

Computational Photography and Sensor Size

It is an incontrovertible fact that larger camera sensors produce objectively better image quality in a single capture than smaller ones. Their larger surface area means they get hit with more photons from the scene and capture more useful information about it. This translates into a better signal-to-noise ratio, more dynamic range, more accurate colours and better low light performance. And, if the photographer is any good, a better image.

This logic, which has driven the digital camera market since its inception, was upended by smartphone companies who realised they could use another dimension to improve the performance of their tiny sensors: time. By capturing a sequence of images in rapid succession and merging them with an algorithm, smartphone cameras capture more useful information about the scene, improving their signal-to-noise ratio, dynamic range, colour reproduction and low light performance. They use computational photography to produce the image quality of a much larger sensor at the cost, size and weight of a much smaller one.

The Micro Four Thirds system uses a sensor about eight times bigger than a typical flagship smartphone, but four times smaller than a full-frame camera. By applying similar computational photography techniques to a sensor of this size, there is no reason that the already-excellent image quality of an MFT camera cannot be increased beyond the level of a larger full-frame camera, at a significantly lower cost and weight.

That is exactly what we are doing with Alice.

​What about AI?

Multi-shot stacking algorithms are not the only computational photography techniques capable of increasing image quality. Artificial intelligence algorithms, or more specifically image processing techniques powered by deep neural networks and machine learning, are also capable.

There is a lot hype and fear around AI, often justifiably so. There is also a lot of confusion, disinformation and misunderstanding. In the wrong hands it can be used to deceive and fabricate dangerously convincingly, but when used well it has the power to profoundly change photography for the better, to an extent not seen since the transition from film to digital technology.

The magic of AI comes from the fact that a programmer does not have to reason about the logic of an algorithm and communicate that explicitly to a machine which will then follow their instructions precisely.

Instead, an AI algorithm just has to be shown what its inputs will be like and what its outputs should be, and it learns the algorithmic logic automatically. By training on a large number of input/output pairs AI algorithms are able to replicate almost any image-to-image transformation, even complicated, rich, subjective transformations like those performed by a skilled human retoucher.

Instead of directly capturing more light from the scene, AI algorithms draw from the data provided to them during training to infer more information about the scene. They can be used to objectively improve image quality in the way that using a bigger sensor does, but they can also be used to improve image quality in a far more subtle, expressive and subjective way that was previously only possible for those deeply skilled in post-processing techniques.

With Alice, these algorithms can be run in real-time on the camera, improving the image quality of an already exceptionally high quality MFT sensor in ways that have not been possible before.

The Four Rules of Computational Photography

AI-driven computational photography algorithms will only be effective if they are built responsibly and with the genuine needs of content creators in mind. To that end we have developed the following design principles for our core algorithms, heavily inspired by those stated by Hasinoff et al in their seminal work on the HDR+ algorithm:

There is plenty to fear about AI but its power is undeniable, and one way or another it’s going to change photography and content creation in a very profound way. With Alice we are embracing it to increase the expressivity of our tools and drive the photographic art form and modern content creation forward.

Written by Dr Liam Donovan, CTO of Photogram.

Photogram is a computational photography startup building the Alice Camera, AI-accelerated computational camera for creators

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store