Click “I Love Computer Vision” to follow and receive messages faster!
Author: Wang Hawk
Original: https://zhuanlan.zhihu.com/p/51490200
1. Introduction to Computational Photography
What is computational photography? How does it differ from digital image processing? How does it relate to the currently popular computer vision? How is it different from traditional photography?
There is a significant lack of talent in the field of computational photography, and many people are interested in this area, especially those with an engineering background. However, while there are many books on computer vision and computer graphics in the country, there is not enough systematic material dedicated specifically to computational photography, and much of it is scattered, making it difficult for those with an engineering background to understand.
I hope this article can partially answer your questions.
2. What is Computational Photography?
According to the translated Wikipedia: “Computational photography refers to techniques for capturing and processing digital images using digital computation rather than optical processing. Computational photography can enhance the capabilities of cameras, introduce features that are fundamentally impossible with film-based photography, or reduce the cost or size of camera components. Examples of computational photography include in-camera computation for digital panoramas, high dynamic range images, and light field cameras. Light field cameras use new optical elements to capture 3D scene information, which can then be used to produce 3D images, enhanced depth of field, and selective focus (or ‘post-focus’). Enhanced depth of field reduces the need for mechanical focusing systems. All these features utilize computational imaging techniques.”
This lengthy explanation can seem quite obscure to those encountering it for the first time. In fact, as shown in the figure below, computational photography is an interdisciplinary field that intersects computer graphics, computer vision, optics, and sensors, and it has significant applications in many areas. [The following illustrations and videos, unless otherwise noted, are cited from [1], the CMU Fall 2017 Computational Photography course lecture notes.]
Computational photography is an interdisciplinary field Let’s start with photography. Traditional photography consists of three parts, as shown in the figure below, where optical devices are used to converge incoming light onto film, the film is photosensitive, and finally, the image is formed through chemical means in a darkroom.
In the digital photography era, sensors replaced film, and the camera body processor replaced the darkroom, as shown in the figure below:
Now, in the field of computational photography, each of the components can take on new forms. For example, we can use powerful image algorithms to process the information obtained by the sensors, achieving a variety of rich effects.
Typical applications include:
Image Stylization
HDR
Image Fusion
Automatic Image Stitching
Large-Scale Image Dataset Information Mining
Mining Time-Lapse Videos
Not only can we use powerful image algorithms for image processing, but we can also make revolutionary updates to the optical components, as shown in the figure below:
Here, light field cameras and the dual-camera systems widely accepted in smartphones are typical applications that allow for post-capture focusing and simulating shallow depth of field effects like DSLRs.
Some have even proposed eliminating the lens in the optical system to create the so-called FlatCam:
From the perspective of sensors, in traditional photography, whether using film or standard sensors, the goal is to capture visible light. In the field of computational photography, sensors can also undergo new changes. For example, Microsoft’s Kinect is a typical TOF camera used to capture depth information of scenes. The front camera used in the iPhone X employs laser speckle structured light technology to obtain precise 3D data of faces.
In summary, compared to traditional photography, computational photography can vary in aspects such as optics, sensors, and image processing, achieving effects that were unimaginable in traditional photography.
Let me show you a fascinating video: using computational photography technology, we can even see objects behind walls.
How to see around corners
3. References
This series of articles is primarily based on the lecture notes of famous foreign universities’ computational photography courses, supplemented by my own simulations of the content discussed in Python Notebook. This approach can help readers grasp the necessary theoretical knowledge of computational photography while connecting theory with practice to clarify the relationship between formulas and actual code. If you find inspiration from these articles and even develop new interesting applications and products, that would be wonderful.
The current plan is to write a series of articles centered around CMU’s Fall 2017 Computational Photography course 15-463, with additional references as necessary:
Computational Imaging and Display (Gordon Wetzstein, Stanford) Computational Photography (Derek Hoiem, UIUC) Computational Photography (James Hays, Brown) Computational Photography (Ramesh Raskar and Jack Tumblin, SIGGRAPH course) Digital and Computational Photography (Fredo Durand, MIT) Introduction to Computational Photography (Oliver Cossairt, Northwestern) Introduction to Visual Computing (Kyros Kutulakos, University of Toronto) Visual Computing Systems (Kayvon Fatahalian, CMU) I hope I am not merely translating the lecture notes but adding my own interpretations and even simulations I have done in Python.
If you are interested in my articles and find them useful, please give me a thumbs up, thank you!
References:
CMU 2017 Fall Computational Photography Course 15-463, Lecture 1
Join the group for communication
Follow computer vision and machine learning technology, welcome to join the 52CV group, scan the code to add 52CV to pull you into the group,
(Please be sure to specify: 52CV)
If you prefer to communicate on QQ, you can add the official QQ group: 702781905.
(I may not be online all the time, so please forgive me if I can’t verify you promptly)
Long press to followI love computer vision
Please give me a “look”!