Skip to content

Camera

Focus

The focal point f=lim⁡d→∞uf = \lim_{d\rightarrow\infty} u where
ff is the focal length
dd is the distance of the object from the lens
uu is the distance of the image plane

Thin lens law f−1=u−1+d−1f^{-1} = u^{-1} + d^{-1}

Depth of Field

Range of distances where blur < 1 sensor pixel.

DOF≈2u2NCf2DOF \approx \frac{2u^2 NC}{f^2} where
C=C= given circle of confusion (pixel size)
f=f= focal length
N=N= f-stop
u=u= subject distance

Typically, callphone camera has wider-angle lens (short focal length), hence larger DoF. Therefore, it can capture all objects in different distance, and then fake the blur by algorithm

Camera controls

Aperture expressed as D:=f/ND:=f/N,the relative size of the area in which light is collected through the lens

Shutter speed Δt\Delta t, the duration of the exposure, often expressed as fractions of a second

DoF vs. Aperture and Shutter Speed

The capture photons ∝D2Δt\propto D^2 \Delta t
The number of photons captured will influence the exposure.

Also, consider DoF,
D=f/N↑⇒DOF↓D = f/N\uparrow \Rightarrow DOF\downarrow

ISO film speed

The sensitivity of film/sensor to light

Given exposure (D2ΔtD^2\Delta t) being constant, ISO↑⇒brightness↑ISO\uparrow\Rightarrow brightness\uparrow

Color image

All sensor pixels have same response curve, i.e. monochromatic and can only accepts intensity. To obtain a colored image, typically each pixel will be made to be sensitive or one of RGB by filters, typically 25% R, 25% B, 50% G. And full-color images can be obtained by demosaicing each pixel with missing RGB.

Steps of image formation

Photons to Digital Numbers (Linear operations)

radiant power from scene Arriving photons causes photo-electrons and the charge accumulates are more photons hits the photo-diode.

The radiant power

Φ=∫q∫λH(qˉ,λ)S(qˉ)Q(λ)dqˉdλ\Phi = \int_{q} \int_\lambda H(\bar{q}, \lambda) S(\bar{q}) Q(\lambda)d\bar{q} d\lambda

where
qˉ=\bar q = pixel footprint
λ=\lambda = wave length
H=H= incidcent spectral irrandiance (the flux that can be received per/at that surface per/at that wavelength)
S=S= spatial response (the sensitivity of the sensor to radiation from different directions)
Q=Q= quantum device efficiency (electrons that can be collected per incident photon at given wavelength)

Exposure After exposure time, amplifier converts charge to measurable voltage

  • With the exposure time Δt\Delta t, now the total illuminance/irradiance is ΦΔt\Phi\Delta t
  • At same time, black level, a non-photoelextric current from photo diode, I0I_0 is added, to make the least black level. The result is ΦΔt+I0\Phi \Delta t + I_0

Sensor saturation The current cannot exceed saturation current a set-maximum non-discarded current from photodiode
- $I_m = $ saturation current, adjust the voltage by min⁡(ΦΔt+I0,Im)\min(\Phi\Delta t + I_0, I_m)

Gain factor Apply an amplifier gain gg, which is controlled by ISO (the larger gg the darker the darker) - the result will be min⁡(ΦΔt+I0,Im)g\frac{\min(\Phi\Delta t + I_0, I_m)}{g}
- Since digital numbers is discrete, apply a flooring function ⌊min⁡(ΦΔt+I0,Im)g⌋\lfloor\frac{\min(\Phi\Delta t + I_0, I_m)}{g}\rfloor

Therefore,

DN=⌊min⁡(ΦΔt+I0,Im)g⌋DN = \lfloor\frac{\min(\Phi\Delta t + I_0, I_m)}{g}\rfloor

And for the whole process, the relationship is linear

Gamma correction (camera response function)

Since human visual system doesn't have a linear response to light, DNs are passed through a gamma function to compensate, such function is f(DN):=β(DN)1/γf(DN) := \beta(DN)^{1/\gamma} where β,γ\beta, \gamma are constants that varies among different manufactures.

Image Noise

Defocus Blur
when the scene points of interest are "out of focus" and not within the DoF

Motion Blur
Camera moves significantly during exposure time
More like with long exposures and long focal length (zooming in)

Pixel noise
Incorrect exposure, not enough photons reaching sensor
High ISO (gain) causes noise

Rolling shutter wiki
When captured by scanning across the scene rapidly

Sources of Noise

radiant power from scene

Dark Current Noise

  • free electrons due to thermal energy, depends on temperature
  • ∼Poisson(λ=DΔt),D:=\sim \text{Poisson}(\lambda = D\Delta t), D:=thermal electron rate (e−/sec)(e^-/sec)

Exposure

Photon (Shot) Noise

  • ∼Poisson(λ=ΦΔt)\sim \text{Poisson}(\lambda = \Phi\Delta t )
    Pλ(k events in Δt)=λke−λk!P_\lambda(\text{k events in }\Delta t) = \frac{\lambda ^k e^{-\lambda}}{k!}, i.e. P(#received photons=k)=ΦΔtke−ΦΔtk!P(\text{\#received photons}=k)= \frac{\Phi\Delta t ^k e^{-\Phi\Delta t}}{k!}
  • Largest source of noise for high exposures
  • For large ΦΔt\Phi\Delta t, by LLN, we can approximate by Poisson(λ)≈N(λ,λ)\text{Poisson}(\lambda)\approx N(\lambda, \sqrt \lambda)

Sensor saturation

Readout Noise

  • ∼N(0,σr)\sim N(0, \sigma_r), σr\sigma_r depends on characteristics of electronics

Gain factor

Amplifier, ADC, and Quantization Noise

  • ∼N(0,σADC)\sim N(0, \sigma_{ADC})
  • The amplifier noise is dependent on gg, i.e. gain or ISO.
  • Largest source of noise for low exposures

Put all Noises Together

Since all four types of noise are independent of each other,

E(e−)=Black level+dark current+photon noise=min⁡{I0+DΔt+ΦΔt,Im}\begin{align*} E(e^-) &=&\text{Black level} + &\text{dark current} + &\text{photon noise}\\ &= \min\{&I_0 + &D\Delta t + &\Phi\Delta t, I_m\}\\ \end{align*}
var(e−)=dark current+photon noise+black level+readout noise+ADC noisevar(e−)=DΔt+ϕΔt+I0+σr2+σADC2g2\begin{align*} var(e^-) &= \text{dark current} &+ \text{photon noise} &+ \text{black level} &+ \text{readout noise} &+ \text{ADC noise}\\ var(e^-) &= D\Delta t &+ \phi\Delta t &+ I_0 &+ \sigma_r^2 &+ \sigma^2_{ADC} g^2 \end{align*}

Hence
E(DN)=min⁡{I0+DΔt+ΦΔtg,Img}E(DN) = \min\{\frac{I_0 + D\Delta t + \Phi\Delta t}{g}, \frac{I_m}{g}\}
var(DN)=DΔt+ϕΔt+I0+σr2g2+σADC2g2var(DN) = \frac{D\Delta t + \phi\Delta t + I_0 + \sigma_r^2}{g^2} + \sigma^2_{ADC} g^2

Signal-to-noise ratio SNR=10log⁡10E(DN)2var(DN)SNR = 10\log_{10} \frac{E(DN)^2}{var(DN)}