 Research
 Open Access
 Published:
Recovering low spatial frequencies in wavefront sensing based on intensity measurements
Advanced Structural and Chemical Imaging volume 2, Article number: 3 (2016)
Abstract
The transport of intensity equation (TIE) offers a convenient method to retrieve the phase of a wave function from maps of the irradiance (images) recorded at different planes along the optic axis of an optical system. However, being a secondorder partial differential equation, even for noisefree data a unique solution of the TIE requires boundary conditions to be specified which are generally not accessible experimentally, jeopardizing retrieval of the lowfrequency information in particular. Here we introduce an iterative algorithm which forgoes the need for explicit boundary conditions and combines the wellknown reciprocal space solution of the TIE with the chargeflipping algorithm that has originally been developed to solve the crystallographic phase problem in Xray diffraction. Application of this algorithm to experimental data and comparison with conventionally used algorithms demonstrates an improved retrieval of the low spatial frequencies of the phase.
Background
Wavefront sensing, i.e., the detection of relative phase shifts in propagating waves provides essential information in imaging applications where the scattering process affects the phase of the probing wave. Examples which highlight the importance of being able to detect phase shifts of waves passing through transparent objects include imaging of unstained cells under the optical microscope and imaging of soft matter (e.g., DNA, viruses, proteins and other macromolecules, polymers, etc.) in the transmission electron microscope (TEM). In 1953, Frits Zernike received the Nobel Prize in Physics for the development of phase contrast microscopy, a technique which allows part of the phase information carried by a wave to be converted into an amplitude signal, making it detectable as part of the intensity variations in the image. In 1971, Dennis Gabor received the Nobel Prize in Physics for developing the holographic principle [1], a technique by which the phase of a wave could be extracted by postprocessing images. Later, iterative [2, 3] and deterministic [4–7] mathematical formulations and associated computer algorithms were developed by which both phase and amplitude of a wave could be recovered from intensity measurements at different planes along the optic axis, a socalled focal series.
One very popular approach toward wavefront reconstruction from intensity measurements at different planes of focus is the transport of intensity equation (TIE) [5, 8] which, due to its simple mathematical formulation and straightforward computational implementation, has attracted much attention in research communities as diverse as cold atom clouds [9], digital optical holography [10], and medical Xray imaging [11].
Many algorithms such as the fast Fourier transform [6], the finite element method [12, 13], multigrid methods [14], a special symmetrization approach [15], each requiring Neumann, Dirichlet, or periodic boundary conditions have been proposed and applied for solving the TIE.
For wavefront reconstruction from focal series of images, the high spatial frequency components of the phase are welldefined by the data, but the low spatial frequency components are largely determined by the boundary conditions, which are usually unknown. Gureyev et al. [16, 17] and later Zuo et al. [18] introduced hardedge apertures or, more generally nonuniform illumination during the experiment and thus physically enforced Neumann boundary conditions, allowing orthogonal series expansionbased approaches to be used to solve the TIE. Such an approach to make the boundary conditions physically accessible may be feasible in some setups, but not generally. In the TEM, for example, the field of view is often so small that no aperture with perfectly abrupt edges exists, in particular not at atomic resolution.
There have been attempts to improve the recovery of low spatial frequency information in the context of the TIE by reformulating it as a totalvariation optimization problem, [19, 20]; however, these approaches require a piecewise constant phase. Other approaches include the application of structured illumination [21]; the experimentally much more complicated interferometric setup [22]; or prior knowledge of the measurement variation [23]. Therefore the problem of faithfully recovering low spatial frequency components of arbitrarily shaped phases remains, at least for a very large range of applications of wave front sensing.
In this work, we propose a simple iterative algorithm, gradient flipping (GF), with an emphasis on objects that are nonperiodic and nonpiecewise linear. GF imposes sparsity on the gradient of the phase by either driving a certain percentage of the phase gradient to zero, or forcing all phase gradients below a certain positive threshold to zero. By combining the conventional Fourier method to solve the TIE with principles adapted from the chargeflipping algorithm in crystallography, GF determines boundary conditions on the phase, while preserving consistency with the higher frequencies of the experimental data.
In this research work, first the TIE and its conventional Fourier solution are introduced; then, the gradientflipping algorithm is presented and it is demonstrated with simulations that the GF algorithm retrieves the boundaries and low spatial frequencies of two test objects. Furthermore, experimental results on a fly wing are presented in the experiment section; and, finally, conclusions are drawn.
The transport of intensity equation
The TIE is a secondorder elliptical, nonseparable, and inhomogeneous partial differential equation which relates the irradiance as well as the variation of the irradiance along the direction of propagation to a Laplacianlike function of the phase:
where k denotes the wave number of the incident radiation, and \(\vec {r}_\bot\) is a vector in the plane normal to the optic axis. \(\frac{\partial I\left( \vec {r}\right) }{\partial z}\) denotes the variation of intensity along the optical axis z. This quantity is most often approximated by the simple firstorder finite difference approximation
Here \(\Delta z\) is a small distance along the optic axis. If the image \(I(\vec {r})\) is recorded in the exact focus of the imaging system, then \(I\left( \vec {r}, +\Delta z \right)\) and \(I\left( \vec {r}, \Delta z \right)\) are images recorded under overfocus and underfocus condition, respectively. Note that \(I\left( \vec {r} \right)\) has to be nonzero, for this equation to have a welldefined solution.
Expression (1) can be rewritten in the following form
where \(\nabla ^{2} = (\vec {\nabla }_\bot \cdot \vec {\nabla }_\bot )^{1}\). A detailed discussion on the validity and range of applicability of Eq. (3) can be found in [24].
The nature of this equation implies that boundary conditions must be applied to solve it. Assuming different boundary conditions will yield different solutions for the phase \(\varphi \left( \vec {r}_\bot \right)\). A number of different algorithms have been developed to solve the TIE (e.g., [6, 15, 16, 25–29]), many of which are based on the very popular approach by Paganin and Nugent [6] which makes use of the fact that
where \(\mathcal {F}\) and \(\mathcal {F}^{1}\) are the twodimensional forward and inverse Fourier transform, respectively, and \(\vec {q}_\bot\) is the twodimensional reciprocal space coordinate in the plane of \(f\left( \vec {r}_\bot \right)\). At \(q_\bot  = 0\) this expression diverges, so at that reciprocal space point one can simply multiply by zero instead. This is a physically legitimate procedure, since this defines the mean value of the phase—a physically undefined quantity—as zero.
This expression is straightforward to implement computationally, since it makes the expression (3) fully deterministic. However, using discrete Fourier transforms periodic boundary conditions are implicitly imposed. Also for iterative approaches, such as finite element [12, 13] or multigrid [27] methods the boundary conditions must be specified and are often chosen to either be periodic, or of the Neumann type, or even both [27].
In the context of wavefront sensing, the investigated objects often have sparse phase gradients
This means that they contain areas where the phase is rather flat. Examples of such sparse objects include live cells in biological, biochemical, or biophysical applications, a large fraction of objects (e.g., nanoparticles) observed in the TEM, but also objects that extend well beyond the detected field of view, but have regions of constant optical thickness (e.g., the experimental example shown below).
Gradient flipping
Gradient flipping (GF), is based on the chargeflipping (CF) algorithm which was originally developed for Xray crystallography [30] where it is very effective in finding sparse solutions of the charge density consistent with experimental diffraction data.
GF pads the input data \(I(\vec {r})\) with its mean value so that the padded image is twice as large along each of its two dimensions as the original image [31]. Also \(\partial I\left( \vec {r}\right) / \partial z\) is padded to the same size, with zeros around its perimeter. The data in the padded area are then iteratively updated such that the phase gradient within the area corresponding to the measurement either has a certain percentage driven to zero, or has the gradient in all pixels the absolute value of which is below a certain positive threshold minimized.
The GF algorithm iterates between \(\vec {G}(\vec {r})\) in Eq. 5 and the following expression for \(\partial I\left( \vec {r}\right) / \partial z\):
where gradient flipping is applied as
The parameter \(\beta\) is chosen slightly below 1, i.e., \(\beta = 0.97\) in order to improve convergence. Furthermore, \(\delta\) defines a threshold between 5 and 20 % of the maximum value of \(\Vert \vec {G}\left( \vec {r_\bot }\right) \Vert _1\), this proved to keep the balance between perturbation and algorithmic stability as suggested in [30].
At each iteration the lefthand side of (6) is updated with the experimental data \(dI^\mathrm{{exp.}}_z \left( \vec {r_\bot }\right)\) by
where h is defined in reciprocal space as
The mask h acts as a Gaussian lowpass filter for the flipped gradient \(\vec {G}'(\vec {r})\) with a characteristic length of \(2 \pi R_{LP}\). This updating rule thus preserves the high spatial frequencies from the measurements, which are generally welldefined by the experiments, and lets the lowfrequency information, which is only weakly present in the measurements, be dictated by the gradient flipping.
\(dI_z \left( \vec {r}\right)\) is initialized with the experimental values \(dI^\mathrm{{exp.}}_z \left( \vec {r}\right)\) and zeropadded. Then it is fed into an iterative procedure which loops over the operations defined in expressions (3), (5), (7), and (8), feeding the updated \(dI_z \left( \vec {r_\bot }\right)\) again into expression (3). Convergence is reached when successive estimates of the phase are sufficiently similar. Figure 1 shows a flowchart of the proposed algorithm.
Free parameters δ and R _{ LP }
A careful selection of the threshold parameter \(\delta\) is a matter of great importance owing to its role as a tradeoff between stabilization and perturbation of the algorithm. The threshold is defined as \(\delta = \zeta \sigma,\) where \(\sigma\) is the standard deviation of the phase gradient and \(\zeta\) is a constant. As shown in Fig. 2, despite the variation of \(\sigma\) during the initial iterations, it remains almost constant throughout the rest of the proposed iterative algorithm. This confirms the eligibility of \(\sigma\) to be a reasonable basis for the optimum choice of \(\delta\). Following the suggestion of Oszlányi et al. [30], we chose the value of \(\zeta\) between 1.0 and 1.2.
The characteristic length scale of the mask h in (8), \(R_{LP}\), is the second free parameter in the proposed algorithm and is determined entirely from the experimental data by setting it to the value that minimizes \(\chi ^2\) in (10). The \(\chi ^2\) figureofmerit is defined as,
where the values of z being summed over all the under and overfocus at which the experimental images have been recorded, and x and y span the area of those images. Furthermore, \(I^\mathrm{{sim.}}(\vec {r}_\bot ,R_{LP})\) are the images simulated from the phase \(\varphi (\vec {r},R_{LP})\) that has been reconstructed with \(R_{LP}\) and the amplitude \(A(\vec {r}) = \sqrt{I(\vec {r},z=0)}\). \(I^\mathrm{{exp.}}\left( \vec {r},z\right)\) denotes the experimental data.
Simulations
In this section the performance of GF is demonstrated on simulations of two specimens: the projection of a cube and a Lshaped membrane.
Projected cube
Images of a test object are simulated for a wavelength of \(\lambda = 500\) nm, a defocus step of \(\Delta f = 1\) mm, a numerical aperture of 0.3, and a pixel size of 1 μm. Figure 3 shows an underfocused, and an overfocused image, as well as the finite difference estimate of the intensity variation along the optical axis determined from those. The images were padded by a factor close to 2, i.e., from \(624 \times 624\) to \(1200 \times 1200\) pixels.
The threshold \(\delta\) was set to \(6.64\,e^{4}\), corresponding to a \(\zeta\) of 1.2. From the graph in Fig. 4 it is apparent that \(\chi ^2\) is minimal for values of \(R_{LP}\) greater than 17.77 μm and \(R_{LP}\) is thus set to this value.
Figure 5b shows the retrieved phase by means of the proposed approach and Fig. 5a displays the original phase of the wave used to simulate the input data shown in Fig. 3. In Fig. 5c the boundaries of the original phase and its reconstruction are compared.
Lshaped membrane
To further investigate the performance of the proposed algorithm for properly recovering also slowly varying phases, we constructed a phase object with the phase given by a continuous function that is not piecewise constant (Fig. 6a), namely the Lshaped membrane; this function can be obtained by the MATLAB expression ’membrane()’. Figure 6c shows the phase map retrieved by means of the FFT approach, which assumes periodic boundary conditions, while Fig. 6b shows the reconstruction obtained by the symmetrization method [15]. As clearly shown in Fig. 6d, the proposed algorithm yields a more accurate reconstruction than the aforementioned approaches. Furthermore, from Fig. 6d it is clear that GF retrieves the boundaries much better.
For this reconstruction the parameter \(\delta\) has been set such that the number of pixels being flipped corresponded to one quarter of the total number of the pixels in the field of view. And \(R_{LP}\) has chosen to be 34.52 μm by minimizing \(\chi ^2\).
Experiment
The GF algorithm is tested further using experimental data acquired from the simple optical setup shown in Fig. 7. The system comprised a laser with integrated collimator emitting green light at a wavelength of \(\lambda = 520\) nm, two lenses with a focal length of \(f = 150\) mm, an iris diaphragm, and a for constructing the new \(2048 \times 2048\) pixel CCD detector. The wing of a fly was used as a test object positioned at a distance r in front of the first lens, with \(f<r<2f\). The diaphragm was placed at the back focal plane of the first lens in order to limit the numerical aperture of the system to about 0.1. Images at the three focal planes \(z=\Delta z\), \(z=0\), and \(z=+\Delta z\) were acquired by translating the camera along the optic axis with a defocus step of \(\Delta z = 1\) mm.
All three images were darkcurrent corrected, and a gain reference image with no object in place was used for normalization. Since a difference in defocus leads to a difference in contrast between these three images and motivated by the fact that \(I(\vec {r},+\Delta z) + I(\vec {r},\Delta z) \approx 2 I(\vec {r})\) the two defocused images \(I(\vec {r},\pm \Delta z)\) were registered to the focused image \(I(\vec {r})\) by iterating the following two steps until convergence:

Step 1:
Shift \(I(\vec {r},+\Delta z)\) to the position of the maximum of the cross correlation of that image with \(2 I(\vec {r})I(\vec {r},\Delta z)\), and

Step 2:
Shift \(I(\vec {r},\Delta z)\) to the position of the maximum of the cross correlation of that image with \(2 I(\vec {r})I(\vec {r},+\Delta z)\),
each time using the shifted defocused image from the previous iteration for constructing the new reference.
The underfocused and overfocused image, as well as \(dI^\mathrm{{exp.}}_z \left( \vec {r_\bot }\right)\) computed according to (8) are shown in Fig. 8. Setting \(\zeta = 1.2\) converged to a \(\delta\) value of \(5.22\times 10^{5}\). Furthermore, the minimum of \(\chi ^2\) occurs at \(R_{LP} = 91.2\) μm (see Fig. 9). The algorithm was iterated for 20 epochs with the abovementioned parameters.
Figure 10 shows phase maps \(\varphi \left( \vec {r_\bot }\right)\) reconstructed by three different techniques: (a) the conventional FFT method applying (3) and (4), (b) the symmetrization mirror padding approach proposed by Volkov et al. [15], and (c) the GF scheme proposed here. For both the conventional FFT method (a), as well as the proposed GF algorithm (c) the experimental data were padded as described above to result in data of twice the original image dimensions.
All three phase maps shown in Fig. 10 are consistent with the experimental data, but the applied boundary conditions differ. The reconstructed phase maps shown in Fig. 10a, b are unphysical, because in both cases the phase shift inside the wing drops below the phase shift in the empty area. The FFT reconstruction shown in Fig. 10a also features a severe overall phase slope in the empty area which cannot be deemed physical.
The line profiles across the reconstructed phase maps show good agreement between the three different reconstruction results for fine details, but they also highlight the large differences at low spatial frequencies.
Conclusion
In this work, we proposed a simple iterative algorithm, gradient flipping (GF), which imposes sparsity on the phase gradient by either driving a certain percentage of values to zero, or forcing all values below a certain positive threshold to zero. By combining the conventional Fourier method with these principles adapted from the chargeflipping algorithm in crystallography, GF determines boundary conditions on the phase, while preserving consistency with the higher frequencies of the experimental data.
It was shown with simulations and experiments of nonperiodic and nonpiecewise linear objects that these boundary conditions contribute to GF’S much improved lower spatial frequencies compared to that of the more conventional FFT method and symmetrization method.
References
 1.
Gabor, D., et al.: A new microscopic principle. Nature 161(4098), 777–778 (1948)
 2.
Gerchberg, R.W., Saxton, W.O.: A practical algorithm for the determination of the phase from image and diffraction plane pictures. Optik 35, 237–246 (1972)
 3.
Kirkland, E.J.: Improved high resolution image processing of bright field electron micrographs: I. theory. Ultramicroscopy 15(3), 151–172 (1984)
 4.
P. Schiske. Zur Frage der Bildrekonstruktion durch Fokusreihen. In Proc. 4th Europ. Congr. on Electron Microscopy, Rome, Vol. 1, pages 145–146, 1968
 5.
Michael, T.R.: Deterministic phase retrieval: a green’s function solution. J. Opt. Soc. Am. 73(11), 1434–1441 (1983)
 6.
Paganin, D., Nugent, K.A.: Noninterferometric phase imaging with partially coherent light. Phys. Rev. Lett. 80, 2586–2589 (1998)
 7.
Van Dyck, D., Op de Beeck, M., Coene, W.: A new approach to object wave function reconstruction in electron microscopy. Optik 93, 103–107 (1993)
 8.
Streibl, Norbert: Phase imaging by the transport equation of intensity. Opt. Commun. 49(1), 6–10 (1984)
 9.
Sheludko, D.V., McCulloch, A.J., Jasperse, M., Quiney, H.M., Scholten, R.E.: Noniterative imaging of inhomogeneous cold atom clouds using phase retrieval from a single diffraction measurement. Opt. Expr. 18(2), 1586–1599 (2010)
 10.
Zuo, Chao, Chen, Qian, Weijuan, Qu, Asundi, Anand: Direct continuous phase demodulation in digital holography with use of the transportofintensity equation. Opt. Commun. 309, 221–226 (2013)
 11.
Leong, A.F., Paganin, D.M., Hooper, S.B., Siew, M.L., Kitchen, M.J.: Measurement of absolute regional lung air volumes from nearfield xray speckles. Opt. Expr. 21(23), 27905–27923 (2013)
 12.
Lubk, Axel, Guzzinati, Giulio, Börrnert, Felix, Verbeeck, Jo: Transport of intensity phase retrieval of arbitrary wave fields including vortices. Phys. Rev. Lett. 111(17), 173902 (2013)
 13.
Parvizi, A., Müller, J., Funken, S.A., Koch, C.T.: A practical way to resolve ambiguities in wavefront reconstructions by the transport of intensity equation. Ultramicroscopy 154, 1–6 (2015)
 14.
Gureyev, T.E., Raven, C., Snigirev, A., Snigireva, I., Wilkins, S.W.: Hard xray quantitative noninterferometric phasecontrast microscopy. J. Phys. D: Appl. Phys. 32(5), 563 (1999)
 15.
Volkov, V.V., Zhu, Y., De Graef, M.: A new symmetrized solution for phase retrieval using the transport of intensity equation. Micron 33(5), 411–416 (2002)
 16.
Gureyev, T.E., Roberts, A., Nugent, K.A.: Phase retrieval with the transportofintensity equation: matrix solution with use of zernike polynomials. JOSA A 12(9), 1932–1941 (1995)
 17.
Gureyev, T.E., Nugent, K.A.: Phase retrieval with the transportofintensity equation. ii. orthogonal series solution for nonuniform illumination. JOSA A 13(8), 1670–1682 (1996)
 18.
Zuo, C., Chen, Q., Asundi, A.: Boundaryartifactfree phase retrieval with the transport of intensity equation: fast solution with use of discrete cosine transform. Opt. Expr. 22(8), 9220–9244 (2014)
 19.
Tian, L., Petruccelli, J.C., Barbastathis, G.: Nonlinear diffusion regularization for transport of intensity phase imaging. Opt. Lett. 37(19), 4131–4133 (2012)
 20.
Kostenko, A., Batenburg, K.J., Suhonen, H., Offerman, S.E., Vliet, L.J.: Phase retrieval in inline xray phase contrast imaging based on total variation minimization. Opt. Expr. 21(1), 710–723 (2013)
 21.
Zhu, Y., Shanker, A., Tian, L., Waller, L., Barbastathis, G.: Lownoise phase imaging by hybrid uniform and structured illumination transport of intensity equation. Opt. Expr. 22, 26696–26711 (2014)
 22.
OzsoyKeskinbora, C., Boothroyd, C.B., DuninBorkowski, R.E., van Aken, P.A., Koch, C.T.: Hybridization approach to inline and offaxis (electron) holography for superior resolution and phase sensitivity. Scientific. Rep. 4, (2014)
 23.
Jingshan, Z., Claus, R.A., Dauwels, J., Tian, L., Waller, L.: Transport of intensity phase imaging by intensity spectrum fitting of exponentially spaced defocus planes. Opt. Exp. 22(9), 10661 (2014)
 24.
Jelena A Schmalz, Timur E Gureyev, David M Paganin, and Konstantin M Pavlov. Phase retrieval using radiation and matterwave fields: Validity of teague’s method for solution of the transportofintensity equation. Phys. Rev. A, 84(2):023808, 2011
 25.
Petersen, T.C., Keast, V.J., Johnson, K., Duvall, S.: Tembased phase retrieval of pn junction wafers using the transport of intensity equation. Philos. Mag. 87, 3565–3578 (2007)
 26.
Waller, L., Luo, Y., Yang, S.Y., Barbastathis, G.: Transport of intensity phase imaging in a volume holographic microscope. Opt. Lett. 35(17), 2961–2963 (2010)
 27.
Allen, L.J., Oxley, M.P.: Phase retrieval from series of images obtained by defocus variation. Opt. Commun. 199(1), 65–75 (2001)
 28.
Zheng, S., Xue, B., Xue, W., Bai, X., Zhou, F.: Transport of intensity phase imaging from multiple noisy intensities measured in unequallyspaced planes. Opt. Expr. 20(2), 972–985 (2012)
 29.
Pinhasi, S.V., Alimi, R., Perelmutter, L., Eliezer, S.: Topography retrieval using different solutions of the transport intensity equation. J. Opt. Soc. Am. A 27, 2285–2292 (2010)
 30.
Oszlányi, Gábor, Suto, Andras: The charge flipping algorithm. Acta Crystallographica Section A: Foundations of Crystallography 64(1), 123–134 (2007)
 31.
Ophus, Colin, Ewalds, Timo: Guidelines for quantitative reconstruction of complex exit waves in HRTEM. Ultramicroscopy 113, 88–95 (2012)
Authors’ contributions
AP carried out the experimental demonstration as well as reconstruction algorithm and drafted the manuscript. WB improved the theory behind the reconstruction algorithm and helped to draft the manuscript. CTK wrote the reconstruction algorithm and finalized the manuscript. All authors read and approved the final manuscript.
Acknowledgements
The authors acknowledge funding from the Carl Zeiss Foundation.
Competing interests
The authors declare that they have no competing interests.
Author information
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 Transport of intensity equation
 Boundary condition
 Gradientflipping algorithm