 Research
 Open Access
 Published:
Poisson noise removal from highresolution STEM images based on periodic block matching
Advanced Structural and Chemical Imaging volume 1, Article number: 3 (2015)
Abstract
Scanning transmission electron microscopy (STEM) provides subångstrom, atomic resolution images of crystalline structures. However, in many applications, the ability to extract information such as atom positions, from such electron micrographs, is severely obstructed by low signaltonoise ratios of the acquired images resulting from necessary limitations to the electron dose. We present a denoising strategy tailored to the special features of atomicresolution electron micrographs of crystals limited by Poisson noise based on the blockmatching and 3Dfiltering (BM3D) algorithm by Dabov et al. We also present an economized blockmatching strategy that exploits the periodic structure of the observed crystals. On simulated singleshot STEM images of inorganic materials, with incident electron doses below 4 C/cm ^{2}, our new method achieves precisions of 7 to 15 pm and an increase in peak signaltonoise ratio (PSNR) of 15 to 20 dB compared to noisy images and 2 to 4 dB compared to images denoised with the original BM3D.
Background
Modern electron microscopy allows for atomic resolution images of crystalline structures [1]. However, the full scope of resolution can be exploited only for materials with little electron beam sensitivity. Lowering the electron dose decreases the signaltonoise ratio (SNR) of the acquired micrographs accordingly, degrading the quality or even completely prohibiting the extraction of desired information from the noisy micrographs. Examples of inorganic materials with high beam sensitivity, where scanning transmission electron microscopy (STEM) images of poor SNR have to be used, are both oxide [2] and metallic [3] catalysts. One important quantity that may be extracted from atomicresolution electron micrographs is the positions of the atoms. The precision with which these can be determined is crucial for the understanding of certain material properties [4,5]. For singleshot STEM images, the best reported precision is about 15 pm [6]. A common technique to increase the precision is to register a series of frames and extract the atom positions from the average of the registered frames. Kimoto et al. [7] achieved a precision of 5 pm using rigid registration. More recently, a nonrigid registration scheme has been developed by Berkels et al. [2] that has achieved subpicometer precision for STEM series [8]. Both registration schemes require many individual frames of moderate SNR resulting in a veryhigh overall electron dose applied to the material. Thus, to widen the applicability of STEM to more beamsensitive inorganic materials, a central objective is to develop effective denoising methods that increase the singleshot image quality. This would enable the extraction of desired information from individual frames or the use of the individual frames for registration, while using a lower electron dose.
Denoising in materials, electron microscopy is often accomplished by simple spatial filters like a Gaussian blur or median filter or frequency space lowpass or Wiener filters (cf. [911]). These examples are all taken from materials robust enough under the electron beam to survive relatively high electron dose to obtain images with high spatial oversampling (small pixels). More sophisticated approaches that take advantage of the highly redundant nature of some highresolution STEM images have been employed [12], building on tools developed for very beamsensitive biological samples [1316]. While these methods, mostly based on manipulations of the image in Fourier space, are well established and have proven to perform well on highresolution transmission electron microscopy (HRTEM) images of organic materials, they are tightly linked to the periodicity of the data and have thus not been widely adopted in STEM studies of aperiodic defects in inorganic materials.
The most successful generic image denoising method for arbitrary images available today rely on nonlocal detection and averaging of selfsimilar image regions. The first algorithm based on this strategy is the nonlocal means filter (NLM) by Buades et al. [17]. Due to the richness in selfsimilarity of electron micrographs of crystals, NLM is in principle very well suited for denoising such micrographs [18], and it has been employed in biological electron microscopy [19]. However, two main properties of lowdose electron micrographs of crystals are disregarded by the original design and implementation of NLM, namely (1) NLM removes additive Gaussian white noise (AGWN) instead of Poisson noise, which is the dominant form of noise in lowdose STEM images; and (2) NLM uses a local similarity search to reduce the computational cost, whereas the distance between selfsimilar points in images of crystals is at least as large as the interatomic distance. In the case of STEM, there is an additional difficulty caused by image distortions resulting from the serial acquisition of the pixel data. In view of these issues, we proposed an enhanced version of NLM tailored to STEM imaging [20]. The most significant enhancement was the development of an efficient nonlocal similarity search, based on the generation of periodic lattices in Fourier space [21]. However, the basic NLM principle can be further improved by replacing the simple weighted average of intensities of pixels with similar neighborhoods by a more advanced collaborative filtering of the corresponding image parts. Several methods of this type have been proposed over the past few years, e.g., optimized blockwise NLM by Coupé et al. [22] and NLPCA by Salmon et al. [23], which is based on principal component analysis. Among the most successful variants is the blockmatching and 3Dfiltering algorithm (BM3D) developed by Dabov et al. [24], which has become a benchmark for image denoising algorithms in the field of image processing. The extension of BM3D to Poisson noise removal via application of the Anscombe variancestabilizing transformation [25] has already been proposed by Mäkitalo and Foi [26] who have contributed an exact unbiased inverse transformation that increases the accuracy especially in the lowcount regime. In [20], it was found that despite being restricted to a local similarity search, the original BM3D filter with extension to Poisson noise outperformed the proposed periodic search NLM filter. Thus, the starting point here is the BM3D filter.
In this paper, we discuss how modifications introduced in STEMtailored NLM [20] can be incorporated into the stateoftheart denoising algorithm BM3D in order to tap its full potential on electron micrographs of crystals. More precisely, as this method has been designed for Gaussian white noise, a central task is to develop suitable modifications that effectively deal with Poisson noise. Another focus is to exploit the atomic lattice structure that entails a repeated appearance of selfsimilar image components within a single frame. While we focus on the application of STEM imaging of inorganic materials, the key features of Poisson noise and oversampling are shared by HRTEM images as well, so the proposed method should also be applicable to HRTEM images. Also note that the procedures proposed in this paper can be applied subsequently for registering a series of lowdose frames to a single reference frame [2].
The paper is organized as follows. First, we briefly recall the original BM3D algorithm. Then, we present two strategies for Poisson noise removal, namely variance stabilization based on the Anscombe transform and using Poisson maximumlikelihood similarity measures due to Deledalle et al. [27] for the block matching. The main contribution is an adaptive nonlocal periodic block matching. It exploits the repetitive structure within the micrograph while keeping the computational cost affordable. Since the spatial distribution of the resulting block estimates is highly nonuniform, potentially resulting in a poor reconstruction in regions with few available block estimates, we discuss possible remedies of this deficiency. Finally, we evaluate the proposed method on simulated data, where the ground truth, i.e., the true, noisefree mean electron count per pixel, is available, showing the improvement both in terms of visual image quality and in quantitative measures such as peak signaltonoise ratio (PSNR), precision, and fidelity achieved by the proposed method.
Methods
Block matching and 3D filtering
BM3D was developed on the basis of NLM. It was originally designed to remove additive Gaussian white noise from natural images or other images exhibiting a sufficient amount of selfsimilarity. The main idea of the algorithm is to find similar parts within an image and remove noise from those parts by sparsifying their common representation in a 3D transform domain. After processing all image parts in this manner, one receives an overcomplete representation of an estimate of the noisefree input image. Averaging all partial estimates in overlapping regions provides an estimate of the full underlying ground truth image. The denoising process in the 3D transform domain is performed in two stages, namely an initial hard thresholding succeeded by a final Wiener filtering. In the following, we briefly recall the definition of the original BM3D filter as published in [28], which serves as a basis for the methods developed in this work. We use a slightly simplified version of the algorithm described in [28] but with the exception of a change of the block size from 8×8 to 16×16, all relevant parameters, as well as the unitary transforms in the 3D domain, are chosen in exact agreement with the original implementation of the BM3D filter [29] when using the default profile, i.e., normal profile.
The original BM3D algorithm for Gaussian noise removal
Let us regard an image:
where f is the (noisefree) ground truth and \( \eta \left(\cdotp \right)\sim \mathcal{N}\left(0,{\sigma}^2\right) \) is AGWN, i.e., a normally distributed random variable with zero mean and standard deviation σ.
Let us fix some \( n\in \mathbb{N} \) with n<N and n<M and some reference pixel:
and denote by Z _{ x } a block of size n×n with upperleft corner \( x\in \overset{\kern0.60em \circ }{X} \). The size n (in units of pixels) should be chosen such that an n×n block is roughly between 0.25 and 1.25 times as large as an atom within the image. Furthermore, we denote with Z _{ x }(y) for y∈X the values of the block in global coordinates, and with Z _{ x }[ i,j] :=Z _{ x }(x _{1}+i−1,x _{2}+j−1) for \( i=1,\dots, n \), its values in local coordinates. Note that we set Z _{ x }(y)=0 outside the block’s support. The blockmatching part of BM3D consists of a local search for a tuple of blocks that are similar to Z _{ x } in the sense that their normalized L ^{2}distance from Z _{ x } is less than some threshold τ>0. More precisely, denoting by N _{ S } the size of the local search window and by \( {\mathcal{N}}_{N_S}(x) \) the N _{ S }×N _{ S } neighborhood centered around \( x\in \overset{\kern0.60em \circ }{X} \), let:
where:
We then sort the set S _{ x } by block similarity, i.e., the set S _{ x } becomes a tuple with \( d\left({Z}_x,{Z}_{{\left({S}_x\right)}_i}\right)\le d\left({Z}_x,{Z}_{{\left({S}_x\right)}_j}\right) \) for 1≤i≤j≤#(S _{ x }). From all matched coordinates in S _{ x }, we now choose the ordered subset \( {\check{S}}_x=\left(\right.{\left({S}_x\right)}_i{\left)\right.}_{i=1}^{N_x} \) consisting of the \( {N}_x:= \min \left\{\#\left({S}_x\right),{N}_{3\mathrm{D}}\right\} \) most similar blocks. Here, N _{3D} is some small number that controls the computational cost. Note that the choice of N _{3D} may also implicitly change the original threshold τ in (3) to a possibly smaller one. Stacking the bestmatched blocks \( {Z}_y,y\in {\check{S}}_x \) on top of each other yields an n×n×N _{ x } 3D tensor:
with high correlation along the coordinate k. Figure 1 shows such a column of stacked blocks using a STEM image as an example. Due to the correlation along the coordinate k (indicated by a blackdotted arrow in Figure 1), an efficient denoising of the tensor \( {\mathcal{Z}}_x \) can now be performed as follows. Let be some 3D unitary transformation (e.g., Fourier or wavelet transform), and let Υ _{ θ } denote the hardthresholding operator with threshold θ, which is chosen with some fixed proportionality to the standard deviation σ of the noise. The denoising procedure for the 3D tensor \( {\mathcal{Z}}_x \) consists of a hard thresholding of the coefficients of the transformed tensor \( \mathcal{T}\left({\mathcal{Z}}_x\right) \), followed by the inverse transformation. This leads to the following estimator:
for the tensor \( {\mathcal{Z}}_x \). A typical transformation would be based, e.g., on a wavelet basis. To this estimator, one assigns a weight \( {w}_x={N}_{\theta}^{1} \), where N _{ θ } is the number of retained nonzero transform coefficients after hard thresholding.
Such block estimators are calculated for each reference point in a subset \( {X}_R\subset \overset{\kern0.60em \circ }{X} \) of all available block corners, given by:
Choosing N _{step}<n, the corresponding reference blocks overlap. This guarantees an overcomplete representation of the estimated image. An intermediate estimator \( {\widehat{f}}^{\mathrm{basic}} \) for the desired image f is then defined as a weighted average over all overlapping block estimations. More precisely, let \( {\hat{Y}}_{x,k}:={\widehat{\mathcal{Y}}}_x\left[\cdotp, \cdotp, k\right] \), k=1,…,N _{ x }, denote the kth slice of the estimator for \( {\mathcal{Z}}_x \), which is a 2D block estimate corresponding to the block \( {Z}_{{\left({\check{S}}_x\right)}_k} \) and set:
where χ _{ Z } denotes the characteristic function of the block \( Z\subset X \). Note that each block Z _{ y } may be matched to different references \( {Z}_x,{Z}_{x^{\prime }} \), resulting in different estimates \( {\hat{Y}}_{x,k},{\hat{Y}}_{x^{\prime },{k}^{\prime }} \) for the same block, i.e., \( {\left({\hat{S}}_x\right)}_k={\left({\hat{S}}_{x^{\prime }}\right)}_{k^{\prime }}=y \) in this case.
The final estimate \( \widehat{f} \) of the desired image f is now calculated in a second iteration, where the whole procedure described above is repeated with some modifications briefly indicated next, referring to [28] for the details. Denoting by F _{ x }, the n×n block with upperleft corner x extracted now from the basic estimate \( {\widehat{f}}^{\mathrm{basic}} \), the above procedure is applied to the blocks F _{ x } in place of Z _{ x }, where the analogs to the sets S _{ x } in (3) are formed, however, with respect to a possibly different threshold parameter τ ^{′}. Moreover, the analog to (6) may involve a different unitary transformation (e.g., using local cosine transforms) and, perhaps more importantly, hard thresholding is replaced by Wiener filtering. Again, we refer to [28] for the details.
Figure 1 illustrates the BM3D procedure using a STEM image as an example. It demonstrates the three main steps of BM3D, which are performed for each reference block: 1) matching blocks to a reference and stacking the matched blocks into a 3D tensor, 2) denoising this column of matched blocks, and 3) aggregating the resulting denoised blocks at their original positions and averaging overlapping parts of adjacent blocks (green). For simplicity, the illustration uses simple averaging along the column of matched blocks (black dotted arrow) as an example procedure for denoising the 3D tensors. In practice, the approaches described above provide better performance.
Like any blockaveraging or nonlocal means approach, our method has reduced performance near the edges of the image. Since N _{step}<n for every y∈X, there is a block with corner x∈X _{ R } such that y∈Z _{ x }. In other words, for any pixel in the image (even at the boundary), there is a block containing this pixel, resulting in an estimated intensity of the pixel after the corresponding block has been denoised in 3D transform domain. But because there exist fewer overlapping blocks near the image boundaries, there are fewer available estimates of boundary pixels that can be used for further averaging. Therefore, both the basic and final estimates \( {\widehat{f}}^{\mathrm{basic}} \) and \( \widehat{f} \), i.e., the intermediate result and the final denoised image, will show a slightly reduced quality towards the image boundaries. However, the denoising of each block together with its matched blocks in 3D transform domain contributes much more to the image quality than the averaging of spatially overlapping blocks. Thus, the reduced quality at the boundaries of images denoised with BM3D is usually not substantial in practice.
Extension to Poisson noise removal via variance stabilization
As mentioned before, BM3D is designed to remove additive Gaussian white noise of some uniform standard deviation σ from an input image. In order to adapt this filter to Poisson noise removal, Mäkitalo and Foi [26] proposed to use a variancestabilizing transform (VST) that approximately transforms any given image affected by Poisson noise, i.e., of the form:
into an image with AGWN of standard deviation one. Here, v∼Pois(λ) denotes a Poisson distributed random variable with mean (and variance) λ, i.e.:
The transformation:
is known as the Anscombe transform [25]. Mäkitalo and Foi proposed an exact unbiased inverse transform of the form [26]:
Here, given random variables V,W, \( \mathbb{E}\left\{V\BigW\right\} \) denote the expectation of V given W. In their implementation [30], Mäkitalo and Foi provide tabulated values of this inverse for arguments within the range [ A(0),100]. For smaller arguments, the inverse is extended by zero, and for higher arguments, the asymptotic unbiased inverse transform is used, see [25].
The procedure for Poisson noise removal is then given by the following threestep procedure: first, apply the forward Anscombe transform on the input image v, receiving an input image z=A(v) with a noise model similar to AGWN; second, apply the BM3D filter to this image z, receiving an estimate \( \widehat{f} \) of E{A(v)λ}; and lastly, apply the exact unbiased inverse Anscombe transform on \( \widehat{f} \) to obtain an estimate \( \widehat{\lambda}:={A}^{1}\left(\widehat{f}\right) \) of λ.
Direct block matching for Poisson noise statistics
As pointed out by Salmon et al., the Anscombe transform is not accurate in the extreme lowcount regime, i.e., if the average number of counts λ(x) is smaller than three for some x∈X (cf. [23]). STEM images with such a small number of counts are generally deemed useless, so they are rarely acquired or published. However, the ability to extract information from extreme lowdose images would be a substantial advantage for a variety of problems, such as characterization of metallic catalyst particles [3] and polymers and molecular crystals [31].
The performance of two variants of nonlocal means on images affected by Poisson noise are compared in [20]. The first one features the Anscombe transform and the standard L ^{2}patch similarity measure, while the second one uses a Poisson maximumlikelihoodbased patch similarity measure from [27] without a preceding VST on the input data. The results show that when the PSNR of the input image is below 10 dB, the Poisson maximumlikelihood ratiobased patch distances outperform the strategy of using standard L ^{2}distances on the Anscombe transformed data.
The idea of using a Poisson maximumlikelihoodbased similarity measure can be easily adopted to block matching when calculating the basic estimate via hard thresholding within the BM3D algorithm, see (3), (6), (8). To this end, we replace the L ^{2}distance (4) by the following distance:
where f is defined through the Poisson maximumlikelihood ratio:
as follows [27]:
Using the distance (13), we then replace the set of matched blocks S _{ x } (cf. (3)) with:
where τ ^{P} is a new threshold related to the maximumlikelihood ratios (14). Note that \( {d}_P\left({Z}_x,{Z}_y\right)< \log \left({\tau}^P\right) \) is equivalent to:
i.e., τ _{ P } is in fact a threshold on the geometric mean of the maximumlikelihood ratios. Now, since:
we have that 0≤pml(k _{1},k _{2})≤1 for any \( {k}_1,{k}_2\in \mathbb{N} \). From this, it follows that also \( \overline{\mathrm{pml}}\left({Z}_x,{Z}_y\right) \), the geometric mean of the maximumlikelihood ratios between the blocks Z _{ x } and Z _{ y } takes values between 0 (very bad match) and 1 (perfect match). Thus, the corresponding threshold τ ^{P} should be chosen between 0 (no thresholding) and 1 (only accepting identical blocks). We found that the choice τ ^{P}=0.55 works well in practice and thus we use this value wherever the Poisson maximumlikelihood ratios are employed in this work. The reduced set \( {\check{{S}_{x}^{P}}} \) and the block stack \( {\mathcal{Z}}_x^P \) are defined analogously to \( {\check{S}}_x \) and \( {\mathcal{Z}}_x \), respectively, just replacing S _{ x } by \( {S}_x^P \).
Since the denoising in 3D transform domain is designed to remove AGWN, we still have to use the Anscombe transform before applying to the 3D block, i.e., the estimated 3D block stacks (6) are replaced by:
In other words, when using this strategy, the Anscombe transformed data is used to fill the 3D block stacks with the values of the matched blocks but not to determine the positions of the matched blocks in the 3D stacks.
The weights \( {w}_x^P={N}_{\theta}^{1} \) are again equal to the number of nonzero transform coefficients after hard thresholding the Anscombe transformed block stack in the 3D transform domain. Accordingly, the basic estimate (8) is replaced by:
Here, \( {\hat{Y}}_{x,k}^P(y) \) is defined in analogy to (8). Also note that due to the application of the Anscombe transform to the 3D block stacks in (19), the values of the basic estimate (20) after Poisson maximumlikelihoodratiobased block matching are still in the Anscombe transform domain. Thus, the Wiener filtering iteration remains unchanged.
Adaptive nonlocal periodic block matching
Images of crystals at atomic scale acquired by electron microscopy exhibit highly repetitious patterns of only a few basic objects corresponding to the different types of atoms. Therefore, such images are rich in selfsimilarity; however, blocks of equal structure are typically not found in local neighborhoods. Based on this observation, we developed a periodic search strategy for nonlocal means that is based on a Fourier analysis of the input image and distributes small nonlocal search windows periodically throughout the entire image [20]. In our previous work, we found that a significant increase in both visual image quality and of quantitative measures such as the PSNR can be achieved using this periodic search strategy. Nevertheless, we also found that BM3D, even with local search, in most cases still outperforms our proposed periodic search nonlocal means. Still, the local search used in the original BM3D filter is expected not to be well suited for most electron microscopy images of crystals. Hence, we now propose to combine our periodic search strategy for block matching with the BM3D filter for electron microscopy images. First, we recall the definition of the nonlocal periodic search grid [20]:
Here, α _{1},α _{2} denotethe angles or directions of the periodic pattern within the input image and Δ x _{1},Δ x _{2} the spacings between the selfsimilar objects along those axes and \( {N}_S^{\pi } \) is the size of the small local search windows (in units of pixels). For an automatic estimation strategy of these parameters from the input image, we refer to [20]. Note that the \( \arg \min \) expression causes an adaptive reset of the search pattern that gives some robustness against errors in both the estimation of the grid parameters and slight variations of the periodic pattern within the image.
The search grid (21) is used to replace the local search window \( {\mathcal{N}}_{N_S}(x) \) in the set of matched blocks (3) in the hard thresholding iteration and analogously in the Wiener filtering iteration with the sets:
Here, the super scripts ht and wie refer to the first and second stage involving hard thresholding and Wiener filtering. In this paper, we have used small \( {N}_S^{\pi}\times {N}_S^{\pi } \) local search windows of size \( {N}_S^{\pi }=5 \). We found that this choice retains reasonable computational efficiency, while being large enough (about 0.1 to 0.25 times as small as the atoms in the images we used) to correct the positions of the nonlocal periodic search steps within each atom. \( {\check{\varPi}}_x^{\mathrm{ht}} \) and \( {\check{\varPi}}_x^{\mathrm{wie}} \) are defined analogously to \( {\check{S}}_x \). Note that for crystals at atomic scale the selfsimilarity within the corresponding image is so rich that with a nonlocal search like this, for nearly all x∈X _{ R }, one finds many more than N _{3D} blocks that are similar to the reference at x, i.e., N _{ x }=N _{3D} holds for nearly all x∈X _{ R }.
In order to reduce the computational cost of the nonlocal periodic search, we made the following technical adjustments to its implementation in comparison with its description in [20] and the expression (21). First, for each reference point x among the two directions α _{1} and α _{2}, the corresponding axis with the largest intersection with the image X is declared to be the primary search axis. Then, after performing one step along the primary search axis, nonlocal periodic search steps along both positive and negative directions of the secondary search axis are carried out only until the image boundary is reached. This process is repeated until the image boundary is also reached along the positive and negative direction of the primary search axis. On the one hand, this implementation requires less computational cost than computing the whole set of points within the image that could be reached by steps along either of the two periodicity axes, while on the other hand, it still gives a sufficiently large subset of all of these points. This efficient periodic blockmatching strategy increases the computational cost by a factor of about 1.5 to 2 (depending on the density of the atoms within the image) compared to our implementation of BM3D with local block matching. Note that due to the overhead produced by unoptimized parts of our implementation this relative comparison may not be exact. Nevertheless, we believe that the factor between the runtimes of local and periodic block matching would be of the same order of magnitude in optimized code as well.
Uniform distribution of block estimates
A key difference between the nonlocal means algorithm and BM3D is that the estimated intensity \( \widehat{f}(y) \) in some pixel y∈X does not only depend on the similarity between the block Z _{ y } (if \( y\in \overset{\kern0.60em \circ }{X} \)) and other blocks Z _{ x }, \( x\in \overset{\kern0.60em \circ }{X} \), in the image but also on the similarity between overlapping blocks \( {Z}_{x^{\prime }}\ni y \), \( {x}^{\prime}\in \overset{\kern0.60em \circ }{X} \), and other blocks Z _{ x }. This is due to the mechanism in BM3D that aggregates the intensities of all pixels of all denoised blocks at their original positions in order to obtain the estimate (8) from the overcomplete representation given by the collection of all block estimates. As noted before, there can even be different block estimates corresponding to the same position, if they originate from denoised 3D block stacks matched to different references. Therefore, when applying the BM3D filter, the preciseness of the intensity estimate \( \widehat{f}(y) \) may not only depend on the block distances of all references to their matched blocks but also on the distribution of the positions corresponding to all block estimates. For adaptive periodic blockmatching, this is:
Analogously, we define n _{aggr} for local block matching but with \( {\check{\varPi}}_x \) replaced by \( {\check{S}}_x \). Figure 2 shows these quantities for both the hard thresholding and Wiener filtering iteration. The images B and D in Figure 2 reveal that the periodic block matching may lead to a significant concentration of block estimates in certain regions. Consequently, the number of block estimates has to be very low elsewhere. The effect is much weaker with the local block matching (cf. Figure 2A,C) since the local search windows naturally restrict blocks from one area to be matched to a small set of blocks within their neighborhood, which prevents the number of aggregates to concentrate significantly in one specific area of the image.
Here, we propose a modification to the adaptive periodic blockmatching strategy that aims at steering it towards a more uniform distribution of the positions of all block estimates as observed for the original local blockmatching method. The task is to maximize the minimal number of aggregates while still choosing blocks with least distances during block matching. As we consider block similarity to be the more important than uniformity of the block estimates, we still primarily sort the matched blocks by block distance. However, we create additional degrees of freedom by choosing a slightly larger limit \( {N}_{3\mathrm{D}}^{\ast }>{N}_{3\mathrm{D}} \) of the number of matched blocks than before. This gives larger sets \( {\varPi}_x^{\ast }={\left({\varPi}_x\right)}_{i=1}^{N_x^{\ast }} \), where \( {N}_x^{\ast }:= \min \left\{\#\left({\varPi}_x\right),{N}_{3\mathrm{D}}^{\ast}\right\} \). Reducing these sets for all reference points x∈X _{ R } to the final size N _{ x } while driving the distribution of the resulting block estimates throughout the image towards a more uniform one gives rise to the following optimization problem:
where \( \mathcal{P}:=\left\{{\left({\varPi}_x^{\prime}\right)}_{x\in {X}_R}:{\varPi}_x^{\prime}\subset {\varPi}_x^{\ast },\#\left({\varPi}_x^{\prime}\right)={N}_x,\kern0.60em \forall x\in {X}_R\right\} \) denotes the feasible set. Unfortunately, this optimization problem is of combinatorial type and due to its global coupling of all reference coordinates would be computationally too costly to solve. Hence, we propose the following simplification of the uniform block matching through an iterative procedure.
Here, the reference coordinates have been numbered \( {X}_R=\left({x}^1,\dots, {x}^{\#\left({X}_R\right)}\right) \) and block matching is performed for one after the other in an iterative fashion. In this paper, we use the number of local \( {N}_S^{\pi}\times {N}_S^{\pi } \) search windows as the limit \( {N}_{3\mathrm{D}}^{\ast } \). Note that as the number of search window is dependent on the reference position x∈X _{ R }, so is the new limit \( {N}_{3\mathrm{D}}^{\ast }={N}_{3\mathrm{D}}^{\ast }(x) \), unlike the limit N _{3D} chosen before, which is independent of x. This choice can be motivated as follows: if the adaptive periodic search succeeds in placing the local search windows roughly according to the pattern of the observed crystal, then for a reference pixel within an atom, each local search window within the corresponding similarity search should overlap a similar part of another atom. Therefore, we expect to find at least one wellmatching block per local search window. Note that we define \( {n}_{\mathrm{aggr}}^U \) in analogy to (23) but with \( {\check{\varPi}}_x \) replaced by \( {\check{\varPi}}_x^U \).
Results and discussion
We have presented several modifications of the original BM3D filter as it was proposed by Dabov et al. [24], including the extension to Poisson noise removal due to Mäkitalo and Foi [26]. The main intention was to improve the filter within the context of electron microscopy images of crystals at atomic scale. To ensure a clear understanding of which modifications or settings were used to produce the results presented in this section, we use the following prefixes:

np ⋆: normal profile and Anscombebased threestep procedure

pml ⋆: Poisson maximumlikelihood ratiobased block matching within hard thresholding iteration

⋆l ⋆: local block matching with N _{ S }=39

⋆ π⋆: adaptive periodic block matching with \( {N}_S^{\pi }=5 \)

⋆ π N⋆: selection of N _{3D} bestmatched blocks with thresholds N _{3D}=16 (HT) and N _{3D}=32 (Wiener)

⋆ π U⋆: uniformly distributed selection of N _{3D} blocks from \( {\widehat{N}}_{3\mathrm{D}} \) bestmatched blocks as in Algorithm 1


⋆ ⋆n: block size is n×n
A commonly used quantitative measure to compare the image quality after application of different image processing algorithms is the PSNR of the corresponding estimates:
This quantitative measure requires the knowledge of the underlying ground truth, i.e., the exact average counts λ(x) in each pixel x∈X, which is not available in experimental STEM images. Therefore, we have simulated STEM images of various materials using the frozen phonon multislice algorithm [32] to obtain representative ground truth micrographs that are free of Poisson noise (cf. Figure 2A,C,E,G). To make these images representative of experimental images, typical image distortions that are caused by instabilities of the sample and electron beam during experimental STEM image acquisition have been artificially introduced to these simulated images. Known material crystallographic data was used to create the gallium nitride and silicon atomic models. Molecular dynamics was used to calculate the silicon dislocation atomic model [8]. In order to compare the algorithms at different noise levels, we scaled the intensities of the ground truth images to different average electron counts per pixel before applying random Poisson noise. This simulates the usage of different beam currents, resulting in different electron doses. Examples of Poisson noise affected versions of the simulated images can be found in Figure 3B,D,F,H.
In this work, we use our implementation of BM3D with local search as the ‘original’ BM3D benchmark. This comparison is justified, since we verified our implementation against the implementation provided by the original authors [29] and found the results to be consistent both in terms of the resulting peak signaltonoise ratios and a visual comparison of the retrieved estimates. Note that our implementation even gives slightly better PSNRs on most images, since it uses a full local search for each reference coordinate instead of reducing the size of the local search window based on previous blockmatching results, as it was suggested in [24]. However, due to both this fact and the lack of code optimization, our implementation is currently also slower by a factor of between 5 and 10, depending on the input.
At http://nmevenkamp.github.io/ELMA/, we provide a current version of our proposed method as C++ source code, as well as Windows and MacOSX applications with graphical user interface.
Adaptive nonlocal periodic block matching
A major goal of this work is to show that local block matching, with a search window small enough to warrant practical efficiency, is not able to properly benefit from globally recurring selfsimilar features. Instead, in such cases, real nonlocal blockmatching strategies are able to exploit the capabilities of the BM3D filter to a significantly higher extent.
We compare the PSNRs of the BM3D estimates for local block matching to those attained with our proposed nonlocal adaptive periodic block matching in Table 1 for the block sizes n=8 and n=16. The results clearly show that our proposed nonlocal periodic block matching leads to significantly better estimates (up to 4 dB gain) than the local block matching for all input images, even in the case of the silicon lattices with a defect, where the atoms are not arranged in a perfectly periodic pattern. This shows that the adaptive reset within the small local search windows defined in (21) yields sufficient robustness to cope with slight deviations from a perfect periodic structure of the input image. Figure 4 compares the resulting visual image quality when using local (left column) or periodic (right column) block matching within the BM3D filter. Comparing with the ground truths in Figure 3, one sees that both the local and the periodic blockmatching methods smooth out the scan distortions. This effect is due to the nonlocal averaging of blocks extracted from essentially identical atoms that just differ slightly in shape due these distortions. Since the local distortion of a scan line with respect to the atom center is mostly random, these distortions cancel out on average. In Figure 4A,B,C,D,E,F, one sees that for the cases of perfect gallium nitride and silicon lattices at very low PSNR, the adaptive periodic block matching leads to a significantly better estimate than the local block matching. The example shown in Figure 4G,H, where a silicon lattice with a dislocation was used, shows visually that even in the regions where the silicon atom pairs are slightly tilted, and where therefore selfsimilarity does not coincide with the global periodic pattern of the crystal, our proposed adaptive periodic block matching is robust enough to still find wellmatching blocks. The denoised images of the silicon lattice with a dislocation indicate that nonlocal averaging does not noticeably affect the orientation of the silicon atoms around the dislocation, despite the fact that most silicon dumbbells within the image have a vertical orientation.
Comparing the two middle columns with the two right columns in Table 1, one clearly sees that using the larger block size n=16 consistently yields significantly better results than using n=8. Note that there is one exception: the silicon lattice with dislocation at lowest PSNR (2.7 dB). However, the denoised images shown in Figure 5 confirm the importance of a larger block size for robust block matching, especially within the low electron count regime, as the estimates for n=16 (cf. Figure 5E,F) show an image quality far superior to the ones for n=8 (cf. Figure 5C,D). Again, regarding the case of the silicon lattice with dislocation (left column in Figure 5), one sees that the shapes of the atoms are still estimated more correctly using the larger block size n=16 (cf. Figure 5E). Notice however, that there appears to be a blurring of whole subregions within the estimate for n=16, which is not present for n=8 (cf. Figure 5C). This effect will be discussed in the next section.
Uniform distribution of block estimates
As described earlier, the adaptive periodic block matching, due to its true nonlocal similarity search, may lead to local concentrations of the number of available block estimates, thereby leaving fewer block estimates for other regions of the image. This effect can lead to an overall poor image quality in the image regions with few block estimates, as was observed in the case of the silicon lattice with dislocation in Figure 5E. In comparison to this, we see in Figure 5G,H the estimates retrieved from the uniform blockmatching strategy described earlier. Figure 5G shows an example of the improvement obtained by steering the distribution of the number of available block estimates towards global uniformity. Exemplary distributions of the number of aggregates after uniform periodic block matching are shown in Figure 6 for the gallium nitride lattice in Figure 3B. In Table 2, we compare our proposed plain adaptive periodic block matching versus the one with additional uniform distribution of the number of available estimates per pixel. First of all, the previous anomalous result for the silicon lattice with dislocation at lowest PSNR (2.7 dB), in which a block size of n=8 outperformed the larger block size of n=16 using plain adaptive periodic block matching, is resolved. Furthermore, the results in Table 2 show that for both the block sizes n=8 and n=16 uniform block matching increases the quality of the estimate in the majority of the cases, compared to plain periodic block matching without this addition, except for the silicon dislocation images with n=16. Our proposed threshold for the number of blocks that are regarded equally well matched (the number of nonlocal search windows) is not well suited for the rotated atoms near the dislocation. Therefore, for such images, reduced reliability is expected from the uniform blockmatching approach.
Direct block matching for Poisson noise statistics
Table 3 shows a comparison of (a) the PSNR of estimates retrieved by application of the BM3D filter in a threestep procedure using the Anscombe transform with (b) replacing the L ^{2}distances in the Anscombe transformed data within the block matching of the hard thresholding iteration by the Poisson maximumlikelihoodbased distances within the original data. The results indicate that at low PSNR of the input image, employing Poisson maximumlikelihood ratios is slightly better than using the L ^{2}distance within the Anscombe transformed data. For intermediate PSNR however, there seems to be very small or no difference between the two distance measures.
Although the improvement so far is small, we have just modified a small part of the BM3D algorithm to work directly on the original Poisson statistics, namely the blockmatching part within the initial hard thresholding iteration. The results we presented in [20] show a more significant advantage of the Poisson maximumlikelihood ratiobased similarity measure over the Anscombe transformed L ^{2}distances. However, this was for the NLM algorithm, where in the case of the Poisson maximumlikelihood ratiobased similarity measure, no Anscombe transform of the input data was required at all. Therefore, we believe that future research towards adopting the whole BM3D algorithm directly to Poisson noise statistics may lead to more substantial improvements of the reconstruction in the extreme lowcount regime.
Atomic column detectability, position precision, and reconstruction fidelity
As mentioned in the beginning, the positions of the atoms are an important quantity that material scientists would like to extract from atomicresolution electron micrographs. In the following, we analyze how well the positions can be estimated on the noisy images referred to in the previous sections and by how much our proposed denoising algorithm improves this estimation.
In order to extract the atom positions, we adopt a twostep procedure: first, the individual atoms within the image are detected through segmentation and the geometrical centers of the resulting atomic regions are used as an initial guess for the atom centers; second, a 2D Gaussian function (or a sum of two 2D Gaussian functions in the case of the silicon lattices) is fit on a small area around each atomic region via nonlinear regression in order to determine the final estimate of the position of each atom. For further details regarding this procedure, we refer to [2,33].
The detectability of the atomic columns in the noisy and denoised images can be assessed by the detection fraction. It is defined as follows. Let c _{ i }=c _{ i }(λ)∈[ 1,N]×[ 1,M], \( i=1,\dots, {N}_{\lambda } \) denote the centers of the atoms as detected from the ground truth λ and \( {\hat{c}}_j={\hat{c}}_j\left(\widehat{\lambda}\right)\in \left[\kern0.3em 1,N\right]\times \left[\kern0.3em 1,M\right] \), \( j=1,\dots, {N}_{\widehat{\lambda}} \) the ones detected from the corresponding noisy (or denoised) image \( \widehat{\lambda} \), using the same atomfinding procedure as described above. Furthermore, let \( \varDelta {c}_{\min }:=\underset{i,j=1,\dots, {N}_{\lambda }}{ \min}\left{c}_i{c}_j\right \) denote the shortest interatomic distance within the centers detected from the ground truth. Then, the detection fraction is defined as the fraction of atoms detected in the ground truth, which could be matched to a corresponding atom detected in the noisy (or denoised) image:
Here, the \( \min \) expression selects the atom detected in the noisy (or denoised) image, which is nearest to the one detected in the ground truth and the comparison with Δ c _{min} ensures that atoms are only matched if their distance is less than half the minimum interatomic distance. The latter is required for an unambiguous correspondence. Table 4 gives the detection fractions for noisy and denoised versions of low dose examples of perfect gallium nitride and silicon lattice images, as well as images of silicon lattices with a dislocation, with peak electron counts in the range of 6 to 49 electrons per pixel, corresponding to incident electron doses in the range of 0.55 to 28.47 C/cm ^{2}. These are low electron doses for inorganic materials, which is the main application intended for the algorithms presented in this paper. For organic and biological samples, however, a low electron dose is on the order of 0.01 C/cm ^{2}. The detection fraction is very small for the lower dose noisy images (0.55 to 5.68C/cm ^{2}), as well as for the local blockmatching BM3D estimate of the lowest dose gallium nitride (1.54 C/cm ^{2}) and silicon dislocation (00.55 C/cm ^{2}) lattices. For the other images (especially the silicon lattice), the detection fraction is significantly increased by denoising the image (even with local block matching). Furthermore, the periodic search BM3D method achieves detection fractions near or equal to 100% in all cases, except the lowest dose silicon dislocation lattice. In terms of the detection fraction, the uniform periodic block matching is superior to plain periodic block matching. While the difference is slight in most cases (up to 1%), the detection fraction is significantly increased by uniform block matching on the two lowest dose images, namely from 94.2% to 98% on the perfect silicon lattice (0.79 C/cm ^{2}) and from 48.4% to 92.5% on the silicon lattice with dislocation (0.55 C/cm ^{2}). Note that the detection fraction alone is not sufficient to assess the quality of the atomic column detection, as can be seen from the following example: say that due to the noise or poor quality of the image, the segmentation falsely detects many more atoms within the noisy (denoised) image than in the ground truth and that some of them are, by coincidence, near some of the atoms within the ground truth. In this case, the detection fraction could still be large. In order to cover this issue, we additionally give corresponding misdetection fractions in Table 5:
These specify the fraction of atoms detected from the noisy (denoised) image that could not be matched to any of the atoms detected from the ground truth. As seen in Table 5, these fractions are zero, except for the noisiest input images (6 to 15 peak electrons/pixel) and the local blockmatching BM3D estimate of the silicon images with doses 0.79 and 7.85 C/cm ^{2}, as well as the lowest dose silicon dislocation image (0.55 C/cm ^{2}). Note that for the periodic blockmatching BM3D the misdetection fractions are zero in all cases, except for the lowest dose silicon dislocation image. While the local blockmatching BM3D has a misdetection fraction of 100% in this case, the periodic and uniform periodic blockmatching BM3D methods achieve misdetection fractions of 12% and 4%, respectively.
A known quality measure for the atomic column position estimation is the socalled precision. It is defined as the standard deviation of the interatomic distances and ideally should be as small as possible for perfect single crystal samples because all the interatomic distances should be identical. In STEM, the precision is often regarded separately parallel and perpendicular to the scan (which is horizontal and vertical for all of the images here). However, we just intend to show that denoising improves the quality of the atomic column detection. Hence, we simply regard the total precision:
where N _{ Δ x },N _{ Δ y } are the number of, respectively, horizontal and vertical interatomic distances between neighboring atoms detected in the image \( \widehat{\lambda} \), Δ x _{ i },Δ y _{ j }, \( i=1,\dots, {N}_{\varDelta x} \), \( j=1,\dots, {N}_{\varDelta y} \) are the corresponding distances and \( \bar{\varDelta }x,\bar{\varDelta }y \) are their mean values, respectively. In Table 6, the precision, in picometers, is given for all images used in the atomic column detectability analysis above, except for the silicon dislocation images, where this measure is not meaningful due to the aperiodicity of the true atom positions. For images, where not a single pair of neighboring atoms could be detected, the precision is denoted by ‘n/a.’ Table 6 shows that this is the case for the noisy images with peak electron counts in the range of 6 to 12, as well as the local blockmatching BM3D estimate for the lowest dose gallium nitride lattice (1.54 C/cm ^{2}), which is in line with the small corresponding detection fractions seen in Table 4. For all other images, the according detection fractions in Table 4 are above 90% and the misdetection fractions in Table 5 are below 2%, which means that the precision is of relevance in these cases. Revisiting Table 6, we see that, except for the moderatedose gallium nitride image (3.85 C/cm ^{2}), where the precision is already quite high after local block matching, our proposed (uniform) periodic blockmatching BM3D is able to consistently enhance the precision by a factor of two, compared to local blockmatching BM3D. Even for the lowest dose input (0.79 C/cm ^{2}), we observe a precision below 15 pm, which is the best precision reported for single shot STEM images before [6]. However, our high precision is obtained in a simulated image, and to be completely relevant, it must be confirmed in experimental images. In the case of the gallium nitride lattice (3.85 C/cm ^{2}), where individual atoms could not reliably be detected in the noisy input, the uniform adaptive periodic BM3D estimate achieves a precision of below 7.5 pm, improving the precision by a factor of two compared to those reported in [6].
We conclude the ‘Results and discussion’ section with the analysis of another quality measure for the atom detection, which we call fidelity. We define this measure as the root mean square of the distances between corresponding atoms, as detected in both the ground truth and the noisy (denoised image):
As in the definition of the detection fraction (26), here the \( \min \) expression selects the nearest atom and the thresholding operator \( {Y}_{\frac{1}{2}\varDelta {c}_{\min }} \) ensures that atoms are only matched if their distance is less than half the minimum interatomic distance. Note that unlike accuracy, where the exact atom positions used to simulate the ground truth would replace the detected ground truth centers c _{ i }, the fidelity does not measure the error in the actual positions of the atomic columns. Instead, it measures how much the atomic column detection is influenced by the addition of shot noise and, more importantly, by the reconstruction. Looking at the corresponding values given in Table 7 for the examples discussed before, we see the same effect that for the very noisy input images (6 to 12 peak electrons/pixel) the fidelity is very low. Again, the local blockmatching BM3D estimates show a significantly increased fidelity compared to the noisy input images. Furthermore, except for the moderate dose gallium nitride (3.85 C/cm ^{2}) and silicon dislocation (2.8 to 28.47 C/cm ^{2}) images, the fidelity is also enhanced by a factor of two by using our proposed (uniform) periodic block matching instead of the local blockmatching strategy. For the single shot STEM images with an incident electron dose above 1.57 C/cm ^{2}, we achieve subpicometer fidelity by our proposed BM3D filter with uniform adaptive periodic block matching. Note that this subpicometer fidelity is also observed on the aperiodic silicon lattices with a dislocation, indicating that the adaptiveness of the proposed periodic search strategy is able to cope with localized irregularities in the crystal structure. Overall, this result shows that the denoising with our proposed method does not introduce artifacts exceeding the subpicometer level. Again, it should not be taken as evidence of subpicometer precision or accuracy in locating the atom columns in the reconstructed images, since it does not account for the STEM instabilities built in to the ground truth images. Removing instabilities still requires an approach like nonrigid registration of a series of images [2].
A comparison with linear filters
In practice, simple linear filters, such as the median or lowpass Wiener filter, are still commonly used for denoising in electron microscopy. In the following, we show that such filters provide a substantially worse denoising performance than the proposed modified BM3D method. In particular, we demonstrate that the application of such linear filters does not enable a proper analysis of extremely noisy images (less than 10 peak electron counts per pixel). We used the MATLAB functions medfilt2 for median filtering and wiener2 for lowpass Wiener filtering to produce the results presented here. In both cases, default parameters were used, which results in a windows size of 3×3 pixels and, in case of the Wiener filter, the noise power being automatically estimated.
Figure 7 shows denoised version of the gallium nitride and perfect silicon lattices from Figure 3 after application of a median filter (left column) and a lowpass Wiener filter (right column). A comparison with the corresponding estimates provided by our proposed method shown in Figure 5B,D reveals that the visual quality after denoising is much better when the modified BM3D method is used instead of either the median or lowpass Wiener filter.
A quantitative analysis based on the measures described above is shown in Table 8. The best results among the linear filters were achieved in the case of the noisy gallium nitride lattices processed with a median filter. Here, all atoms were correctly detected, i.e., the detection fraction is one and the misdetection fraction is zero. However, note that the recorded precisions in these two cases are worse by a factor of about two compared to those achieved with the proposed modified BM3D. In the other cases, i.e., where the median filter was applied to the silicon lattices and in all cases where the lowpass Wiener filter was used, the results indicate that atom detection is not feasible in the denoised images at all, which is either due to the detection fraction ρ _{+} being significantly less than 100% or the misdetection fraction ρ _{−} being significantly larger than 0% or both. The proposed BM3D filter achieved detection fractions of 100% on all noisy perfect crystal images except for the lowest SNR silicon lattice and misdetection fractions of 0% in all periodic crystal cases. In the case where the detection fraction after application of the modified BM3D filter is just 98%, the median filter yields a similar result with 96%. However, the misdetection fraction after application of the median filter is 81%. The lowpass Wiener filter yields worse results, with a detection fraction of 4% and a misdetection fraction of 50%. Overall, the results indicate that the proposed modified BM3D method is far superior to simple linear filters such as the median or Wiener filter in terms of both atom detectability and precision of atom positions.
A remark on further quantitative measures
We are aware that other quantities like the intensity and shape of atomic columns are also important for materials science. In future work, we plan to conduct a survey in this direction, especially treating the question whether the nonlocal averaging procedure will be able to retain different intensities in atomic columns of the same type, thus enabling the determination of the number of atoms in each column within the denoised image.
Conclusions
We proposed key modifications of the blockmatching and 3Dfiltering algorithm, which were aimed at enhancing the filter when applied to atomicresolution electron micrographs of periodic crystals. We have shown that, through the proposed modifications, the denoising performance is significantly improved compared to the original BM3D on all tested images. It also substantially outperforms common linear filters such as medianfiltering and lowpass Wiener filtering. The major advances are the adoption of a Fourierbased periodic similarity search [20] within the nonlocal means setting to the BM3D algorithm, as well as the treatment of an issue regarding spatial block concentration, which only occurs in the new BM3D setting. Furthermore, we showed that the proposed filter with its uniform adaptive periodic block matching, specifically tailored to perfect crystal structures, is able to significantly enhance both visually and quantitatively the image quality of lowdose electron micrographs. Quantitative measures of interest to the material science community, namely atomic column detectability and position precision, are significantly improved by application of the new denoising algorithm, without the introduction of artifacts such as falsepositive identification of atomic columns or shifts in the atomic column image positions beyond the subpicometer level.
The proposed algorithm for steering the spatial distribution of block estimates towards global uniformity achieves a significant improvement over the periodic blockmatching BM3D for certain images. However, we also observed cases where the results are slightly worse compared to the periodic block matching without this addition. Note that our uniform adaptive periodic blockmatching BM3D is still significantly better than the original BM3D in all cases. Nevertheless, we plan to further investigate this phenomenon. The goal is to find a strategy that will at least sustain the quality of the estimate compared to plain periodic block matching while improving it in the majority of the cases.
According to the results we presented for the case of a silicon dislocation, the adaptiveness of the periodic block matching copes fairly well with localized irregularities in the crystal structure. Nevertheless, the proposed periodic block matching is generally limited to the assumption of a perfectly periodic crystal, which is usually of less interest to material scientists than crystals with (possibly multiple) defects or changes in the lattice orientation. Thus, we plan to improve and properly extend the blockmatching strategy to these more complex geometries.
While the presented methods can be easily adopted to a series of images and thus might directly enable the registration of a series of lowdose electron micrographs, we expect a direct coupling of denoising and registration to be superior. Developing such a combined algorithm and analyzing its performance on series of low dose frames will be a main goal of our future work.
Abbreviations
 STEM:

scanning transmission electron microscopy
 BM3D:

blockmatching and 3Dfiltering algorithm
 PSNR:

peak signaltonoise ratio
 SNR:

signaltonoise ratio
 NLM:

nonlocal means
 AGWN:

additive Gaussian white noise
 HRTEM:

highresolution transmission electron microscopy
 VST:

variancestabilizing transform
References
 1
Batson, PE, Dellby, N, Krivanek OL: Subångstrom resolution using aberration corrected electron optics. Nature. 418(6898), 617–620 (2002).
 2
Berkels, B, Binev, P, Blom, DA, Dahmen, W, Sharpley, R, Vogt, T: Optimized imaging using nonrigid registration. Ultramicroscopy. 138, 46–56 (2014).
 3
Ortalan, V, Uzan, A, Gates, BC, Browning, ND: Direct imaging of single metal atoms and clusters in the pores of dealuminated HY zeolite. Nat. Nanotechnol. 5, 506–510 (2010).
 4
Houben, L, Thust, A, Urban, K: Atomicprecision determination of the reconstruction of a 90 degree tilt boundary in YBa _{2} Cu _{3} O _{7−δ } by aberration corrected HRTEM. Ultramicroscopy. 106(3), 200–214 (2006).
 5
Jia, CL, Urban, K, Alexe, M, Hesse, D, Vrejoiu, I: Direct observation of continuous electric dipole rotation in fluxclosure domains in ferroelectric Pb(Zr,Ti)O _{3}. Science. 331(6023), 1420–1423 (2011).
 6
Schmid, H, Okunishi, E, Oikawa, T, Mader, W: Structural and elemental analysis of iron and indium doped zinc oxide by spectroscopic imaging in cscorrected. STEM Micron. 43(1), 49–56 (2012).
 7
Kimoto, K, Asaka, T, Yu, X, Nagai, T, Matsui, Y, Ishizuka, K: Local crystal structure analysis with several picometer precision using scanning transmission electron microscopy. Ultramicroscopy. 110(7), 778–782 (2010).
 8
Yankovich, AB, Berkels, B, Dahmen, W, Binev, P, Sanchez, SI, Bradley, SA, Li, A, Szlufarska, I, Voyles, PM: Picometreprecision analysis of scanning transmission electron microscopy images of platinum nanocatalysts. Nat. Commun (2014). doi:10.1038/ncomms5155.
 9
Nellist, PD, Pennycook, SJ: Accurate structure determination from image reconstruction in adf stem. J. Microsc. 190(12), 159–170 (1998).
 10
Rosenauer, A, Mehrtensx, T, Müller, K, Gries, K, Schowalter, M, Venkata Satyam, P, Bley, S, Tessarek, C, Hommel, D, Sebald, K, Seyfried, M, Gutowski, J, Avramescu, A, Engl, K, Lutgen, S: Composition mapping in ingan by scanning transmission electron microscopy. Ultramicroscopy. 111(8), 1316–1327 (2011).
 11
Kim, YM, He, J, Biegalski, MD, Ambaye, H, Lauter, V, Christen, HM, Pantelides, ST, Pennycook, SJ, Kalinin, SV, Borisevich, AY: Probing oxygen vacancy concentration and homogeneity in solidoxide fuelcell cathode materials on the subunitcell level. Nat. Mater. 11(10), 888–894 (2012).
 12
Morgan, DG, Ramasse, QM, Browning, ND: Application of twodimensional crystallography and image processing to atomic resolution zcontrast images. J. Electron Microsc. 58(3), 223–244 (2009).
 13
Unwin, PNT, Henderson, R: Molecular structure determination by electron microscopy of unstained crystalline specimens. J. Mol. Biol. 94(3), 425–440 (1975).
 14
Klug, A: Image analysis and reconstruction in the electron microscopy of biological macromolecules. Chemica Scripta. 14, 1978–1979.
 15
Hovmöller, S, Sjögren, A, Farrants, G, Sundberg, M, Marinder, BO: Accurate atomic positions from electron microscopy. Nature. 311, 238–241 (1984).
 16
Hovmöller, S: Crisp: crystallographic image processing on a personal computer. Ultramicroscopy. 41(1), 121–135 (1992).
 17
Buades, A, Coll, B, Morel, JM: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005).
 18
Binev, P, BlancoSilva, F, Blom, D, Dahmen, W, Lamby, P, Sharpley, R, Vogt, T: Modeling Nanoscale Imaging in Electron Microscopy. Nanostructure Science and Technology. In: Vogt, T, Dahmen, W, Binev, P (eds.), pp. 127–145. Springer, New York City (2012).
 19
Wei, DY, Yin, CC: An optimized locally adaptive nonlocal means denoising filter for cryoelectron microscopy data. J. Struct. Biol. 172(3), 211–218 (2010).
 20
Mevenkamp, N, Yankovich, AB, Voyles, PM, Berkels, B: Nonlocal means for scanning transmission electron microscopy images and poisson noise based on adaptive periodic similarity search and patch regularization. In: Bender, J, Kuijper, A, von Landesberger, T, Theisel, H, Urban, P (eds.)VMV 2014: Vision, Modeling & Visualization, pp. 63–70. Eurographics Association, Darmstadt, Germany (2014).
 21
Bergmann, R: The fast Fourier transform and fast wavelet transform for patterns on the torus. Appl. Comput. Harmonic Anal. 35(1), 39–51 (2013).
 22
Coupé, P, Yger, P, Prima, S, Hellier, P, Kervrann, C, Barillot, C: An Optimized Blockwise Nonlocal Means Denoising Filter for 3D Magnetic Resonance Images. IEEE Trans. Med. Imaging. 27(4), 425–441 (2008).
 23
Salmon, J, Harmany, Z, Deledalle, CA, Willett, R: Poisson noise reduction with nonlocal PCA. J. Math. Imaging Vis. 48(2), 279–294 (2013).
 24
Dabov, K, Foi, A, Katkovnik, V, Egiazarian, K: Image Denoising by sparse 3D transformdomain collaborative filtering. Image Process. IEEE Trans. 16(8), 2080–2095 (2007).
 25
Anscombe, FJ: The transformation of poisson, binomial and negativebinomial data. Biometrika Trust. 35, 246–254 (1948).
 26
Mäkitalo, M, Foi, A: Optimal inversion of the anscombe transformation in lowcount poisson image denoising. Image Process. IEEE Trans. 20(1), 99–109 (2011).
 27
Deledalle, CA, Tupin, F, Denis, L: Poisson NL means: Unsupervised non local means for Poisson noise. In: Paper presented at the 17th Internal Conference on Image Processing, pp. 801–804. IEEE, Hong Kong (26–29 September 2010).
 28
Dabov, K, Foi, A, Katkovnik, V, Egiazarian, K: Image denoising with blockmatching and 3D filtering. Proc. SPIE. 6064, 606414–60641412 (2006).
 29
Maggioni, M, SánchezMonge, E, Foi, A, Danielyan, A, Dabov, K, Katkovnik, V, Egiazarian, K: Image and Video Denoising by Sparse 3D Transformdomain Collaborative Filtering: Blockmatching and 3D Filtering (BM3D) Algorithm and Its Extensions. Tampere University of Technology, http://www.cs.tut.fi/~foi/GCFBM3D/ Accessed date November, 10th 2014.
 30
Mäkitalo, M, Foi, A: Optimal Inversion of the Anscombe and Generalized Anscombe Variancestabilizing Transformations. Tampere University of Technology. http://www.cs.tut.fi/~foi/invansc/ Accessed date November, 10th 2014.
 31
Martin, DC, Chen, J, Yang, J, Drummy, LF, Kübel, C: High resolution electron microscopy of ordered polymers and organic molecular crystals: Recent developments and future possibilities. J. Polymer Sci. Part B: Polymer Phys. 43(14), 1749–1778 (2005).
 32
Kirkland, EJ: Advanced Computing in Electron Microscopy. 2nd edn. Springer, New York City (2010).
 33
Van Aert, S, Den Dekker, AJ, Van Den Bos, A, Van Dyck, D, Chen, JH: Maximum likelihood estimation of structure parameters from high resolution electron microscopy images. Part II: a practical example. Ultramicroscopy. 104(2), 107–125 (2005).
Acknowledgements
Electron microscopy simulations and other work by ABY and PMV were supported by the US Department of Energy, Office of Basic Energy Sciences, Division of Materials Sciences and Engineering under Award #DEFG0208ER46547. PB and WD acknowledge funding from the National Science Foundation under Grant No. DMS1222390. The authors at RWTH Aachen were funded in part by the Excellence Initiative of the German Federal and State Governments.
Author information
Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
NM and BB developed the denoising algorithm. WD and PB assisted in the development of the denoising algorithm. ABY and PMV simulated the STEM images. PMV and BB designed the analysis procedures. NM applied the denoising algorithm and performed the analyses. NM and WD drafted the manuscript, and all authors discussed and revised it. All authors read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Mevenkamp, N., Binev, P., Dahmen, W. et al. Poisson noise removal from highresolution STEM images based on periodic block matching. Adv Struct Chem Imag 1, 3 (2015). https://doi.org/10.1186/s4067901500048
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s4067901500048
Keywords
 Image denoising
 Poisson noise
 Nonlocal means
 Block matching
 3D transform shrinkage
 Periodic search
 STEM
 Precision