# Physically motivated global alignment method for electron tomography

- Toby Sanders
^{1}Email author, - Micah Prange
^{2}, - Cem Akatay
^{3}and - Peter Binev
^{1}

**1**:4

**DOI: **10.1186/s40679-015-0005-7

© Sanders et al.; licensee Springer. 2015

**Received: **25 November 2014

**Accepted: **13 February 2015

**Published: **8 April 2015

## Abstract

Electron tomography is widely used for nanoscale determination of 3-D structures in many areas of science. Determining the 3-D structure of a sample from electron tomography involves three major steps: acquisition of sequence of 2-D projection images of the sample with the electron microscope, alignment of the images to a common coordinate system, and 3-D reconstruction and segmentation of the sample from the aligned image data. The resolution of the 3-D reconstruction is directly influenced by the accuracy of the alignment, and therefore, it is crucial to have a robust and dependable alignment method. In this paper, we develop a new alignment method which avoids the use of markers and instead traces the computed paths of many identifiable ‘local’ center-of-mass points as the sample is rotated. Compared with traditional correlation schemes, the alignment method presented here is resistant to cumulative error observed from correlation techniques, has very rigorous mathematical justification, and is very robust since many points and paths are used, all of which inevitably improves the quality of the reconstruction and confidence in the scientific results.

### Keywords

Electron tomography Image alignment## Background

Ideally, between two consecutive projections acquired at nearby tilts of the sample, one would observe only a small rotation of the projected image. However, due to unavoidable mechanical limitations, significant translation shifts are present. Therefore, the projections must be aligned into a common coordinate system to be properly interpreted. Once the projections are aligned, they can then be merged to approximate the 3-D structure of the sample. The alignment is a crucial part of the process, for the resolution of the reconstructed 3-D structures are limited to the accuracy in the alignment. In this paper, we demonstrate a new mathematically justified method for the alignment based on the apparent motion of the center of mass of many 2-D cross-sections of the sample.

Over the years, many traditional alignment techniques have been developed by the biological sciences [6]. The most commonly practiced are correlation techniques, feature tracking, and fiducial marker tracking. Correlation techniques are performed by selecting one of the projections as a reference image and aligning each pair neighboring images by selecting the cross-correlation peak between the images for the shift [7]. This method has been proven useful but can yield poor results, as small cumulative errors may result in a serious drift of the sample [8]. As we will show, cross-correlation will not recover the correct alignment even for noise-free data subjected to random shifts. The current work finds a solution without this deficiency.

Fiducial marker tracking is done by decorating the sample with small high-density particles that create high contrast in the projection images [9-12]. Individual markers are then identified in all projections. The alignment is determined based on tracking of the path of each marker through the projections. This method can be very accurate but requires a lot of manual interaction to properly locate and center the markers. The main drawback of marker tracking is that the markers will be present in the reconstruction and must be removed for accurate characterization of the sample. Since the markers are of such high density, the reconstruction of the markers will inevitably mix with the reconstruction, making the task of removal nontrivial and possibly inaccurate.

Feature tracking uses regions of high contrast or intensity as fiducial markers [13,14]. It requires the identification of suitable regions of high contrast that remain visible throughout the tilt series.

Others have begun to perform alignment techniques based on a refinement approach [6]. After a coarse alignment from cross-correlation, one proceeds in computing an initial 3-D reconstruction. This 3-D reconstruction is then reprojected and compared with the original projections. A new alignment arises from aligning the reprojected reconstruction with the original projections, and this process is iterated until convergence is met. In our experience with this method, the reconstruction always satisfies the projections, even if they’re misaligned, so that insignificant refinement occurs from updating.

Most recently, Scott et al. [15] introduced a technique based on the observation that as the sample is tilted about a fixed axis, the center of mass of the sample will spin in a circle, and if the center of mass is on the tilt axis, then it remains fixed. In this way, it was suggested to shift each projection so that the center of mass in each projection is fixed on a point and taking the line through this point parallel to the axis of rotation as the tilt axis. We believe this is not always applicable and can yield poor results in many settings. First, it requires a tilt series in which the total projected volume is fixed for each projection. However, in most practical settings, some mass will move in and out of the projection range as the sample is tilted, which will then significantly affect the location of the center of mass within the projection along both axes of the projection images. This transition of mass must be accounted for, as this transition will be along the edges of the projections, far from the center, and will thus weigh heavily on the calculated center of mass. Figure 1 demonstrates this transition of mass, with the small ball located on the left edge of the object that has only been projected at certain angles. An additional drawback is that using only the single center of mass point in each projection removes the use of any local structure of the projections as criteria for alignment.

In this paper, we give an alignment method that makes more detailed use of the path of the projected center of mass along many cross-sections of the object, perpendicular to the axis of rotation. In an ideal experiment, points on the sample move in circular trajectories. We define a viable path as the projection of such a circular orbit. By simple calculation, we derive an equation which describes all such viable paths of the projected centers of mass, as opposed to the one trivial path of a single point. From here, we show how one can determine a shift for each projection so that the center of mass of all cross-sections perpendicular to the axis of rotation nearly follows a viable path. In this way, since all cross-sections are considered in our alignment method, we will be able to avoid problems involved with error in the calculated centers of mass due to transition of volume in and out of the projections, and we maintain local analysis of the projections as means for the alignment. Additionally, our model aligns the projections based on the rotation about a chosen axis, so that manual interaction for determining the positioning of the tilt axis is avoided. In general, our method can be considered more statistically accurate, and we will show that it provides very dependable alignment and definitively improves the resolution of the reconstruction.

## Methods

### Notation

*f*(

*x*,

*y*,

*z*)=

*f*(

*x*,(

*y*

*z*)), with (

*y*

*z*) a 2-D row vector. The data generated are the projections of

*f*in the

*z*-axis, about rotations around the

*x*-axis. A rotation of

*f*through

*θ*about the

*x*-axis can be written as:

*θ*is then defined as:

*x*=

*x*

_{0},

*P*

_{ θ }(

*f*)(

*x*

_{0},

*y*) only contains information from

*f*(

*x*

_{0},

*y*,

*z*), and therefore, many of the alignment and reconstruction processes can be considered as 2-D rather than 3-D. Therefore, for convenience, we will sometimes denote:

*x*

_{ θ },

*y*

_{ θ }) are the shifts to be determined for the alignment. Similarly, we will denote:

*x*

_{ θ }is not included. We do not include it, for determining the shifts

*x*

_{ θ }is a much more trivial task, so that most of our work here focuses on determining

*y*

_{ θ }after the

*x*-axis alignment is completed.

*x*by \( {M}_x=\underset{{\mathbb{R}}^2}{\int }{f}_x\left(y,z\right)\kern1em dy\kern1em dz \). Then, the coordinates for the center of mass of a cross-section are denoted as:

*f*by:

*L*

_{ p }norm (denoted by ∥·∥

_{ p }) of a function, say

*g*, defined over

*ℝ*

^{ n }to be:

*x*∈

*ℝ*

^{ n }, we take the

*ℓ*

_{ p }norm (denoted ∥·∥

_{ p }) to be:

### Theoretical model

*k*, can be from 50 to 200, with maximum tilts of ± 70°. The domain is of course limited, but for theoretical purposes, we will assume that the domain for

*y*is all of

*ℝ*. The problem is then to approximate the set of shifts \( \left({x}_{\theta_i},{y}_{\theta_i}\right) \) for alignment, so that \( {\left\{{\overset{\sim }{P}}_{\theta_i}(f)\left(x,y\right)\right\}}_{i=1}^k \) correspond to the aligned projections \( {\left\{{P}_{\theta_i}(f)\left(x,y\right)\right\}}_{i=1}^k \). Determining the shifts for the

*x*-axis is much simpler, since the

*x*-axis is the axis of rotation. We simply observe that the total mass in each cross-section should remain fixed, so that:

*x*-axis alignment is given in Figure 2.

One could also perform a similar ‘local’ method, by comparing the consecutive projections to each other instead of the average. This approach is subject to cumulative error in the alignment similar to cross-correlation; therefore, we avoid this approach.

*y*-axis, we again want to make use of physical properties. It has been noted, as

*f*

_{ x }(

*y*,

*z*) is rotated about the origin, the center of mass \( \left({c}_x^y,{c}_x^z\right) \) will spin in a circle around the origin. It is not immediately clear, however, how this property can be observed within the projections and used for alignment. Computing the center of mass of a projected slice, we obtain:

*x*, and then, the center of mass is computed for the total sum of the cross-sections, that is:

where *c*
^{
y
} and *c*
^{
z
} here denote the center-of-mass coordinates along the *y*- and *z*-axes, respectively, independent of *x*, and *M* denotes the total mass of *f*. Therefore, it is suggested to shift each projection so that \( {t}^{\theta_i}=0 \) for all *i*, so that *c*
^{
y
}=*c*
^{
z
}=0. While this approach is theoretically sound in an ideal setting, summing over *x* immediately removes any consideration of local behavior of the projections of *f*. As we will show, in many settings, this simplification can be a major drawback.

*Θ*and

*t*

_{ x }. Now from Equation 3, if our alignment is good, then for each cross-section

*x*, there should exist some

*c*

_{ x }so that

*Θ*

*c*

_{ x }≈

*t*

_{ x }. Therefore, in order to yield a good alignment, we would like to determine:

*c*

_{ x }satisfying:

*x*

_{ j }, for

*j*=1,2,…

*n*. Then, we would like solve the minimization problem:

*c*

_{ x }directly. Given

*Θ*and

*t*

_{ x }, the least square solution \( {c}_x^{\ast } \), to \( \parallel \varTheta {c}_x-\left({t}_x+{y}_{\varTheta}\right){\parallel}_2^2\kern0.3em \) :

*Θ*

^{+}denotes the pseudo-inverse of

*Θ*, given by

*Θ*

^{+}=(

*Θ*

^{ T }

*Θ*)

^{−1}

*Θ*. It should be noted that

*Θ*

^{ T }

*Θ*is a 2×2 matrix with entries:

which is clearly invertible and without any notable computational cost.

### Practical implementation

*y*for \( {\overset{\sim }{P}}_{\theta_i}\left({f}_x\right)(y) \) is finite, say [−

*m*,

*m*]. As before with

*x*, for all practical purposes, we will now additionally consider the

*y*-axis to be discrete, and for each projection \( {P}_{\theta_i}(f)\left(x,y\right) \), the domain is given as:

*y*symmetrically for convenience in the center-of-mass computations so that the center of the projections is along the modeled axis of rotation at

*y*=0. Computing \( {t}_x^{\theta_i} \) now becomes:

The first issue is that *M*
_{
x
} may vary through the tilt series for each cross-section; in particular, since the domain for *y* is limited, there may be some observable mass moving in and out of the field of view after rotation and projection, as we demonstrated in Figure 1. This is again why it’s important that we choose the alignment to be considered over many projected cross-sections.

- (i)
\( 0\le {\omega}_{\theta_i}\left(x,y\right)\le 1; \)

- (ii)
\( M=\sum_{x=1}^n\sum_{y=-m}^m{P}_{\theta_i}(f)\left(x,y\right){\omega}_{\theta_i}\left(x,y\right) \), for

*i*=1,2,…,*k*; - (iii)
\( {\omega}_{\theta_i}\left(x,y\right)\le {\omega}_{\theta_i}\left(x,y+1\right)\kern1em \mathrm{if}\kern1em y<0, \)

\( {\omega}_{\theta_i}\left(x,y\right)\ge {\omega}_{\theta_i}\left(x,y+1\right)\kern1em \mathrm{if}\kern1em y\ge 0; \)

- (iiii)
\( {\omega}_{\theta_i}\left(x,y\right)={\omega}_{\theta_i}\left(x+1,y\right) \), for

*x*=1,2,…,*n*−1.

The first property simply emphasizes that multiplication of \( {\overset{\sim }{P}}_{\theta_i}(f) \) by \( {\omega}_{\theta_i} \) reweighs the projection values in order to dampen the introduction of new mass in to the frames. The second property then tells us that this dampening of the values of \( {P}_{\theta_i}(f) \) by multiplication of \( {\omega}_{\theta_i} \) yields the same total mass in each projection. Finally, properties (iii) and (iiii) describe how this dampening should be done. Property (iii) says that the window function decreases as the function moves away from the *y*-axis. This is because new mass would be introduced along the edge of the plane of view, so that we dampen these values more significantly. Property (iiii) is an additional property to help us better characterize \( {\omega}_{\theta_i} \) in a simple manner and simply says that we place the same weight for each cross-section *x*. One could remove property (iiii) and change property (ii) so that instead the mass *M*
_{
x
} is fixed for each cross-section of each projection. This could potentially cause bias in the alignment of the cross-sections, especially ones with considerable noise, and it would require much greater computational time to determine a window for each cross-section of each projection.

*j*=1,2,…,

*n*as:

*T*⊂{1,2,…,

*n*}. This subset is chosen so that the selected cross-sections have a significant quantity of mass in each projection so that introduction of new mass along the edges has considerably less effect on the center of mass of this projected cross-section area. In addition, we only choose those in which the observable total mass within that cross-section varies little throughout all projections, to again avoid the cross-sections with large transition of mass.

### Reconstruction method

*f*by \( g={\left\{{g}_{x,y,z}\right\}}_{x,y,z=1}^N \), where for simplicity we now let our discrete 3-D domain be:

*g*agrees with the experimental projections \( {P}_{\theta_i}(f) \), for

*i*=1,2,…,

*k*. In particular, reconstruction techniques typically minimize the distance between the projections of

*g*and the experimental projections, sometimes called the projection error. This projection error can be expressed as:

*g*∥

_{ TV }for noise reduction, so that our method solves:

## Results and discussion

We will give the results for experimental and simulation data. We compare the reconstructions from alignment using cross-correlation and our center-of-mass technique, while also demonstrating the advantage of using many slices for the center-of-mass alignment, as opposed to just one center-of-mass calculation.

### Experimental results

For the experimental data, we have an alumina particle sitting on a holey carbon grid. The sample was prepared by grinding the alumina spheres into powder. A suspension of the powder is prepared in ethanol and sonicated for 5 min. The suspension was then added drop-wise over the lacey carbon film supported on 200 mesh Cu TEM grids (Structure Probe, Inc., West Chester, PA, USA) and dried at room temperature. The sample is analyzed using the FEI Titan 80-300 Scanning Transmission Electron Microscope equipped with a spherical-aberration probe-corrector (CEOS GmbH, Heidelberg, Germany) operating at 200 kV. The images were collected using the high-angle annular detector with the camera length of 195 mm and at 80,000 *X* magnification. The acquisition time was set to 15 s over an image area of 1024 *X* 1024 pixels resulting in a pixel size of 0.2411 nm. The tilt series is collected using linear tilt scheme continuously from -70° to +70°with tilt increments of 2°. Dynamic STEM focus function is used to compensate for change in focus across the image. The projection of the sample at 30°degrees is shown in Figure 2, and the aligned projections are shown in a video in Additional file 1.

*t*

_{ x }, for two cross-sections. Plotted together with

*t*

_{ x }are least squares solutions of the center of mass, \( \left({c}_x^y,{c}_x^z\right) \), based Equation 3 given the computed

*t*

_{ x }. It is evident that our method finds a nearly viable path for the motion of the center of mass, as we set out to do. On the other hand, the alignment from cross-correlation clearly fails to do so, resulting in low-resolution reconstructions.

*y*-axis is shown for the two methods. The plot in Figure 6g shows the only consideration for the originally proposed center-of-mass alignment, as the center of mass in the projections along the

*y*-axis is shifted to the tilt axis. With pixelation of the images, there is still a small negligible distance (less than half a pixel) between the center of mass in each projection and the tilt axis. The location of this center of mass resulting from our approach is shown in Figure 6h and does not necessarily follow a viable path, because we choose a different minimization and allow our approach to avoid problematic cross-sections. In Figure 6i,j, the path of the projected center of mass is shown for a single cross-section for the two alignment methods, where, for this cross-section, our methods demonstrate a viable path and the approach based on the single global center of mass does not. Inevitably, our method produces better reconstruction results, demonstrating that a more sophisticated alignment approach should be taken for dependable results as we have done, taking into account not one single data point but rather all cross-sections as unique data points. The resulting segmentation of the alumina particle is shown in 3-D in a video in the Additional file 2.

### Simulation results

## Conclusions

Our method has a sound physical basis: the movement of the center of mass in each cross-section. By selecting shifts for individual tilt-series images that globally lead to physically plausible motions for the centers of mass of many cross-sections, our method effectively utilizes the assumption that the sample object is rigid to improve the alignment and the resolution of the final reconstruction. We have shown that conventional alignment procedures, which shift the global center of mass to the origin, may not produce physically plausible motions in other cross-sections. We have generalized these methods in a computationally feasible manner that can be easily be incorporated into electron tomography workflows. We have demonstrated the significance of such consistency between cross-sections and the effectiveness of the presented method by improving the resolution of 3-D reconstructions of simulated and actual data.

## Declarations

### Acknowledgements

The authors would like to thank Dr. Ilke Arslan for her helpful discussions. This research was supported in part by NSF grant DMS 1222390. It was also funded by the Laboratory Directed Research and Development program at Pacific Northwest National Laboratory, under contract DE-AC05-76RL01830.

## Authors’ Affiliations

## References

- Lucic, V, Forster, F, Baumeister, W: Structural studies by electron tomography: from cells to molecules. Ann. Rev. Biochem. 74, 833–865 (2005).View ArticleGoogle Scholar
- Midgley, P, Weyland, M: 3D electron microscopy in the physical sciences: the development of Z-contrast and EFTEM tomography. Ultramicroscopy. 96, 413–431 (2003). International Workshop on Strategies and Advances in Atomic-Level Spectroscopy and Analysis (SALSA), GUADELOUPE, GUADELOUPE, MAY 05-09, 2002.View ArticleGoogle Scholar
- Arslan, I, Yates, T, Browning, N, Midgley, P: Embedded nanostructures revealed in three dimensions. Science. 309(5744), 2195–2198 (2005).View ArticleGoogle Scholar
- Crowther, RA, Amos, LA, Finch, JT, De Rosier, DJ, Klug, A: Three dimensional reconstructions of spherical viruses by Fourier synthesis from electron micrographs. Nature. 226(5244), 421–425 (1970).View ArticleGoogle Scholar
- Arslan, I, Tong, JR, Midgley, PA: Reducing the missing wedge: high-resolution dual axis tomography of inorganic materials. Ultramicroscopy. 106(11–12), 994–1000 (2006). Proceedings of the International Workshop on Enhanced Data Generated by Electrons Proceedings of the International Workshop on Enhanced Data Generated by Electrons.View ArticleGoogle Scholar
- Houben, L, Sadan, MB: Refinement procedure for the image alignment in high-resolution electron tomography. Ultramicroscopy. 111(9–10), 1512–1520 (2011).View ArticleGoogle Scholar
- Guckenberger, R: Determination of a common origin in the micrographs of tilt series in three-dimensional electron microscopy. Ultramicroscopy. 9(1–2), 167–173 (1982).View ArticleGoogle Scholar
- Saxton, W, Baumeister, W, Hahn, M: Three-dimensional reconstruction of imperfect two-dimensional crystals. Ultramicroscopy. 13(1–2), 57–70 (1984).View ArticleGoogle Scholar
- Brandt, S, Heikkonen, J, Engelhardt, P: Multiphase method for automatic alignment of transmission electron microscope images using markers. J. Struct. Biol. 133(1), 10–22 (2001).View ArticleGoogle Scholar
- Fung, JC, Liu, W, de Ruijter, W, Chen, H, Abbey, CK, Sedat, JW, Agard, DA: Toward fully automated high-resolution electron tomography. J. Struct. Biol. 116(1), 181–189 (1996).View ArticleGoogle Scholar
- Masich, S, Östberg, T, Norlén, L, Shupliakov, O, Daneholt, B: A procedure to deposit fiducial markers on vitreous cryo-sections for cellular tomography. J. Struct. Biol. 156(3), 461–468 (2006).View ArticleGoogle Scholar
- Ress, D, Harlow, M, Schwarz, M, Marshall, R, McMahan, U: Automatic acquisition of fiducial markers and alignment of images in tilt series for electron tomography. J. Electron Microsc. 48(3), 277–287 (1999). year=1999,View ArticleGoogle Scholar
- Brandt, S, Heikkonen, J, Engelhardt, P: Automatic alignment of transmission electron microscope tilt series without fiducial markers. J. Struct. Biol. 136(3), 201–213 (2001).View ArticleGoogle Scholar
- Sanchez Sorzano, CO, Messaoudi, C, Eibauer, M, Bilbao-Castro, JR, Hegerl, R, Nickell, S, Marco, S, Carazo, JM: Marker-free image registration of electron tomography tilt-series. BMC Bioinformatics. 10, 124 (2009). http://biocomp.cnb.csic.es/~coss/Articulos/Sorzano2009b.pdf.View ArticleGoogle Scholar
- Scott, MC, Chen, C-C, Mecklenburg, M, Zhu, C, Xu, R, Ercius, P, Dahmen, U, Regan, BC, Miao, J: Electron tomography at 2.4-angstrom resolution. Nature. 483(7390), 444–U91 (2012).View ArticleGoogle Scholar
- Li, C: Compressive sensing for 3D data processing tasksapplications, models, and algorithms. Dissertation, Rice University (2011).Google Scholar
- Leary, R, Saghi, Z, Midgley, PA, Holland, DJ: Compressed sensing electron tomography. Ultramicroscopy. 131, 70–91 (2013).View ArticleGoogle Scholar
- Goris, B, den Broek, WV, Batenburg, K, Mezerji, HH, Bals, S: Electron tomography based on a total variation minimization reconstruction technique. Ultramicroscopy. 113, 120–130 (2012).View ArticleGoogle Scholar
- Monsegue, N, Jin, X, Echigo, T, Wang, G, Murayama, M: Three-dimensional characterization of iron oxide (alpha-Fe2O3) nanoparticles: application of a compressed sensing inspired reconstruction algorithm to electron tomography. Microscopy Microanal. 18(6), 1362–1367 (2012).View ArticleGoogle Scholar
- Tomviz for tomographic visualization of 3D scientific data. http://www.tomviz.org (2014). 15 August 2014.

## Copyright

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.