Skip to main content

Automated analysis of evolving interfaces during in situ electron microscopy


In situ electron microscopy allows one to monitor dynamical processes at high spatial and temporal resolution. This produces large quantities of data, and hence automated image processing algorithms are needed to extract useful quantitative measures of the observed phenomena. In this work, we outline an image processing workflow for the analysis of evolving interfaces imaged during liquid cell electron microscopy. As examples, we show metal electrodeposition at electrode surfaces; beam-induced nanocrystal formation and dissolution; and beam-induced bubble nucleation, growth, and migration. These experiments are used to demonstrate a fully automated workflow for the extraction of, among other things, interface position, roughness, lateral wavelength, local normal velocity, and the projected area of the evolving phase as functions of time. The relevant algorithms have been implemented in Mathematica and are available online.


Detailed understanding of the evolution of interfaces is of both scientific and practical interest in many disciplines. Scientists wish to identify unifying principles of pattern formation in processes such as solidification and aggregation [1]. Engineers are interested in controlling the evolution of interfaces and their morphologies. For example, morphological instabilities, particularly dendrite formation, can cause catastrophic failure in rechargeable batteries or lower the quality of electroplated coatings, yet they may be useful in forming porous deposits such as porous electrodes and catalytic surfaces; real-time observations provide essential information on the process by which they form [26]. Nanocrystal growth is another example where morphology control enables tuning of mechanical, electrical, and optical properties [710]. Liquid cell electron microscopy [11, 12] can address such problems. It allows us to image, in real time and with nanoscale resolution, processes such as the evolution of the growth front during electrochemical deposition and etching [1319], nanocrystal formation and dissolution [9, 2023], and bubble nucleation, growth, and migration [11, 12, 1416, 20, 21, 24] to provide insights into the mechanisms controlling morphology and mass transport.

Making the best use of the information obtained during liquid cell experiments requires quantitative measurements from each frame in a video sequence as a phenomenon of interest takes place. In particular, tracking the position of interfaces between different materials is a common first step. For example, Woehl et al. [22] studied the nucleation and growth of silver nanocrystals utilizing image processing to obtain particle count and radius as functions of time and beam conditions, and Ievlev et al. [23] used interface detection algorithms to investigate platinum nanocrystal growth properties via local growth velocity. We have therefore developed a general use interface-tracking algorithm based on image analysis needs over several different liquid cell experiments. Here we demonstrate its use, extract several quantitative measures of dynamical processes, and outline the algorithm details for easy adaptation of our open source software [25] to other experimental systems. Our workflow is illustrated with experimental data from three liquid cell experiments: copper electrodeposition [18, 19], electron beam-induced formation of gold nanoprisms [9, 21], and bubble nucleation and growth [26]. For copper electrodeposition from an acidified copper solution [18, 19], we extract the growth front location as a function of time. The data are then used to calculate the projected area and mass of the deposit, the root mean square (RMS) roughness of the interface to identify the growth habit, the local normal velocity distribution (akin to current density) to provide the relationship between geometry and point-wise growth rate, and the wavelengths associated with interface morphology, as their control is important for structured surface manufacturing. For growth and dissolution of single-crystal Au nanoparticles mediated by the electron beam [9, 20, 21], the algorithm adjusts for particle translation and extracts the particle orientation to properly determine growth kinetics. Finally, for nanobubble nucleation, growth, and migration [24, 26], we extract bubble radius and position as functions of time. The automation of the process allows for analysis of datasets with large numbers of frames.

Image processing

Several processes were combined to enable unsupervised, non-parametric image processing, and we outline the individual steps below. By non-parametric, we mean that frame-specific parameters are automatically calculated without user input. The data used for illustration were obtained as described in the experimental methods section. The open source code has been well-documented to make it accessible to new Mathematica users. A video walkthrough of the code is available in the supplement (Additional file 1: Video S1) and on YouTube [27].

Raw data pre-preparation

To perform automated analysis of image sequences, the raw data must be in a consistent, easy to modify form. We assume that the raw data are in a video format that is readable by Mathematica since, by default, this is often the case. Non-video image sequences can be handled with simple modifications to the code. When needed, Virtualdub [28] (Windows Only) and ImageJ [29] are useful open source software packages for basic video editing and format conversion. Once the raw data are in an acceptable video file format, we select a reference frame and manually define the rotation, if any, the crop indices (crop area), and the background for subtraction. It is necessary to crop out any time stamp and experimental data overlays prior to the automated processes described below, as they will interfere with image thresholding. We illustrate pre-preparation of the raw data using a movie recorded during electrodeposition of copper [19, 30], Fig. 1a. In data such as this, which includes an electrode edge, rotation is particularly useful to ensure that the edge has the same orientation, i.e., horizontal, in all the images to simplify post processing. Figure 1a is a raw image with overlaid crop indices (red disks) and rotation angle. Figure 1b is the resulting, rotated, cropped image with the time stamp removed. From here on, we shall refer to images of the form in Fig. 1b as the sub-images.

Fig. 1
figure 1

Bright-field image of copper electrodeposition in the nanoaquarium operating in a three-terminal configuration with Pt electrodes controlled by a Gamry potentiostat. Aqueous solutions of 0.1 M CuSO4 + 0.18 M H2SO4 were prepared in doubly deionized water and were not deaerated before use. The interface morphology evolution was imaged during galvanostatic deposition with a total current of 400 nA. The transmitted beam current of 0.1 nA was spread over ~2-μm diameter resulting in a dose rate of 7.5 × 106 (Gy/s) or 2 (e2-s). The liquid thickness at the edge of the silicon nitride window is fixed to 200 nm by lithography. a Example raw data of Cu electrodeposition with overlaid crop indices (red disks) and the desired angle of rotation. b The resulting, cropped, rotated image. We refer to b as the sub-image. The lateral field of view is approximately 1850 nm in a and 1000 nm in b

Image preparation—noise filtering and background subtraction

To minimize damage caused by the electron beam, it is important to operate at the minimum dose rate required to form a usable image [31, 32]. This results in images with a relatively low signal-to-noise-ratio (SNR). Since the Rose criterion requires the SNR to be about 5 for an image to be detectable by the human eye [33, 34], it is useful to take precautions to reduce the apparent noise in each frame of the video sequence while preserving edges. We achieve this reduction through background subtraction and the application of a total variation filter (TV filter). We illustrate this technique using experimental data from beam-induced gold nanocrystal formation [9, 21].

De-noising is necessary when high accuracy object extraction is desired. Since we are interested in tracking interfaces, which are seen as edges or intensity discontinuities in images, we choose an image restoration algorithm that maintains the fidelity of the edges. The TV filter, pioneered by Rudin et al. [35, 36], poses noise removal as a constrained optimization problem that assumes the noise to be Gaussian and additive (superimposed on top of real object image, or “truth image”). Figure 2a, b, and c shows, respectively, a nanocrystal sub-image, a filtered image, and the difference between the two. Notice that to the eye, the filtered version looks almost unaltered, which is desirable. The smoothing provided by this filter mitigates undesired artifacts during subsequent processing steps.

Fig. 2
figure 2

Beam-induced Au crystal growth in the nanoaquarium with transmitted beam current of 7.8 nA spread over ~2 μm diameter, resulting in a dose rate of 5.9 × 108 (Gy/s) or 160 (e2-s). Stock solutions of chloroauric acid (Sigma-Aldrich) were made by dissolving the salt in deionized (DI) water at concentration of 20 mM. The liquid thickness within the imaged region is 50–100 nm. The left column shows de-noising: a a nanocrystal sub-image, b the same image after TV filter application, and c the difference between images a and b. The second column shows background subtraction: d the image used as the background model, e the frame of interest prior to background subtraction, and f the foreground image. The right column shows edge detection: g a thresholded foreground image (after TV filter), h the extracted edges from the foreground image in g, and i the extracted edges without image preparation (undesired)

Background subtraction is useful for increasing contrast of objects in scenes imaged in a fixed laboratory frame. Several algorithms have been developed to estimate the “background model” using temporal information [37]. Carefully designed experiments to capture reference images can provide a frame or an average of a few frames for the background model. In the case of the nanocrystal of Fig. 2, for the background model, we select a frame before the nucleation of the particle. Figure 2d, e, and f (middle column) shows, respectively, the background model, the frame of interest, and the foreground image. The image difference is obtained using pixel-by-pixel subtraction of the image intensities,

$$I_{\text{F}} \left( {x,y,t} \right) = I\left( {x,y,t} \right) - I_{0} \left( {x,y} \right),$$

where \(I\left( {x,y,t} \right)\) is the intensity of pixel \(\left( {x,y} \right)\) at time t, and \(I_{0}\) refers to the background model. The foreground image is then rescaled to have intensity levels between 0 and 1 as shown in Fig. 2f, where 0 represents black in a digital image.

Thresholding and edge detection

Once noise is filtered and contrast enhanced, edge detection can be performed. Edge detection works best on a binary image. Hence, thresholding is first performed. The standard Otsu’s method [38] is utilized to convert the grayscale image into one with only two values (additional gray levels can be included when needed). This process once again casts the image analysis as an optimization problem, where now we assume that pixels can be classified into two classes, the foreground and background, with maximized separability. That is, the pixels are assumed to have a bi-modal histogram and the algorithm tests all possible threshold values, where the chosen threshold value is the one where the sum of the variances of the two classes is minimal. Figure 2g shows a binarized foreground image that was TV filtered prior to thresholding.

Once the binary image is generated, one can find the edges of the object. The edges are a direct measure of the interface as projected from the image plane. A Canny style edge detector [39] is extremely robust for images of the type depicted in Fig. 2g. Edges are defined as local maxima in the first-order directional Gaussian gradient of the intensity function. Since the image has already been denoised and binarized, the edges are extracted with great fidelity as shown in Fig. 2h. Had the image not been pre-processed with the TV filter and background subtraction, many false positives would have been identified by the edge detection (Fig. 2i).

Translation correction and object centroids

Often, image sequences contain drift and/or migration of the imaged object. When examining the growth of the object, it is convenient to fix the location of its center of mass in all frames. For example, Additional file 2: Video S2 shows the growth of a gold crystal. During the video, due to drift adjustments of the beam location, the relative position of the crystal has changed. Here, we present a means for the translating a single object to fix the location of its center of mass in all the frames. As part of this process, we track the location of the object’s centroid as a function of time. The code can be extended to systems of multiple dynamic objects via particle tracking algorithms such as the methods developed by Crocker and Grier [40] that are available as open source code for several computation platforms. This software relates the positions of entities between frames to statistically identify the displacement of individual entities and establish their track (path in time). For simplicity, here, we will only extract the positions of particles subject to translation.

We consider the nanocrystal growth from Fig. 2 and the corresponding video sequence (Additional file 2: Video S2). Once the binary image is generated (Fig. 2g), the centroid of the foreground pixels is identified for each frame according to the formula

$$\left( {x_{c,i} ,y_{c,i} } \right)_{\Upomega } = \left( {\frac{1}{n}\sum\limits_{\Upomega } {x_{i} } ,\frac{1}{n}\sum\limits_{\Upomega } {y_{i} } } \right),$$

where Ω is the region of pixels identifying the object (white pixels in Fig. 2g), n is the number of pixels in Ω, the (x, y) pair denotes the pixel location, and i is the frame number. By shifting the location of the particle in each frame using the linear transformation

$$I_{\text{Mapped}} \left( {x,y,t} \right) = I\left( {x - x_{c} ,y - y_{c} ,t} \right),$$

we end up with the centers of mass in each frame located at {0, 0}.

Growth front tracking during electrodeposition

The first application of our image processing algorithm is to extract the growth front morphology during copper electrodeposition. Copper electrodeposition experiments were carried out with our custom-made liquid cell, the nanoaquarium [41], operating in a three-terminal configuration with Pt electrodes controlled by a Gamry potentiostat. The liquid cell was filled with an aqueous solution of 0.1 M CuSO4 + 0.18 M H2SO4, prepared in doubly deionized water and not deaerated before use. The interface morphology evolution was recorded using bright-field imaging conditions during galvanostatic (constant current) deposition. Data were recorded at video rate (30 frames per second) in a Hitachi H-9000 TEM at 300 kV. In these types of experiments, liquid cell electron microscopy provides the surface evolution in real time and with nanoscale resolution [11, 12, 14, 19] and the aim of the quantitative analysis is to obtain insight into the mechanisms leading to stable and unstable growth fronts under different conditions.

The algorithm described here robustly extracts quantitative measures of the growth front morphology. Figure 3a and supplemental Additional file 3: Video S3 show the data recorded at selected times during galvanostatic deposition under conditions (high current density) that resulted in layer growth with a diffusive instability. Once we have extracted the edge, we can post-process quantitative data to obtain various figures of merit describing the growth. The simplest metrics are the average layer height, the maximum height, and the minimum height as functions of time (Fig. 3b). We see an approximately linear advancement of the average height (after an initial transient), as expected from galvanostatic conditions where the amount of material deposited per second should be constant. We also see a transition in growth mode around Frame 12. Here, the minimum height becomes constant (no additional material is being deposited at the base of the valleys) and the maximum height grows at a higher rate to maintain the average growth speed mandated by the constant current process.

Fig. 3
figure 3

Bright-field image of Cu electrodeposition in the nanoaquarium using the same electrochemical configuration (400 nA current), electrolyte, beam conditions and liquid thickness as Fig. 1. a An extracted growth edge from copper electrodeposition. Selected frames from Video S3 are shown. b Maximum, minimum, and average growth height as functions of frame number. c RMS roughness of the growth height as a function of time. d Contours of the autocorrelation function of the surface roughness as a function of spatial shift and frame number. e A heat map of the normal growth speed

We can further describe the interface by its amplitude and wavelength. The RMS roughness,

$$w\left( {t,L} \right) = \sqrt {\frac{1}{L}\sum\limits_{i = 1}^{L} {\left[ {h\left( {i,t} \right) - \bar{h}\left( t \right)} \right]^{2} } } ,$$

is a parameter used in kinetic roughening theory [42, 43] to characterize surface morphology. In the above, L is the length of the selected domain, h (i, t) is the height at location i and time t, and \(\bar{h}\left( t \right)\) is the average height at time t. Figure 3c depicts \(w\left( {t, L} \right)\) as a function of t when L ~1 μm is equal to the size of the observation window.

The lateral wavelength of the interface can also be extracted as a function of time. In this particular case, the wavelengths observed are all small (because they must be on the order of the window size). We use the cross-correlation of the interface with itself

$$R\left( {t,u} \right) = \frac{{\sum\nolimits_{i = 1}^{L - u} {\left( {h\left( {i,t} \right) - \bar{h}\left( t \right)} \right)} \left( {h\left( {i + u,t} \right) - \bar{h}\left( t \right)} \right)}}{{\sum\nolimits_{i = 1}^{L} {\left[ {h\left( {i,t} \right) - \bar{h}\left( t \right)} \right]^{2} } }}$$

to identify the dominant wavelengths, if any. In the above equation, u is the shift in pixels, 0 < u < L, and L is the image width. R (t, u) will always have a peak of 1 when u = 0. If additional peaks are present, say at u = λ 1, λ 2, …, we conclude that the \(\lambda_{i}\) are wavelengths. Figure 3d depicts contours of the cross-correlation R (t, u) as a function of the shift u and the frame number t. In all the frames, R (t, u) decays monotonically as u increases from zero, indicating the absence of periodic spatial behavior. This behavior is consistent with previous observations of the evolution of electrodeposited surfaces [43].

The morphology during electrodeposition is directly controlled by the normal current density at any given position along the interface [44, 45], which is in turn, related to the normal local speed of the interface via a mass balance or Faraday law (with appropriate assumptions about the deposition efficiency). It is thus desirable to obtain the normal velocity of the extracted edge [46]

$$v_{N} \left( {i,t} \right) = \mathop {\lim }\limits_{\Updelta t \to 0} \left( {\frac{{S_{t + \Updelta t} \left( {i_{N} } \right) - S_{t} (i)}}{\Updelta t}} \right),$$

where v N (i, t) is the normal velocity at position i and time t along the temporally evolving interface, S t . The velocity is determined by the vector distance between the point of interest, i, and the point that lies along the normal to S t (i), called i N , on the interface \(S_{t + \varDelta t}\). Essentially, this problem is reduced to finding the distance from a point on the current surface and the point of intersection between the normal to S t and the surface after it has evolved by the time interval Δt. Figure 4a illustrates the process when the surfaces (curves) are described with continuous functions.

Fig. 4
figure 4

Illustration of normal velocity extraction for a a smooth continuous surface, b a discretized surface specified with pixels and approximated with a least square fitted curve, and c the minimum distance approach. Figure 3a and e presents an example of this analysis

When processing experimental data, the situation is more complicated. Instead of a smooth curve, we have a noisy 2D point cloud (Fig. 4b). To address the noisy data, we have implemented two methods for extracting the normal direction to the surface. The first method is based on a moving least square algorithm [47]. To construct a normal to the surface at point i, we fit the experimental data locally, around the point i, with the m-degree polynomial \(P_{m} = \sum\nolimits_{j = 0}^{m} a_{j} x^{j}\), where we maintain the laboratory frame of reference. The coefficients a i are calculated by minimizing the error

$$E^{2} = \sum_{{k = i - \frac{n - 1}{2}}}^{{i + \frac{n - 1}{2}}} {\left[ {S_{t} \left( k \right) - P_{m} \left( {x_{k} ,a_{0} ,a_{1} , \ldots ,a_{m} } \right)} \right]^{2} } ,$$

where n is an odd integer number of points along the extracted edge’s point cloud centered about i. This method works best for low noise data and when the frame rate is low (i.e., the interface moves more than a few pixels per frame).

An alternative method exploits the fact that the shortest distance from a point to a surface/curve lies along the normal to the curve. To construct a normal to the surface St that intersects point j on surface St+Δt, we seek point i * on St such that the distance between points i and j is minimized. Figure 4c illustrates this process. In fact, Mathematica has built-in routines to quickly find which point from a point cloud (ordered or not) is closest to a point of interest. This proved to be a very effective and fast means for the extraction of point-wise normal velocity, especially in the presence of noise.

We applied the velocity routine to the evolving interface of Additional file 3: Video S3. Figure 3e shows the resulting “heat map” (contour plot) of the normal speed in space (along the interface) as a function of time. Initially, we see uniform distribution of speeds and therefore uniform current density. As the roughness develops due to natural fluctuations, the current density becomes less uniform locally, leading to continued amplification of the roughness.

Facet-dependent growth and dissolution rates in nanoparticles

Size and morphology control of nanocrystals is essential to tailor the physical and chemical properties of the resulting material [7, 8]. Liquid phase synthesis is a convenient, low-cost and versatile technique for fabricating well-controlled nanocrystals, since solution chemistry can be tuned to give specific morphology. A fundamental understanding of the driving force and principles that govern morphologies during nucleation and growth is essential to optimize existing synthesis methods and to discover novel methods. Experiments on the dissolution and growth of gold nanorods were carried out with the nanoaquarium in an FEI Quanta 600 FEG Mark II SEM with a transmission detector. The aqueous solution contained gold nanorods [48] with a trace amounts of the surfactant cetrimonium bromide (CTAB) and HAuCl4 at pH ~7. The devices were filled under atmospheric conditions and the solution was not deaerated. Production of radiolytic species by the electron beam, as described in Schneider et al. [20], causes the Au nanorods to either grow or dissolve. During dissolution, we hypothesize that the radicals produced during water irradiation react with the CTAB, whose role is to stabilize the nanorods; in the absence of CTAB, nanorods are known to be unstable and dissolve [49, 50]. Analysis of morphology and growth dynamics at specific facets provides information about the atomic level processes that control shape [51].

Additional file 4: Video S4 shows beam-induced dissolution of gold nanorods. Focusing on a single nanorod undergoing dissolution, we use the vertices of the minimum bounding box to obtain both the particle’s aspect ratio and its orientation as functions of time. Figure 5a shows the image sequence of the nanorod after binarization and with the superposition of the minimum bounding box vertices. The smallest bounding box is used due to the nearly rectangular shape of the particle in projection. We use Toussaint’s rotating caliper method [52] to rotate the box around the object and extract the smallest bounding rectangle. Alternative geometric shapes and minimum area algorithms can be selected for other particle morphologies. Many of the convex hull algorithms, including the minimum bound box, are available in Mathematica. See the supplemental Code 1. Figure 5b, c, and d depicts, respectively, the particle’s aspect ratio, perimeter, and rotation from the initial orientation as functions of time. Interestingly, the etching is not isotropic, as is evident from the non-monotonic variation of the aspect ratio. The particle first etches at the long facets, increasing its aspect ratio, and only later do the tips etch and the particle becomes isotropic. This is likely related to the distribution of ligands (CTAB) that remain from the manufacturing process [48]. CTAB binds more strongly to the long facets. During growth, this effectively reduces access of reducible species to the long facets and encourages the formation of rods [53, 54]. Conversely, we hypothesize that as the beam creates an oxidizing environment and degrades the surface bound CTAB [20], the long edge is affected first, resulting in the observed anisotropic etching. This is somewhat analogous to growth studies [51] that suggest that the position and dynamics of ligands on a nanoparticle surface determine the growth rate for each facet. The local pH (driven toward acidic by the beam [20]) is also likely to play a role in the degree of anisotropy during the etching process [48]. We can track the perimeter of the particle (Fig. 5c) and its rotation (Fig. 5d). The rotation can be used along with the center of mass to extend the code to extract the point-wise normal etching (or growth) velocity similar to the analysis in [23].

Fig. 5
figure 5

Beam-mediated etching of gold nanorods (nominally 50 × 80 nm) suspended in water (pH ~7) with trace amounts of the surfactant cetrimonium bromide (CTAB) and chloroauric acid. Imaging was carried with a FEI Quanta 600 in STEM mode (30 keV) beam currents of 0.07 nA, beam radius of ~1 nm resulting in a time average dose rate of 7.4 × 106 (Gy/s) or 2 (e2-s). a An image sequence of a gold nanorod dissolution. The images are binarized and superposed with the minimum bounding box (green). The box vertices are shown in red. bd Particle aspect ratio, perimeter, and inclination angle from initial orientation, respectively, as a function of frame number

Bubble nucleation, growth, and migration

The dynamics of gas–liquid interfaces plays an important role in many physical processes such as high efficiency phase change heat transfer. In recent years, the existence and stability of nanobubbles have been debated [5558]. Recently, atomic force microscopy (AFM) measurements have confirmed the existence and unraveled the geometry of nanobubbles [55, 56, 58, 59]. However, such measurements have limited spatial resolution and there is an additional concern that the mechanical perturbation applied by the cantilever is too invasive. Electron microscopy imaging of nanobubbles provides information that complements AFM measurements.

Here, we consider the motion of radiolytically formed nanobubbles confined in a tapered conduit. Given the bubbles’ size, both the capillary and Bond numbers are essentially zero. In this fluid dynamical regime, motion and geometry of the nanobubbles are governed by three-phase contact line wetting and de-wetting. Identifying the interfaces of single bubbles and tracking their trajectories is essential in obtaining the quantitative information needed to model contact line dynamics. In this case, both the contact line velocities and trajectories of the centroids are of interest.

Additional file 5: Video S5 shows the nucleation, growth, and migration of a nanobubble. The bubble is shown at a particular instance in Fig. 6a (left). Figure 6b depicts the edge of the bubble at various times. We first extract the effective radius of the bubble by finding the circular area equivalent radius from the binarized image

$$r = \sqrt {\frac{{A_{{Binarized}} }}{\pi }} .$$

The area can be obtained by counting the number of pixels in the binary convex hull of the bubble object. Figure 6c depicts the bubble radius as a function of the frame number. To track the bubble’s location throughout the growth process, instead of taking the area centroid, we account for the three-dimensional shape of the bubble by taking the detected electron intensity-weighted centroid, where each pixel is multiplied by its intensity value. This serves to find the centroid most closely aligned with the center of mass of the vapor bubble, while using the two-dimensional projection.

Fig. 6
figure 6

Nanobubble formation in an aqueous solution of gold nanorods with a measured pH ~7 and trace amounts of the surfactant cetrimonium bromide (CTAB). Imaging was performed with at beam current ~1 nA and beam radius ~2 μm resulting in a dose rate of 7.5 × 107 (Gy/s) or 20 (e2-s). a Bubble radius as a function of frame. b Comparison of raw, binary, and masked images. c Bubble intensity weight centroid (squares) with bubble edge outline, color-coded in time

Data processing and analysis tutorial

For each of the three examples given above, i.e., growth front tracking during electrochemical deposition, nanoparticle dissolution, and bubble motion, an overview is described in the supplemental Additional file 6: Code S1. This is available on GitHub [25] with an accompanying video tutorial Additional file 1: Video S1. Fully automatic and unsupervised analysis of the three sample datasets presented in this paper is available in the supplemental Additional file 7: Code S2, Additional file 8: Code S3, Additional file 9: Code S4, also maintained on GitHub [25].


Liquid cell electron microscopy is a powerful technique with many applications to dynamic processes in liquid media. The extraction of quantitative measures is essential for proper interpretation of the data. However, typical experiments generate very large quantities of data, which can be a challenge to analyze quantitatively. We have presented a workflow that is suited to analyze typical data of the type collected during liquid cell electron microscopy. We chose Mathematica as the platform to implement the various procedures because of its ease of use, availability of numerous built-in algorithms for image processing, automation, and flexibility. We hope that the tools presented here and the tutorials provided in the supplement will be useful to the microscopy community for diverse applications of data analysis for dynamical processes both in liquid cell experiments and for in situ microscopy in general.


  1. Nicoli, M., Castro, M., Cuerno, R.: Unified moving-boundary model with fluctuations for unstable diffusive growth. Phys Rev E 78, 021601 (2008). doi:10.1103/PhysRevE.78.021601

    Article  CAS  Google Scholar 

  2. White, E.R., Singer, S.B., Augustyn, V., et al.: In situtransmission electron microscopy of lead dendrites and lead ions in aqueous solution. ACS Nano 6, 6308–6317 (2012). doi:10.1021/nn3017469

    Article  CAS  Google Scholar 

  3. Sun. M., Liao. H-G., Niu K., Zheng. H.: Structural and morphological evolution of lead dendrites during electrochemical migration. Sci Reports 3 (2013). doi:10.1038/srep03227

  4. Sacci, R.L., Black, J.M., Balke, N., et al.: Nanoscale imaging of fundamental Li battery chemistry: solid-electrolyte interphase formation and preferential growth of lithium metal nanoclusters. Nano Lett 15, 2011–2018 (2015). doi:10.1021/nl5048626

    Article  CAS  Google Scholar 

  5. Leenheer, A.J., Jungjohann, K.L., Zavadil, K.R., et al.: Lithium electrodeposition dynamics in aprotic electrolyte observed in situ via transmission electron microscopy. ACS Nano 9, 4379–4389 (2015). doi:10.1021/acsnano.5b00876

    Article  CAS  Google Scholar 

  6. Zeng. Z., Liang. W-I., Liao. H-G., et al.: visualization of electrode-electrolyte interfaces in lipf6/ec/dec electrolyte for lithium ion batteries via in-situ TEM. Nano letters 140120111747008 (2014). doi: 10.1021/nl403922u

    Article  CAS  Google Scholar 

  7. Tao, A.R., Habas, S., Yang, P.: Shape control of colloidal metal nanocrystals. Small 4, 310–325 (2008). doi:10.1002/smll.200701295

    Article  CAS  Google Scholar 

  8. Jin, R., Cao, Y., Mirkin, C.A., et al.: Photoinduced conversion of silver nanospheres to nanoprisms. Science 294, 1901–1903 (2001). doi:10.1126/science.1066541

    Article  CAS  Google Scholar 

  9. Park. JH., Schneider. NM., Grogan JM., et al.: control of electron beam-induced au nanocrystal growth kinetics through solution chemistry. Nano letters 150729101858009 (2015). doi: 10.1021/acs.nanolett.5b01677

    Article  CAS  Google Scholar 

  10. Hermannsdörfer, J., de Jonge, N., Verch, A.: Electron beam induced chemistry of gold nanoparticles in saline solution. Chem Commun (2015). doi:10.1039/C5CC06812F

    Article  Google Scholar 

  11. de Jonge, N., Ross, F.M.: Electron microscopy of specimens in liquid. Nature Nanotech 6, 695–704 (2011). doi:10.1038/nnano.2011.161

    Article  CAS  Google Scholar 

  12. Grogan, J.M., Schneider, N.M., Ross, F.M., Bau, H.H.: The nanoaquarium: a new paradigm in electron microscopy. J Indian Inst Sci 92, 295–308 (2012)

    CAS  Google Scholar 

  13. Mehdi. BL., Qian. J., Nasybulin. E., et al.: Observation and quantification of nanoscale processes in lithium batteries by operando electrochemical (s)TEM. Nano letters 150225082830003 (2015). doi: 10.1021/acs.nanolett.5b00175

    Article  CAS  Google Scholar 

  14. Williamson, M.J., Tromp, R.M., Vereecken, P.M., et al.: Dynamic microscopy of nanoscale cluster growth at the solid–liquid interface. Nat Mater 2, 532–536 (2003). doi:10.1038/nmat944

    Article  CAS  Google Scholar 

  15. Radisic, A., Vereecken, P.M., Hannon, J.B., et al.: Quantifying electrochemical nucleation and growth of nanoscale clusters using real-time kinetic data. Nano Lett 6, 238–242 (2006). doi:10.1021/nl052175i

    Article  CAS  Google Scholar 

  16. den Heijer, M., Shao, I., Radisic, A., et al.: Patterned electrochemical deposition of copper using an electron beam. APL Materials 2, 022101 (2014). doi:10.1063/1.4863596

    Article  CAS  Google Scholar 

  17. Leenheer. AJ., Sullivan. JP., Shaw. MJ., Harris. CT.: A sealed liquid cell for in situ transmission electron microscopy of controlled electrochemical processes. Microelectromechanical Systems J 1–1. doi: 10.1109/JMEMS.2014.2380771

  18. Schneider, N.M., Park, J.H., Grogan, J.M., et al.: In-situ electron microscopy of electrochemical deposition, dendrite growth, and etching. J Heat Transfer 136, 080910 (2014). doi:10.1115/1.4027528

    Article  Google Scholar 

  19. Schneider, N.M., Hun Park, J., Grogan, J.M., et al.: Visualization of active and passive control of morphology during electrodeposition. Microsc Microanal 20, 1530–1531 (2014). doi:10.1017/S1431927614009386

    Article  Google Scholar 

  20. Schneider, N.M., Norton, M.M., Mendel, B.J., et al.: Electron-water interactions and implications for liquid cell electron microscopy. J Phys Chem C 118, 22373–22382 (2014). doi:10.1021/jp507400n

    Article  CAS  Google Scholar 

  21. Park, J.H., Grogan, J.M., Bau, H.H., et al.: Mechanism of electron-beam induced au formation in aqueous solution. Microsc Microanal 19, 486–487 (2013). doi:10.1017/S143192761300442X

    Article  Google Scholar 

  22. Woehl, T.J., Evans, J.E., Arslan, I., et al.: Direct in situ determination of the mechanisms controlling nanoparticle nucleation and growth. ACS Nano 6, 8599–8610 (2012). doi:10.1021/nn303371y

    Article  CAS  Google Scholar 

  23. Ievlev, A.V., Jesse, S., Cochell, T.J., et al.: Quantitative description of crystal nucleation and growth from in situ liquid scanning transmission electron microscopy. ACS Nano 9, 11784–11791 (2015). doi:10.1021/acsnano.5b03720

    Article  CAS  Google Scholar 

  24. Norton. M., Park. J., Kodambaka. S., et al.: Dynamics of Sub-micron bubbles growing in a wedge in the low capillary number regime. bulletin of the american physical society 59, 20 (2014)

  25. Schneider NM NMSchneider/Quantitative-Image-Analysis. In: Accessed 13 Feb 2015

  26. Grogan, J.M., Schneider, N.M., Ross, F.M., Bau, H.H.: Bubble and pattern formation in liquid induced by an electron beam. Nano Lett 14, 359–364 (2014). doi:10.1021/nl404169a

    Article  CAS  Google Scholar 

  27. Schneider. NM.: Video S1 Code S1 walk through. Online video clip. YouTube. YouTube, 9 Aug 2015. Web. 24 Jan 2016. (2015)

  28. Lee. A.: Virtualdub. (2015). Accessed 24 Jan 2015

  29. Rasband, W.S., ImageJ, U. S.: National Institutes of Health, Maryland (1997–2015). Accessed 24 Jan 2015

  30. Schneider. NM.: liquid cell electron microscopy with the nanoaquarium: radition and electrochemistry. 1–222 (2015)

  31. Woehl, T.J., Jungjohann, K.L., Evans, J.E., Arslan, I.: Experimental procedures to mitigate electron beam induced artifacts during in situ fluid imaging of nanomaterials. Ultramicroscopy 127, 53–63 (2013). doi:10.1016/j.ultramic.2012.07.018

    Article  CAS  Google Scholar 

  32. Abellan, P., Woehl, T.J., Parent, L.R., et al.: Factors influencing quantitative liquid (scanning) transmission electron microscopy. Chem Commun 50, 4873–4880 (2014). doi:10.1039/C3CC48479C

    Article  CAS  Google Scholar 

  33. Rose, A.: Television pickup tubes and the problem of vision. Adv Electron (1948). doi:10.1016/S0065-2539(08)61102-6

    Article  Google Scholar 

  34. Jonge, N.D., Peckys, D.B., Kremers, G.J., Piston, D.W.: Electron microscopy of whole cells in liquid with nanometer resolution. Proc Natl Acad Sci 106, 2159–2164 (2009). doi:10.1073/pnas.0809567106

    Article  Google Scholar 

  35. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992). doi:10.1016/0167-2789(92)90242-F

    Article  Google Scholar 

  36. Chan, T.F., Osher, S., Shen, J.: The digital TV filter and nonlinear denoising. Image Process IEEE Trans 10, 231–241 (2001). doi:10.1109/83.902288

    Article  CAS  Google Scholar 

  37. Piccardi. M.: Background subtraction techniques: a review. ICSMC-04 4. 4, 3099–3104. doi: 10.1109/ICSMC.2004.1400815

  38. Otsu. N.: A threshold selection method from gray-level histograms. systems, man and cybernetics, IEEE transactions on 9, 62–66 (1979). doi: 10.1109/TSMC.1979.4310076

    Article  Google Scholar 

  39. Canny. J.: A computational approach to edge detection. pattern analysis and machine intelligence, IEEE Transactions on 679–698. (1986). doi: 10.1109/TPAMI.1986.4767851

    Article  Google Scholar 

  40. Crocker, J.: Methods of digital video microscopy for colloidal studies. J Colloid Interface Sci 179, 298–310 (1996). doi:10.1006/jcis.1996.0217

    Article  CAS  Google Scholar 

  41. Grogan, J.M., Bau, H.H.: The nanoaquarium: a platform for in situ transmission electron microscopy in liquid media. Microelectromechanical Systems J 19, 885–894 (2010). doi:10.1109/JMEMS.2010.2051321

    Article  CAS  Google Scholar 

  42. Barkey, D.P.: Structure and pattern formation in electrodeposition. Adv Electrochem Sci Eng 7, 151–192 (2002). doi:10.1002/3527600264.ch3

    Article  Google Scholar 

  43. Schwarzacher, W.: Kinetic roughening of electrodeposited films. J Phys Condens Matter 16, R859–R880 (2004). doi:10.1088/0953-8984/16/26/R01

    Article  CAS  Google Scholar 

  44. Haataja, M., Srolovitz, D.: Morphological instability and additive-induced stabilization in electrodeposition. Phys Rev Lett 89, 215509 (2002). doi:10.1103/PhysRevLett.89.215509

    Article  CAS  Google Scholar 

  45. Barkey, D.P., Muller, R.H., Tobias, C.W.: Roughness development in metal electrodeposition II. Stability theory. J Electrochem Soc 136, 2207–2214 (1989). doi:10.1149/1.2097260

    Article  CAS  Google Scholar 

  46. Grinfeld. P.: Introduction to tensor analysis and the calculus of moving surfaces. (2013). doi: 10.1007/978-1-4614-7867-6

    Book  Google Scholar 

  47. Kolluri, R.: Provably good moving least squares. ACM Trans Algorithms (TALG) 4, 18–25 (2008). doi:10.1145/1361192.1361195

    Article  Google Scholar 

  48. Ye, X., Jin, L., Caglayan, H., et al.: Improved size-tunable synthesis of monodisperse gold nanorods through the use of aromatic additives. ACS Nano 6, 2804–2817 (2012). doi:10.1021/nn300315j

    Article  CAS  Google Scholar 

  49. Jiang, Y., Zhu, G., Lin, F., et al.: In situ study of oxidative etching of palladium nanocrystals by liquid cell electron microscopy. Nano Lett 14, 3761–3765 (2014). doi:10.1021/nl500670q

    Article  CAS  Google Scholar 

  50. Hellebusch, D.J., Manthiram, K., Beberwyck, B.J., Alivisatos, A.P.: In situ transmission electron microscopy of cadmium selenide nanorod sublimation. J Phys Chem Lett 6, 605–611 (2015). doi:10.1021/jz502566m

    Article  CAS  Google Scholar 

  51. Liao, H.G., Zherebetskyy, D., Xin, H., et al.: Facet development during platinum nanocube growth. Science 345, 916–919 (2014). doi:10.1126/science.1253149

    Article  CAS  Google Scholar 

  52. Toussaint. GT.: Solving geometric problems with the rotating calipers. Proc IEEE Melecon. (1983)

  53. Katz-Boon, H., Walsh, M., Dwyer, C., et al.: Stability of crystal facets in gold nanorods. Nano Lett 15, 1635–1641 (2015). doi:10.1021/acs.nanolett.5b00124

    Article  CAS  Google Scholar 

  54. Bullen, C., Zijlstra, P., Bakker, E., et al.: Chemical kinetics of gold nanorod growth in aqueous CTAB solutions. Cryst Growth Des 11, 3375–3380 (2011). doi:10.1021/cg101636r

    Article  CAS  Google Scholar 

  55. Tyrrell. JWG., Attard. P.: Atomic force microscope images of nanobubbles on a hydrophobic surface and corresponding force-separation data. 18, 160–167 (2002). doi: 10.1021/la0111957

    Article  CAS  Google Scholar 

  56. Tyrrell, J.W.G., Attard, P.: Images of nanobubbles on hydrophobic surfaces and their interactions. Phys Rev Lett 87, 176104 (2001)

    Article  CAS  Google Scholar 

  57. Attard, P., Moody. MP., Tyrrell. J.: Nanobubbles: the big picture. Physica A: statistical mechanics and its ….. (2002). doi: 10.1016/S0378-4371(02)01191-3

    Article  CAS  Google Scholar 

  58. Attard, P.: Nanobubbles and the hydrophobic attraction. 104, 75–91 (2003). doi: 10.1016/S0001-8686(03)00037-X

    Article  CAS  Google Scholar 

  59. Lou, S.-T., Ouyang, Z.-Q., Zhang, Y., et al.: Nanobubbles on solid surface imaged by atomic force microscopy. J Vac Sci Technol B 18, 2573–2575 (2000). doi:10.1116/1.1289925

    Article  CAS  Google Scholar 

Download references

Authors’ contributions

NMS, FMR, and HHB conceived the project and all interpreted the results and wrote the manuscript. NMS, JHP, MMN, and FMR designed and carried out the experiments. NMS implemented image processing schemes and conducted data analysis. All authors read and approved the final manuscript.


The nanoaquarium fabrication was carried out at the Cornell NanoScale Facility (NSF Grant ECS-0335765), a member of the National Nanotechnology Infrastructure Network. Electron microscopy was carried out at the Penn Regional Nanotechnology Facility and the IBM T. J. Watson Research Center with the valuable assistance of Dr. Joseph M. Grogan and Mr. Peter Szczesniak of UPenn and Dr. Mark C. Reuter and Mr. Arthur Ellis of IBM. Gold nanorods were generously provided by Dr. Christopher Murray. The work was supported, in part, by the National Science Foundation Grants 1129722 and 1066573 to the University of Pennsylvania, and 1310639 to the University of California Los Angeles.

Competing interests

The authors declare that they have no competing interests.

Author information

Authors and Affiliations


Corresponding authors

Correspondence to Nicholas M. Schneider, Frances M. Ross or Haim H. Bau.

Additional files


Additional file 1: Video S1. Code Sample Walkthrough This narrated screencast is a walkthrough of Code S1, which shows the fundamentals of the image processing. Using Video S2 and images from the experiments described in the main text, we recreate the processed images shown in Figs. 1b, 2 of the main text. The video is hosted on the journal website and on YouTube under a Creative Commons CopyRight License [].


Additional file 2: Video S2. This Video shows gold crystal growth in the nanoaquarium [41] with transmitted beam current of 7.8 nA spread over 2 μm diameter resulting in a dose rate of 5.9x108 (Gy/s) or 160 (e2-s). Stock solutions of chloroauric acid (Sigma-Aldrich) were made by dissolving the salt in deionized (DI) water at concentration of 20 mM. The liquid thickness within the imaged region is 50-100 nm. Irradiation by the electron beam creates hydrated electrons that reduce the Au3+ ions to Au and form nanocrystals attached to either the bottom or the ceiling of the nanoaquarium’s observation chamber. The experiments took place in a Hitachi H-9000 TEM at 300 kV using bright-field imaging conditions. The growth process was examined in more details by Park et al. [9]. This video is used in conjunction with Code S1 to generate Fig. 2 in the main text.


Additional file 3: Video S3. This Video shows copper electrodeposition in the nanoaquarium [41], operating in a three-terminal configuration with Pt electrodes controlled by a Gamry potentiostat. Aqueous solutions of 0.1 M copper sulfate with 0.18 M sulfuric acid solutions were prepared in doubly deionized water and were not deaerated before use. The interface morphology evolution was imaged at video rate (30 frames per second) during galvanostatic deposition with a total current of 400 nA with transmitted beam current of 0.1 nA spread over ~ 2 μm diameter resulting in a dose rate of 7.5 × 106 (Gy/s) or 2 (e2-s). The liquid thickness at the edge of the silicon nitride window is fixed to 200 nm by lithography. The experiments took place in a Hitachi H-9000 TEM at 300 kV using bright-field imaging conditions. This video is used in conjunction with Code S2 to generate Fig. 3 in the main text.


Additional file 4: Video S4. This Video shows examples of beam-mediated etching of gold nanorods (nominally 50x80 nm). The nanorods are suspended in water (pH ~7) with trace amounts of the surfactant cetrimonium bromide (CTAB) and chloroauric acid. Imaging was carried with a FEI Quanta 600 in STEM mode (30 keV) beam currents of 0.07 nA, beam radius of ~ 1 nm resulting in a time average dose rate of 7.4 x 106 (Gy/s) or 2 (e2-s). This video is used in conjunction with Code S3 to generate Fig. 5 in the main text.


Additional file 5: Video S5. Nanobubble formation experiments in an aqueous solution of gold nanorods with a measured pH ~ 7 and trace amounts of the surfactant cetrimonium bromide (CTAB). Imaging was performed with a Hitachi 9000 TEM (300 keV) at beam current ~ 1 nA and beam radius ~ 2 μm resulting in a dose rate of 7.5 x 107 (Gy/s) or 20 (e2-s). The bubbles are discussed in Grogan et al. [26] and Norton et al. [24]. This video is used in conjunction with Code S4 to generate Fig. 6 in the main text.


Additional file 6: Code S1. Code S1 is a Mathematica notebook file that gives an overview of the image processing work flow described in the text. It can be used with Video S2 to generate the raw image assets need to construct Figs. 1, 2 in the main text. A narrated screencast outlining the use of this file can be found in Video S1.


Additional file 7: Code S2. Code S2 is a Mathematica notebook file that gives an overview of the image processing work flow described in the text. It can be used with Video S3 to generate the raw image assets need to construct Fig. 3 in the main text.


Additional file 8: Code S3. Code S3 is a Mathematica notebook file that gives an overview of the image processing work flow described in the text. It can be used with Video S4 to generate the raw image assets need to construct Fig. 5 in the main text.


Additional file 9: Code S4. Code S4 is a Mathematica notebook file that gives an overview of the image processing work flow described in the text. It can be used with Video S5 to generate the raw image assets need to construct Fig. 6 in the main text.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schneider, N.M., Park, J.H., Norton, M.M. et al. Automated analysis of evolving interfaces during in situ electron microscopy. Adv Struct Chem Imag 2, 2 (2016).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: