Advertisement

Diffractive Imaging of Single Particles

  • John C. H. SpenceEmail author
Chapter
Part of the Springer Handbooks book series (SHB)

Abstract

The computational methods and applications of diffractive (lensless) imaging are reviewed. Far-field scattering (e. g., by neutrons, electrons, light, or x-rays) by a nonperiodic localized potential is detected and the phase problem solved using iterative optimization methods, which allows a three-dimensional image of the object potential to be reconstructed without the need for a lens. The history of the subject and its relationship to the crystallographic phase problem are reviewed, together with a summary of theory, algorithms, uniqueness issues, resolution limits, the constraint ratio concept, and coherence requirements. Applications, from various forms of microscopy (electron, optical, and x-ray) to snapshot x-ray laser single-particle imaging, are reviewed. The method of ptychography, which achieves a similar aim, is reviewed elsewhere in these volumes.

Diffractive (or lensless) imaging refers to the use of theoretical methods and computer algorithms to solve the phase problem for scattering by a nonperiodic object. The topic is also covered in the chapter Ptychography (Chap.  17). Here we first review the history of the noncrystallographic phase problem by other methods, describe early experimental results with electron and x-ray beams, and the iterative phasing algorithms used, and then specialize to their application to imaging nonperiodic biological samples using coherent x-ray lasers ( ), which are not suited to ptychography when using a destructive-readout mode. We do not cover the rapidly-growing related field of phase-contrast x-ray imaging in medicine [20.1].

The name coherent x-ray diffractive imaging ( ) is used in the x-ray community, which we could generalize to coherent diffractive imaging ( ). Additional information about the object, such as the sign of the scattering potential and the approximate boundary of the object, may be combined with the measured scattered intensity to solve for the phases of the scattered amplitudes. In this way, under conditions of single scattering (and other approximations which often apply in optics, electron, x-ray, and neutron diffraction) one aims to reconstruct a real-space image of an object by Fourier transform of the complex scattering distribution, or Fraunhofer far-field diffraction pattern. This noncrystallographic phase problem has been described as interferometry without an interferometer. Applications to Fresnel near-field imaging are also possible, where, however, resolution may be limited by detector pixel size, since the magnification is unity, unless a lens or a diverging beam are used. The powerful transport of intensity method has been used for this purpose in the near field [20.2].

By avoiding the need for a lens, CDI avoids the aberrations and resolution limits introduced by lenses. Within the past two decades this method has been applied to experimental neutron, x-ray, light, and electron scattering data. The electron work has produced atomic-resolution experimental images in projection [20.3], while experiments with soft x-rays have produced three-dimensional () (tomographic) reconstructions at \({\mathrm{10}}\,{\mathrm{nm}}\) resolution [20.4]. It now offers the real possibility of diffraction-limited imaging with any radiation for which lenses do not exist. Since each radiation interacts differently with matter, the method can be expected to provide us with new information on matter in fields as diverse as biology, materials science, and astronomy. Much current interest focuses on tomographic imaging of whole cells, nanoparticles, chemical mapping  [20.5], and mesoporous materials, and on the imaging of proteins and viruses at near-atomic resolution using x-ray lasers [20.6]. While it can be shown rigorously that some of the iterative algorithms described here do not diverge, because of the nonconvex nature of the constraints involved, no formal proof of convergence or of the uniqueness of solutions exists. Yet computational trials for simulated data with little noise practically always converge rapidly to the known solution, while the inevitable stagnation occurs in the presence of excessive noise. (The ability of these algorithms to climb out of the local minimum and find a global minimum in a data set with perhaps a million unknown parameters (phases) is one of their most remarkable features.) In summary, over the last two decades many algorithms for lensless imaging have been published, which work well on simulated noisy data, while increasing success has been achieved with experimental data.

20.1 History

This review section presents the history of ideas which led to the solution of the noncrystallographic phase problem in qualitative terms. The phase problem in optics has early origins: in a letter to Michelson, Rayleigh comments that ‘‘the phase problem in interferometry is insoluble without a priori information on the symmetry of the data'' [20.7]. Several fields have contributed to the development of our current working solution to this problem, including signal processing, wave-front sensing, astronomy, electron microscopy, image processing, and x-ray crystallography, in addition to applied mathematics. Thus, a wonderfully rich set of ideas has contributed, from communications theory to diffraction physics and nonconvex optimization by iterated projections.

The origins of the solution to the noncrystallographic phase problem lie in crystallography, however this was not recognized until recently [20.8]. The most successful iterative phasing methods for single particles are akin to the solvent flattening methods of crystallography. The review by Millane and Lo [20.9] describes iterative phasing in crystallography, where questions of uniqueness, and the powerful constraint ratio \(\Omega\) are discussed. (A unique solution, with the number of Fourier equations equal to the number of unknown phases, requires \(\Omega> {\mathrm{1}}\), which holds only for a protein crystal whose unit cell contains mostly solvent, as is further discussed below.) In retrospect, we can now see that the key to a successful solution was given in the crystallography literature as early as 1952. D. Sayre, in Some implications of a theorem due to Shannon [20.10], appears to have been the first to consider the relationship between Shannon's sampling theorem and Bragg's law. He pointed out that, for centrosymmetric crystals, where only the sign of structure factors \(F_{g}\) is unknown, these could be determined if intensities \(|F_{g}|^{2}\) were observable at half-integral values of \(\boldsymbol{g}\) (a reciprocal lattice vector), since a change of sign requires a zero-crossing of the molecular scattering factor there. By Shannon's theorem , these fractional and integral order intensities are needed and sufficient to completely define the autocorrelation function of the molecule. Sayre's fractional orders could thus only be generated by a molecule which filled half (in one dimension) of the cell, leaving the remaining half empty. If the density in the empty half of the cell is assumed known, it will be found that the number of Fourier equations relating real and reciprocal space are equal to the number of unknown phases, and the problem reduces to solving these (nonlinear) equations. This is best done by iterative search methods. (A modern application of exactly this method to a protein crystal in which the unit cell is occupied mostly by solvent can be found in He and Su [20.11].) Sayre concludes that a solution to the phase problem for centric crystals could as well be obtained from measurements of the intensity of half-order reflections as from study of the (complex) Bragg reflections. The paper was prompted by the observation of nonintegral reflections (following hydration) in experimental patterns from haemoglobin by Perutz and others and a talk by Gay in 1949. (Similarly, the phase problem may be solved if one has measured structure factors from the same molecule crystallized in a second space group, to provide additional sampled intensities between Bragg reflections of the first space group.) Note that the Patterson or autocorrelation function of a molecular crystal may differ from that of the isolated molecule, since the autocorrelation of a molecule which fills the cell will be twice as large as the unit cell in each direction and so produce overlap. Since diffraction from an isolated nonperiodic object is a continuous function of the scattering angle, it provides immediate access to these fractional orders. Thus, a solution in principle to the phase problem for a real, centrosymmetric single particle has existed in the literature since 1952.

Sayre's paper had no impact in the image processing or signal processing communities. For imaging in two dimensions (in the projection approximation), the next important development was work by Gerchberg and Saxton (G–S ) [20.12], which posed the following question: if the intensity of a general complex two-dimensional () image and its corresponding diffraction pattern intensity is known, can the complex image (and diffraction pattern) be reconstructed? (By a complex image we refer here to the sample exit-face wavefunction, as defined in the next section. The reconstruction of the sample structure from this may involve further approximations.) This data is available in a modern electron microscope. By solving the resulting quadratic equations (equal in number, as they comment, to the number of unknown phases) using an iterative algorithm, they showed successful inversion for one-dimensional ( ) data. It has often been noted that these Fourier equations, one for each pixel in the detector, may not be independent, are nonlinear if a sign constraint is used, and are corrupted by noise. Dependence in the equations may be introduced by the choice of support shape. Other approaches have been explored for solving these nonlinear Fourier equations, but the G–S paper established the use of iterations between real and reciprocal space, with known information imposed repeatedly in each domain. G–S do not establish uniqueness or convergence in general. The connection with Sayre's paper was not made, however, in retrospect we see that the sampling interval used by G–S for their analysis of a nonperiodic object necessarily corresponded to the Shannon half-integral orders of Sayre's paper.

Modern work is focused on 3-D data in which only the diffraction pattern intensity from a real or complex object is measured. (A complex object is one whose refractive index at each voxel may be complex, doubling the amount of information required.) In that case, the curvature of the Ewald sphere cannot be ignored, the projection approximation fails, and a simple Fourier transform of the phased 2-D diffraction pattern does not give a projection of the object density. However, for a pure phase object in transmission electron microscopy ( ), the modulus of the exit face wavefunction is known a priori to be unity everywhere, greatly reducing the number of unknowns in the G–S analysis. (This unit modulus constraint is, unfortunately, nonconvex.) For a real object (or, equivalently, a weak phase object), the diffraction pattern has Hermitian symmetry (Friedel's law) , resulting in a reduction in unknowns. Following the G–S paper (and a second paper [20.13]), it remained to determine the minimum information needed about a real object to solve the phase problem if the diffracted intensity was given. The role of the object boundary (defining a support) and the need to sample the continuous scattering finely enough to satisfy Shannon's theorem, (treating the autocorrelation function as the bandlimit) were all recognized at this time. The autocorrelation of a 2-D object is twice as large as the object in any direction, so Shannon's theorem requires sampling at half the Bragg angle for the periodically continued object. Bates referred to this as oversampling, a term still in use. (Since it is, in fact, optimum sampling of the diffracted intensity (for reconstruction of the autocorrelation function), we avoid that term here.) The problem was vigorously attacked by authors such as Bates, Fiddy, Fienup, Gonslaves, and Papoulis in the early 1980s, and the use of lensless imaging based on these ideas using x-ray scattering was advocated [20.14]. The early series of papers by Bates co-workers are especially significant and can be traced through Bates and McDonnell [20.15], while Fiddy provided a new approach based on the analysis of zeros in the diffraction pattern ([20.16] and earlier work). It was shown, for example, that the set of all bandlimited functions having a given set of real zero crossings is convex, and that, in the absence of noise, the data might be factorizable. By 1982, a useful working solution had been obtained [20.17] for real 2-D images, by the addition of a feedback feature to the G–S error-reduction (ER ) algorithm. An important realization at about this time was that the landscape (a map of error metric in the \(N\)-dimensional search space) for the phase problem was not rugged and usually consisted of a single shallow global minimum with a few small bumps—the essential difficulty is not caused by the number of local false minima, but rather by the large number of directions in which search is possible, in a space containing one dimension for each image pixel. This Fienup algorithm has come to be known as the hybrid input–output or HIO algorithm . Feedback greatly speeds up convergence and improves the ability of the algorithm to escape from local minima, however most modern work is based on a combination of the ER and HIO algorithms, since only the ER algorithm supplies a reliable error metric. Fienup showed that the ER algorithm converges, in the sense that the error monotonically decreases [20.18]. Real images could then be reconstructed from the Fraunhofer diffraction pattern intensity data, provided that the sign of the scattering potential was known together with an approximate estimate of the object boundary (support). Important work on the uniqueness problem for the HIO algorithm was published by Bruck and Sodin [20.19] and Barakat and Newsam [20.20], who showed that ambiguous solutions are obtained with these constraints only on pathologically rare occasions. The aim of this work is to show that only one image function satisfies the known sign, support, and Fourier modulus constraints. The HIO algorithm, which we give below, is the basis for much successful modern work in CDI and has led to the development of many variants and much detailed mathematical analysis. The book by Stark  [20.21] then provided an excellent overview of the subject at that time and can be strongly recommended to students, together with the important review articles by Millane  [20.8, 20.9] and Marchesini  [20.22], and the very detailed description of the application of these methods to real experimental data given in Chapman et al [20.4]. Millane [20.8] unified the crystallographic and single-particle approaches and so brought the field to the attention of a much larger audience. (The resemblance of the HIO algorithm to the independently developed solvent flattening methods of x-ray crystallography can also be traced through Wang [20.23].) Here, the zero-density region outside the support of our single particles plays the same role as the water jacket around a protein in a crystal [20.11].

The HIO algorithm and its variants have been highly successful for real objects, however rather less attention has been paid to the more difficult problem of complex image reconstruction where the sample is defined by a complex number (such as a complex refractive index) at each pixel or voxel. Except for special cases, it is then found that a very precise knowledge of the support is needed, and that, in one and two dimensions this support should be disjoint (separated into two parts) [20.24]. Iterative inversion schemes fail for 1-D real data which is not disjoint and work better in higher dimensions.

The third more recent period of work since about 1990, during which the field grew rapidly, is discussed more quantitatively in the next sections. During that time a major effort was made to apply these methods to soft x-ray transmission data at the Brookhaven synchrotron [20.25, 20.26]. Our understanding of the success of the HIO algorithm has increased considerably in that period due to the contribution of mathematicians using powerful methods based on convex set theory. Bregman projections and constraint theory were first described as early as 1982 [20.27] but not taken up until recently (an excellent review is given in [20.28]). Questions of uniqueness in dimensions greater than two have been considered, where the problem is overdetermined [20.29], and it is generally found that the iterative algorithms work better. In addition, a number of developments of the HIO algorithm have been published [20.30, 20.31], the standard symbols used by mathematicians for the theory of projections has been adopted to describe the problem, and a new algorithm, which dispenses with any need for knowledge of the object support (the shrinkwrap algorithm [20.32, 20.33]), has been published. The problem of lost information within the synchrotron beam stop has been addressed by reconstructing the diffuse scattering around a Bragg reflection from a nanocrystal [20.34]. In 2001, the first of a bi-annual series of conferences on the noncrystallographic phase problem was held at Lawrence Berkeley Laboratory [20.35]; they have continued since, with published proceedings.

These theoretical developments, often based on computer simulations, have been supported by far fewer demonstrations of inversions from experimental data. Only experimental results can truly give confidence in theory, and early results were somewhat disappointing. An early application of the HIO algorithm to experimental optical speckle data can be found in [20.36], and applications to images formed with laser light can be found in Spence et al [20.37], with emphasis on the depth of focus problem. In the x-ray imaging community, the work at Brookhaven in Janos Kirz's group finally paid off in 1999, when images of lithographed characters were reconstructed from (soft) x-ray diffraction patterns for the first time [20.38]. A low-resolution optical image was used to provide the support. First results of lensless imaging applying the HIO algorithm to electron diffraction data are reported in Weierstall et al [20.39]. More recently, an atomic-resolution image of a single double-walled carbon nanotube was reconstructed from its electron microdiffraction pattern [20.3]. These successes have led to rapid growth of the subject in the years since.

20.2 The Projection Approximation, Multiple Scattering, Objects, and Images

The theory of lensless CDI can be simply understood initially in two dimensions, in the projection approximation, where we assume that the wavelength of the radiation is sufficiently small (or the sample thickness sufficiently small) that the Ewald sphere can be taken to be planar from the origin of reciprocal space out to the highest resolution of interest. Then the scattering on this plane represents a projection of the object density, or of a similar property responsible for scattering. The terms object, image, diffraction pattern, exit-face wavefunction , real and complex object, and transmission function have sometimes been confused in the literature and are defined here for clarity. We define the object by its ground state charge density \(\rho(\boldsymbol{R})\) (which diffracts x-rays) and by the corresponding electrostatic potential \(V(\boldsymbol{R})\) (which diffracts electrons); \(V(\boldsymbol{R})\) is related to the density \(\rho(\boldsymbol{R})\) by Poisson's equation. As a result of inelastic processes, both may be complex and could be referred to generally as the complex optical potential , but in this chapter unless otherwise stated, we take them to be real. Here, \(\boldsymbol{R}\) is a 3-D vector, while \(\boldsymbol{r}\) will be 2-D. We define the exit-face wavefunction \(\Uppsi(\boldsymbol{r})\) across the downstream face of a thin slab of sample in the transmission diffraction geometry with transmission function \(\mathrm{T}(\boldsymbol{r})\) by
$$\Uppsi(\boldsymbol{r})=\mathrm{T}(\boldsymbol{r})\Uppsi_{0}(\boldsymbol{r})\;,$$
where \(\Uppsi_{0}(\boldsymbol{r})\) is the wavefield incident on the sample. (For perfectly coherent radiation from a point source, collimated by a lens, \(\Uppsi_{0}(\boldsymbol{r})\) is approximately a plane wave.) Transmission functions for x-ray and electron diffraction are derived below in terms of the wanted object properties \(\rho(\boldsymbol{R})\) or \(V(\boldsymbol{R})\). We define the image as any magnified, resolution limited, or aberrated copy of \(\Uppsi(\boldsymbol{r})\), formed by a lens. Unfortunately, this term is now widely used in the CDI literature as a synonym for either object or exit-face wavefunction. This use of image to refer to an exit-face wavefunction is now firmly established and will be continued here, however the more important distinction between object and exit-face wave function is likely to become important in future work and should be preserved. (Further confusion arises when x-ray diffraction patterns are referred to as images, making it impossible to distinguish between real and reciprocal space.) The terms real object and complex object have also been widely adopted in optics but should strictly refer to the nature of the exit-face wavefunction for 2-D imaging. (For a strong phase object described by (20.1) below, the object \(\Updelta n\) may be real but the exit-face wavefunction complex. Most authors would refer to this as a complex object, however the object property we wish to recover is \(\Updelta n(\boldsymbol{r})\).) It is also useful to reserve the word image for real-space functions and diffraction for reciprocal space, so that the term diffraction image, when referring to a diffraction pattern, should be avoided. The measured diffraction pattern intensity is \(I(\boldsymbol{u})=|\Upphi(\boldsymbol{u})|^{2}\), with \(\Upphi(\boldsymbol{u})\) the Fourier transform of \(\Uppsi(\boldsymbol{r})\) and scattering vector \(|\boldsymbol{u}|=\theta/\lambda\) for small scattering angles \(\theta\). (Here \(\lambda\) is the x-ray, or relativistically corrected de-Broglie wavelength for electrons.) In this projection approximation, \(I(\boldsymbol{u})=I(-\boldsymbol{u})\) if \(\Uppsi(\boldsymbol{r})\) is real, or, for complex objects, if \(\Uppsi(\boldsymbol{r})=\psi(-\boldsymbol{r})\); \(\Upphi(\boldsymbol{u})=\Upphi^{*}(-\boldsymbol{u})\) for real \(\Uppsi(\boldsymbol{r})\), while \(\Upphi(\boldsymbol{u})\) is a signed real quantity if \(\Uppsi(\boldsymbol{r})=\psi(-\boldsymbol{r})\), and \(\Uppsi(\boldsymbol{r})\) is real. The aim of diffractive imaging is to reconstruct the object from the scattered intensity \(I(\boldsymbol{u})\), however, as a first step, the exit-face wavefunction \(\Uppsi(\boldsymbol{r})\), which is simply related to \(\Upphi(\boldsymbol{u})\) by a Fourier transform, is obtained. (This reduces to determination of the sign of \(\Upphi(\boldsymbol{u})\) if \(\Uppsi(\boldsymbol{r})=\Uppsi(-\boldsymbol{r})\) and \(\Uppsi(\boldsymbol{r})\) is real.) The further recovery of object properties \(\rho(\boldsymbol{R})\) or \(V(\boldsymbol{R})\) from \(\Uppsi(\boldsymbol{r})\) may only be possible in the absence of multiple scattering or inelastic scattering. For electron diffraction, the weak-phase approximation generates a real object in the language of optics and diffractive imaging, as described below. For hard x-ray diffraction (or soft x-ray scattering from thin samples), where single-scattering conditions are common, the absence of spatially-dependent absorption (due to the photoelectric effect) provides such a real object. A single, spatially uniform absorptive process, however, may allow the phase contrast formulation to be used. For an experimental analysis of complex images in terms of composition variations, see Shapiro et al [20.5]. The introduction of the transmission function allows a simple extension to the case of coherent convergent-beam illumination and related methods for phasing [20.40].

We now relate \(\Uppsi(\boldsymbol{r})\) to the wanted object properties \(\rho(\boldsymbol{R})\) or \(V(\boldsymbol{R})\) for the case of visible light, electron beams, and x-rays in this projection, iconal, or flat Ewald sphere approximation. Both refractive and dissipative (inelastic) processes may occur. In each case, one must consider whether an iconal or projection approximation may be made, and the question of whether 3-D(tomographic) information (discussed later) may be extracted.

The simplest case for each radiation is that in which the image \(\Uppsi(\boldsymbol{r})\) may be treated as a simple projection of some property of the sample, taken in the beam direction. Then, if a transmission sample in the form of a thin plate of thickness \(t\) is illuminated by a plane wave,
$$\begin{aligned}\displaystyle\Uppsi(\boldsymbol{r})&\displaystyle=\mathrm{T}(\boldsymbol{r})\Uppsi_{0}(\boldsymbol{r})\\ \displaystyle&\displaystyle=\exp\left(\frac{-2\uppi\mathrm{i}\Updelta n_{\mathrm{p}}(\boldsymbol{r})}{\lambda}\right)\exp(-2\uppi\mathrm{i}\boldsymbol{u}_{0}\,\boldsymbol{r})\;.\end{aligned}$$
(20.1)
For normal-incidence plane-wave illumination \(\boldsymbol{u}_{0}=0\), and we may set \(\Uppsi_{0}(\boldsymbol{r})=1\), and \(\Uppsi(\boldsymbol{r})=\exp[-\mathrm{i}\theta(\boldsymbol{r})]\). Here, \(\Updelta n_{\mathrm{p}}\) is proportional to the complex refractive index of the sample for the radiation concerned. We see that a real (mask-like) object can only be obtained if the real part of \(\Updelta n_{\mathrm{p}}\) is independent of \(\boldsymbol{r}\), and all structural information is contained in the imaginary part. These experimental conditions (pure absorption contrast) can only be obtained at relatively low spatial resolution (under incoherent conditions) for both electrons and x-rays.
For x-rays,
$$\Updelta n_{\mathrm{p}}(\boldsymbol{r})=\int^{t}_{0}[\delta(\boldsymbol{R})-\mathrm{i}\beta(\boldsymbol{R})]\mathrm{d}z\;,$$
(20.2)
where \(\delta\) is a positive quantity [20.41]. In terms of mean values, the complex index of refraction for x-rays is \(n=1-\Updelta n\), \(=(1-\delta)+\mathrm{i}\beta\), where \(\delta\) describes refraction and \(\beta\) absorption (mainly the photoelectric effect, arising from absorption edges). The linear absorption coefficient is \(\mu=4\uppi\beta/\lambda\). The dependence of \(\Updelta n(\boldsymbol{r})\) on the real and imaginary parts of the atomic scattering factors \(f\) and \(f^{\prime}\) is given by \(\delta=(r_{\mathrm{e}}\lambda^{2}/(2\uppi))n_{\mathrm{a}}\,f\), and \(\beta=(r_{\mathrm{e}}\lambda^{2}/(2\uppi))n_{\mathrm{a}}\,f^{\prime}\), with \(n_{\mathrm{a}}\) atoms per unit volume and \(r_{\mathrm{e}}\) the classical electron radius. Away from absorption edges, the electronic charge density (excluding the nuclear contribution) is
$$\rho(\boldsymbol{R})=\left(\frac{2\uppi}{r_{\mathrm{e}}\lambda^{2}}\right)\delta(\boldsymbol{R})\;.$$
(20.3)
If small bonding effects are ignored, \(\rho(\boldsymbol{R})\) is obtainable from tabulated x-ray scattering factors for neutral atoms.

Since \(\delta\) is about \(\mathrm{10^{-3}}\) at \({\mathrm{6}}\,{\mathrm{kV}}\) for light materials, a thickness of about \({\mathrm{0.3}}\,{\mathrm{\upmu{}m}}\) of sapphire is needed to obtain a phase shift of \(\uppi/2\), allowing a first-order expansion of (20.1). Then, the diffracted amplitudes are simply proportional to the Fourier transform of the projected charge density of the object.

For an electron beam of kinetic energy \(|e|\,V_{0}\),
$$\Updelta n_{\mathrm{p}}(\boldsymbol{r})=\int^{t}_{0}\frac{V_{\mathrm{c}}(\boldsymbol{R})}{2V_{0}}\mathrm{d}z\;,$$
(20.4)
where \(V_{\mathrm{c}}(\boldsymbol{R})\) is the complex optical potential for high-energy electrons [20.42, 20.43], and the mean refractive index for electrons is \(n=1+\Updelta n\). The real part of this, for high beam energies, is the positive electrostatic or Coulomb potential, also obtainable from x-ray scattering factors using Poisson's equation [20.44]. The imaginary part (typically about a tenth of the real part) accounts for depletion of the elastic wavefield by inelastic scattering events such as plasmon, inner-shell, and phonon scattering. The average value \(V_{0}\) of \(\mathfrak{Re}\{V_{\mathrm{c}}\}\) is about \({\mathrm{12}}\,{\mathrm{eV}}\) for light materials, and is positive, so this may be used as a constraint. Both real and imaginary parts of the potential \(V_{\mathrm{c}}(\boldsymbol{R})\) are positive if the mean inner potential is included, since electron beams are attracted predominantly to the positive nuclei. Electron diffraction in the transmission geometry is not, however, sensitive to the mean potential \(V_{0}\), which only produces a constant phase shift. The mean inner potential can only be meaningfully defined for a finite object with zero total charge [20.45]. Although the potential due to an (unphysical) isolated negative ion may have small negative excursions, the mean value depends on the volume assumed, and, in a real crystal, any long-range ionic potential is also screened. In summary, in the absence of its mean value, the real part of the optical potential used to describe electron diffraction (when synthesized from diffraction data, with resolution limited by a temperature factor) is positive, except for possible very small negative excursions around negative ions. The positivity of the imaginary part is guaranteed by the requirement that energy gain is forbidden in inelastic processes if very small virtual processes are ignored. Fourier coefficients of the total optical potential may have either sign, and those of the real (elastic) and imaginary (inelastic) potential become mixed in objects without inversion symmetry. Hence, a sign condition on the optical potential may be used as a convex constraint unless atomic resolution reconstructions are attempted of sufficiently high accuracy to detect the very weak bonding effects.

By estimating the maximum value of \((\mathfrak{Re}\{V_{\mathrm{c}}\}-V_{0})\), the maximum thickness can be estimated for which a first-order expansion of (20.1) may be made, the weak-phase approximation, in which \(\theta(\boldsymbol{r})\ll\uppi/2\). This is also a limited case of the single-scattering approximation . For x-rays, the maximum thickness allowable in the weak-phase object approximation can be estimated using the Center for X-ray Optics (CXRO) web page (x-ray database) supported by the Lawrence Berkeley Laboratory [20.46]. For electrons, the Fourier coefficients of electrostatic potential published for many materials by Radi may be useful [20.42], since these also provide an estimate of the imaginary part of the optical potential for electron diffraction due to inelastic scattering.

For visible light, \(\Updelta n_{\mathrm{p}}(\boldsymbol{r})\) has a similar interpretation as for x-rays (with \(n\) given by the square root of the complex dielectric constant), however \(n> {\mathrm{1}}\). Thus, visible light and electrons (\(n> {\mathrm{1}}\)) are bent toward the normal on entering a denser medium, whereas x-rays are bent away and undergo total external reflection, with \(n<{\mathrm{1}}\).

In summary, if inelastic processes are neglected (away from absorption edges), at high energies, transmission samples of thickness \(t\) are phase objects for electrons and x-rays, for which the refractive index is proportional to the electron density for x-rays and to the total electrostatic potential, including the nuclear contribution, for electrons. Poisson's equation relates these. The magnitudes of these quantities are such that a first-order expansion of (20.1) (weak phase object approximation) is justified (\(2\,\uppi\,\mathrm{i}\,n_{\mathrm{p}}(\boldsymbol{r})/\lambda<\uppi/2\)) if \(t<{\mathrm{20}}\,{\mathrm{nm}}\) for electrons (light elements, \(V_{0}={\mathrm{200}}\,{\mathrm{kV}}\)) but \(t<{\mathrm{0.3}}\,{\mathrm{\upmu{}m}}\) for x-rays (light inorganic material, \({\mathrm{6}}\,{\mathrm{kV}}\)). Higher-order terms in the expansion of (20.1) correspond to the multiple-scattering terms of the Born series in a flat Ewald sphere approximation. Elastic backscattering of x-rays and polarization effects are neglected in these approximations.

This flat Ewald sphere or projection approximation, on which (20.1) is based, depends on the ratio of the wavelength to the smallest detail \(d\) of interest (with spatial frequency \(u=1/d\)), and on the thickness of the sample. Scattering kinematics restrict elastic scattering to regions of reciprocal space near the energy and momentum-conserving Ewald sphere of radius \(1/\lambda\). The projection approximation holds if the excitation error distance \(S_{u}\approx\lambda u^{2}/2\) from this sphere onto a plane in reciprocal space normal to the beam (passing through the origin) is less than either \(1/t\) or \(1/\xi_{u}\), whichever is the smallest (\(\xi_{u}\) is a multiple-scattering extinction distance for spatial frequency \(u\). Thus samples never look thicker than \(\xi_{u}\), to diffracting radiation). Hence, we require
$$\frac{\lambda u^{2}}{2} <\frac{1}{t}\text{ or }\frac{1}{\xi_{u}}$$
(20.5a)
$$\text{or }\quad\left(\frac{\lambda t}{2}\right)^{1/2} <d$$
(20.5b)
for the validity of (20.1). Since the width of the first Fresnel fringe due to propagation over distance \(t\) is approximately \(w=(\lambda t/2)^{1/2}\), this condition requires that the spreading of the wavefield due to free-space propagation over a distance equal to the thickness of the sample be small, compared to the resolution required. The depth of focus is, therefore, \(t=2d^{2}/\lambda\), with \(d\) the lateral resolution. (Resolution along the beam direction may be better than \(t\) if modeling is used for known structures.) It was recently pointed out that object points which lie outside the depth of focus for certain orientations will lie within it for others, so that 3-D reconstruction can provide more favorable results than these 2-D predictions [20.47].

The failure of (20.1) may require either single or multiple-scattering treatments, depending on the strength of the interaction and the sample thickness. We note that (20.1) is an exact solution which sums the Born series, including all multiple scattering effects, in the limit of the vanishing wavelength. For x-ray tomography , use of (20.1) (the projection approximation) greatly simplifies the merging of data for 3-D reconstruction, since it allows the 2-D Fourier transform of the phased diffraction pattern to be treated as a projection of the sample density. This is an important consideration in soft x-ray zone-plate microscopy and in cryoelectron microscopy ( ), where the resolution, now at best about \({\mathrm{0.2}}\,{\mathrm{nm}}\) with the new single-electron detectors, is approaching the limits set by (20.5b) [20.48], on which the merging of data from different projections depends.

The preceding discussion concerned 2-D imaging. For tomography and XFEL 3-D single-particle imaging (described below in more detail), a different analysis is required. Again, a single-scattering approximation must be used. But the introduction of a curved Ewald sphere does not prevent 3-D reconstruction, even though the transform of one phased 2-D diffraction pattern can no longer be treated as a real-space projection. The x-ray scattering theory from a general 3-D nonperiodic compact sample can be applied [20.49], and this scattering may be collected for various object orientations. For each orientation, the scattering is assigned to points on the Ewald sphere in a 3-D reciprocal space, until all of the reciprocal space volume is filled. Experimentally, in CDI this is done by rotating the sample about an axis normal to the beam and recording a diffraction pattern at each orientation. For the XFEL, where each shot destroys the sample and a goniometer is not used, a determination of the sample orientation from its scattering must be made, as is described later. Planar slices taken through the origin of the filled reciprocal space then transform to projections. For microcrystalline samples, indexing of the Bragg beams provides the orientations of the randomly-oriented samples, allowing them to be merged. Use of the Fienup algorithm with 3-D Fourier transform iterations can then take advantage of the improved convergence in three dimensions. A sign constraint may or may not be applied to both real and imaginary parts of the scattering potential, and a support, enclosing the object, must be assumed.

20.3 The HIO Algorithm and Its Variants, Uniqueness, and Constraint Ratio

We can understand the iterative phasing methods by first considering the Fourier equations relating the object to its diffraction pattern. We reverse the domains originally considered by Shannon and treat the object space as if it applies a bandlimit to the diffraction pattern (the object is assumed compact). Shannon's theorem then specifies either the sampling interval on the continuous diffraction pattern intensity needed to fully reconstruct the autocorrelation function of the object, or the sampling of the complex scattered amplitude needed to reconstruct the object. Consider a simple 1-D complex exit-face wavefunction \(f(x)\), which is nonzero only for \(0<x<W\), so the support has width \(W\). We first treat this function as the bandlimit on the complex diffraction pattern \(F(u)\), which must, therefore, be sampled at intervals \(u_{n}=n/W\) by the detector to satisfy Shannon's theorem. We then have the \(N=W/d\) equations, for resolution \(d\) (one for each pixel in the detector)
$$\begin{aligned}\displaystyle|F(u_{n})|&\displaystyle=\left|\sum^{N}_{j=1}f(x_{j})\exp\left(\frac{2\uppi\mathrm{i}nx_{j}}{W}\right)\right|\\ \displaystyle&\displaystyle\qquad n=1,\dots,N\;,\end{aligned}$$
relating measured intensities \(|F|\) to the \(2N\) unknown components of the complex values of \(f(x_{j})\) we seek. Since there are more unknowns than equations, the complex values of \(f(x)\) cannot be found from these measurements.

However, now consider the same function placed within a domain of width \(2W\), so that the values of \(f(x)\) in \(W<x<2W\) are known a priori to be zero. The sampling interval on \(F(u)\) is now \(1/(2W)\), and we thus have \(2N\) equations, but the number of unknown values of \(f(x)\) remains at \(2N\), so that the system of equations now becomes solvable in principle. Loosely speaking, we compensate for the missing half of the data in the diffraction domain (the phases) by requiring that half of the object values be known (they are zero outside the support of width \(W\)). Note that \(2W\) is the width of the autocorrelation function obtained from \(f(x)\), and that the Fourier transform of the diffracted intensities gives this function. Thus, the Shannon sampling of the diffraction pattern \(1/(2W)\) needed to solve the phase problem could be taken to be that which results from treating the autocorrelation function as a bandlimit on the measured quantities, which are the scattered intensities. Since the HIO algorithm assigns a random set of phases initially, the equations are also linear within the algorithm unless a sign constraint is applied. Experimentally, the effects of noise must be considered. Symmetrical support shapes should be avoided, since they may lead to dependent equations. Large systems of nonlinear equations are normally extremely difficult or impossible to solve, so it is remarkable that the HIO algorithm does this so effectively, for reasons which are not fully understood, since few methods exist for analyzing nonconvex optimization. However, although the Fourier modulus constraint is nonconvex, it is known that the error surface in hyperspace is not a rugged landscape, but consists of a single rather smooth minimum with a few small bumps. Use of a boundary estimate (support) without symmetry also drives the algorithm toward one particular enantiomorph \(\rho(\boldsymbol{r})\) rather than a spurious mixture involving \(\rho(-\boldsymbol{r})\).

This example may be extended to other cases: for complex, two-dimensional objects one has \(2N^{2}\) unknowns but \(N^{2}\) equations (if the charge-coupled device (CCD ) has linear pixel dimension \(N\)). But by placing empty space around our compact object, which increases the diffracting volume by \(2^{1/2}\) in both dimensions, we sample the diffraction pattern more finely and recover the \(2N^{2}\) equations needed to solve the phase problem. It is the knowledge of the support (the object boundary) which ensures that known object pixel values may be inserted in the algorithm correctly outside the support. Experimentally, this creates the greatest difficulty of the diffractive imaging method—one must know a priori that the diffraction pattern comes from an isolated object whose size is approximately known. The existence of any material outside the assumed support which unwittingly contributes to the diffraction pattern will result in inconsistent constraints being applied by the algorithm, which will then not converge. Further enlargement of the space around the sample does not contribute independent equations, since sampling finer than the Shannon interval does not add information. The introduction of Friedel symmetry for a real object reduces the number of unknowns by two in both domains [20.26] and makes the method much less sensitive to the initial choice of outer computational boundary.

These considerations establish that we may have as many equations as unknown phases, but not know how to solve them. The modulus operation makes them nonlinear, they may not be independent, and there may be many equivalent solutions, differing perhaps trivially by inversion or origin shifts. The two solutions made possible by the modulus operation would be reduced to one if the sign of the density \(\rho(x)\) were known. The most successful approaches to solving these coupled nonlinear equations were based on iterative optimization, and the first of these to show widespread success was the hybrid input–output method of Fienup [20.17], later modified to allow incorporation of known symmetries [20.50] and further developed as discussed below.

Before describing that method, we introduce the powerful constraint ratio concept \(\Omega\), which indicates immediately whether our equations are invertible and/or unique [20.51]. This is defined by
$$\Omega=\frac{A}{2U}\;,$$
(20.6)
where \(A\) is the volume of the autocorrelation function of the sample (e. g., a virus) and \(U\) is the volume of the sample; \(\Omega\), therefore, depends on the shape of the sample. (The autocorrelation function of a disk, for example, is a disk of twice the diameter.) It may be shown that \(\Omega\) is equal to the ratio of the number of sampled diffraction measurements (bandlimited by \(\Omega\) at the Shannon interval and limited by resolution) to the number of independent sample density parameters (the complex voxels of the density we wish to find). This is consistent with (20.6), since these numbers are proportional to the sampled areas or volumes. A unique solution to the phase problem requires \(\Omega> {\mathrm{1}}\), significantly so in the presence of noise. For convex supports in three dimensions, \(\Omega={\mathrm{4}}\). These ideas are applied to the question of uniqueness for iterative phasing methods in crystallography in Millane and Arnal [20.52].
We now return to the projection approximation and also assume the weak-phase approximation without multiple scattering or inelastic processes, other than an overall, spatially-independent exponential attenuation with thickness. The recorded intensity of the diffraction pattern, excluding the central portion, is then given as
$$I(\boldsymbol{u})=\phi(\boldsymbol{u})\phi(\boldsymbol{u})^{*}=[\mathfrak{F}\{\psi(\boldsymbol{r})\}]^{2}\;.$$
(20.7)
If the phase of the Fourier transform \(\mathfrak{F}\{\psi(\boldsymbol{r})\}\) could be recovered, then the aberration-free complex exit wave \(\psi(\boldsymbol{r})\) could be reconstructed. Aberrations of any diffraction lenses in the electron case which magnify the diffraction pattern (astigmatism, distortion, etc.) could alter the intensity distribution \(I(\boldsymbol{u})\) and, so, complicate recovery.

For nonperiodic objects, the phase problem can now be solved if the object has a finite support i. e., the image is nonzero only within a finite region of image space, and if the image is real and nonnegative. For a weak phase object, the exit-face wavefunction is \(\psi(\boldsymbol{r})\approx 1-\mathrm{i}\uptheta(\boldsymbol{r})\), where \(\theta(\boldsymbol{r})\) may be treated as real and positive. The diffracted amplitude is \(\psi(\boldsymbol{u})=\delta(\boldsymbol{u})-i\phi(\boldsymbol{u})\), for which the scattered part differs in phase by \(90^{\circ}\) from the direct beam given by the first term. This direct unscattered beam is absorbed by the beamstop—difficulties arising from the finite size of this are discussed later, and we note that the average value of \(\theta(\boldsymbol{r})\) is also lost within the beam stop. Including a larger area around the object which is known a priori to have a zero or unit transmission function is equivalent to oversampling in Fourier space, and the spatial coherence of the illumination must span this larger area, at least equal to that of the autocorrelation function of the object. Hence, diffractive imaging requires a spatial coherence width approximately twice that of coherent imaging with a lens [20.53], increasing the exposure time by a factor of 4 for a given source and resolution limit.

We assume that \(|\mathfrak{F}\{\psi(x,y)\}|\) has been measured, and that the support \(S(x,y)\) of \(\theta(\boldsymbol{r})\) (which is the same as that of the object) is known; \(S(x,y)\) is the region outside of which the object density is known to be zero. The iterations start with an initial estimate \(\tilde{G}_{1}(u,v)=|\mathfrak{F}\{\psi(x,y)\}|\exp[\mathrm{i}\theta_{1}(u,v)]\) of the spectrum; \(\theta_{1}(u,v)\) is chosen to be an array of independent pseudo-random real numbers distributed between 0 and \(2\uppi\). The iterative Fourier-transform algorithm consists of the following steps (with subscript \(k\) labeling quantities at the \(k\)-th iteration):
  1. 1.

    Inverse Fourier transform \(\tilde{G}_{k}(u,v)\) to obtain the image \(\tilde{g}_{k}(x,y)\)

     
  2. 2.
    Define \(g_{k+1}(x,y)\) as
    $$\begin{aligned}\displaystyle&\displaystyle g_{k+1}(x,y)\\ \displaystyle&\displaystyle=\begin{cases}\tilde{g}_{k}(x,y)\;,&\text{if }(x,y)\in S(x,y)\;,\\ g_{k}(x,y)-\beta\tilde{g}_{k}(x,y)\;,&\text{if }(x,y)\notin S(x,y)\;.\end{cases}\end{aligned}$$
    (20.8)
    This constitutes the hybrid input–output (HIO ) version of the algorithm; \(\beta\) is a constant chosen between \(\mathrm{0.5}\) and \(\mathrm{1}\). In the error-reduction (ER) version of the algorithm, this step is replaced by
    $$g_{k+1}(x,y)=\begin{cases}\tilde{g}_{k}(x,y)\;,&\text{if }(x,y)\in S(x,y)\;,\\ 0\;,&\text{if }(x,y)\notin S(x,y)\end{cases}$$
    (20.9)
     
  3. 3.

    Fourier transform \(g_{k+1}(x,y)\) to obtain \(G_{k+1}(u,v)\)

     
  4. 4.
    Define a new Fourier domain function \(\tilde{G}_{k+1}(u,v)\) using the known Fourier modulus \(|\mathfrak{F}\{\psi(x,y)\}|\) with the computed phase
    $$\tilde{G}_{k+1}(u,v)=|\mathfrak{F}\{\psi(x,y)\}|\exp[\mathrm{i}\theta_{k+1}(u,v)]$$
     
  5. 5.

    Go to Step 1 with \(k\) replaced by (\(k+1\)).

     
To monitor the progress of the algorithm, the object space error metric \(\varepsilon_{k}\) is calculated during each iteration
$$\varepsilon_{k}=\frac{\sum_{(x,y)\notin S}|\tilde{g}_{k}(x,y)|^{2}}{\sum_{(x,y)}|\tilde{g}_{k}(x,y)|^{2}}\;,$$
(20.10)
\(\varepsilon_{k}\) is the amount by which the reconstructed image violates the image-space constraints. Physically, in the x-ray case, it is the normalized amount of charge which remains outside the boundary of the object and which should be zero. In all our calculations we used \(\beta={\mathrm{0.7}}\) and a combination of the ER and HIO algorithms, with \(\mathrm{20}\) ER iterations followed by \(\mathrm{50}\) HIO cycles, all repeated until the error \(\varepsilon_{k}\) drops below a certain level. Simulations based on the above procedure with small noise levels invariably converge to the correct solution [20.37, 20.39]. Since \(g(x,y)\), \(g^{*}(-x-a_{1},-y-a_{2})\exp(\mathrm{i}\theta)\), and \(g(x-a_{1},y-a_{2})\exp(\mathrm{i}\theta)\) all have the same Fourier modulus, they cannot be distinguished by the algorithm. Each run of the algorithm started with different random phases may thus produce images centered on different origins or related by inversion symmetry. We call these equivalent images. (Similarly, in three dimensions, enantiomorphs cannot be distinguished without prior information such as a support function which favors one hand.)

When experimental data are analyzed, the physical support in the object may not be known accurately. To avoid confusion we call the support actually present in the experiment the physical support and the support estimate used in the data analysis the computational support. The term loose support describes the situation where the computational support is bigger than the physical support. Tight support means the physical support is the same as the experimental support. For 2-D simulations with real objects, it is found that a default triangular-shaped support may invariably be used, if it is chosen to lack any symmetry and enclose the object. (A convex centrosymmetric shape such as a circle or sphere is the worst choice, and an irregular shape may bias the reconstruction toward one of two enantiomorphs.) With experimental data, convergence may be slower (or nonexistent) due to high noise levels, the absence of data around the origin at the beamstop, and an excessively loose support. Then the simplest initial choice of support is the boundary of the autocorrelation function (obtained by Fourier transform of the diffracted intensity). This estimate is rapidly improved upon by the shrinkwrap algorithm , which uses a specified intensity threshold to find an improved smaller boundary at every iteration of the HIO algorithm [20.32, 20.33]. This has become one of the most useful practical algorithms at present. Depending on noise levels, the support estimate has been found to be sufficiently accurate to deal with complex objects in many cases. Another approach to the more difficult case of complex objects has been described by McBride et al [20.54], who find these objects tractable if a difference map is used during iterations, and if additional oversampling is employed.

We conclude this section with some comments on additional constraints, the beam stop problem, support determination, and recent algorithm developments. In the general case of strong multiple scattering, no simple closed-form expression relates the object to the exit-face wavefunction, and the two are related only by symmetry constraints. For a general (strong) phase object (20.1), a unit modulus constraint may be applied, so that reconstructed pixels are forced to lie on a unit circle on an Argand diagram, however, this constraint is nonconvex and so has not been found very useful in practice. For such an object, to which a spatially-independent absorption term is added (so that \(\rho(\boldsymbol{R})\) or \(V_{\mathrm{c}}(\boldsymbol{R})\) have a constant known imaginary part), the reconstructed image pixels may be constrained to lie on a given spiral on an Argand diagram. A summary of the many constraints which have been tried and experimental results is given in [20.39].

Over the past decades, new convex constraints have been devised, such as symmetry (useful mainly for reducing computing time), positivity, atomicity , compact support, and a known histogram of gray levels for the density (as for proteins). A knowledge of phase is a convex constraint. Nonconvex constraints include the given Fourier modulus and the sign constraint. Low-resolution imaging by a different technique (e. g., (small-angle x-ray scattering) for x-rays, or (scanning electron microscopy) for electrons) may be used to provide a support estimate. In the following section, the use of prepared objects of known shape is demonstrated to allow the method of Fourier transform holography to be used to provide a support estimate. The reference object need not be a simple point scatterer [20.55] and may have an extended complicated known shape [20.56]. The shrinkwrap algorithm described below rapidly improves iteratively on any initial support estimate and is found to converge for both real and complex objects under a wide range of conditions. For arrays of semiconductor devices, the support will often be known, so that tomographic imaging of defects within an array element might be based on a support provided by the lithography pattern used to make the array. Much of the pattern on a semiconductor chip is periodic, producing low-angle Bragg x-ray diffraction.

charge-flipping algorithm has been described [20.57] for the crystallographic phase problem which may also be adapted for nonperiodic objects, in which case it reduces to Fienup's output–output algorithm with feedback parameter \(\beta={\mathrm{2}}\) and a dynamic support defined by an adjustable threshold [20.58]. This algorithm operates as follows: first, random phases are assigned to the structure factors, which must extend to atomic resolution. These are transformed to yield a real charge density. All densities values below a certain threshold have their sign reversed. The result is transformed, and Fourier magnitudes replaced with measured values. The process is continued to convergence. This algorithm (far simpler than the direct methods normally used in crystallography) has been used to solve new crystal structures from x-ray data [20.59] and is found to perform very well. It does not require a support estimate or knowledge of atomic scattering factors, but does require atomic-resolution data. The atomicity constraint in crystallography assumes that the solution density consists of a set of smooth peaks, and requires that diffraction data extend to atomic resolution. The support then consists of spheres around each atom—most of the density within a crystal consists of empty space between atoms, akin to the zero-density band generated by oversampling around an object in the HIO algorithm. Since all matter in the cold universe consists of atoms (whose scattering factors are known), this constraint provides an extremely powerful ab-initio assumption if one has atomic-resolution data and is the basis of the direct methods algorithms of x-ray crystallography. Other known building blocks (such as gold balls or lithographed dots) have been used to reduce the number of unknown parameters in image definition [20.55]. (For proteins, for example, both the sequence and the atomic structure of the 20 amino acids of which they consist are usually known a priori, together with a typical gray-level histogram for the density maps and the torsion angles between amino acids, together with interatomic bond lengths.) An algorithm which has used the atomicity idea for nonperiodic data is SPEDEN [20.60], which has been applied to the data of Fig. 20.3.

The HIO algorithm has also been used for 2-D protein crystals in electron microscopy. This technique uses Fourier transforms of conventional weak-phase-object images (formed with a magnetic lens) to provide the phases of low-order structure factors, and the corresponding Bragg diffraction patterns for their magnitudes. The subnanometer resolution images must be obtained over as large a range of tilts as possible, creating serious experimental difficulty. These 2-D monolayer crystals are nonperiodic in the direction normal to the plane of the crystal, so that lines of diffraction are generated in reciprocal space normal to the sample. By oversampling along these lines it is possible to phase these reciprocal lattice rods individually. The phase relation linking them can then be obtained from a few high-resolution images recorded at small tilts. In this way, the number of electron microscope images needed for 3-D imaging of 2-D organic crystals can be greatly reduced, with most of the reconstruction based on easy-to-obtain diffraction data [20.61].

We note in passing that the nonuniqueness of the crystallographic phase problem has been well studied—the so-called homometric crystal structures studied by Pauling, Burger, and others have the same diffraction patterns, but different structures (they are not enantiomorphs). Fortunately these are very rare.

Several approaches have been made to the problem of data lost behind a synchrotron beamstop, which is essential to protect a sensitive area detector. Any additional blooming can mean much loss of low-frequency data. In several HIO applications, these missing values have simply been treated as free adjustable parameters, and the algorithm was found to converge. Calibrated absorption filters have been placed in front of the inner portion of the detector. Another solution is to use a sample consisting of an unknown object filling a small hole in an otherwise opaque mask [20.39, 20.62]. Stray scattering of x-rays from the edges of the holes in the mask can be a difficulty, however surface roughness which is smaller than the wavelength of the x-rays produces little scattering. The use of very small silicon nitride windows (e. g., of \({\mathrm{2}}\,{\mathrm{\upmu{}m}}\) width) greatly reduces the intensity of the direct beam and blooming effects, however, the detailed shape of the partially transparent silicon wedge around the window must then be modeled and used as a support for inversion. (When making samples of small particles deposited from solution, it is found most efficient to use a focused ion beam to remove all but a favorably located and isolated particle in the center of the window.) Finally, the diffuse x-ray scattering around Bragg peaks (which forms the shape transform of the object) from a crystallite has been inverted to an image, thus avoiding the direct-beam scattering [20.34, 20.63].

20.4 Experimental Results

Figure 20.1 shows the first x-ray images reconstructed by this lensless method in 1999 [20.38]. The test object consists of letters formed from gold dots, \({\mathrm{100}}\,{\mathrm{nm}}\) in diameter and \({\mathrm{80}}\,{\mathrm{nm}}\) thick, on a transparent silicon nitride membrane. The transmission x-ray diffraction pattern formed with \({\mathrm{1.7}}\,{\mathrm{nm}}\) monochromatic soft x-rays is shown in Fig. 20.1a, and the reconstructed image in Fig. 20.1b. The object was illuminated through a coherently filled \({\mathrm{10}}\,{\mathrm{\upmu{}m}}\) diameter pinhole, and a \({\mathrm{25}}\,{\mathrm{cm}}\) camera length was used. Missing data from the central region within the beamstop was obtained from a lower-resolution optical image. The exposure time was \({\mathrm{15}}\,{\mathrm{min}}\) at the Brookhaven synchrotron. The Fienup algorithm was used for reconstruction, with a sign constraint applied to both real and imaginary parts of the scattering potential. A square support was used (irregular shapes usually work better), and \(\mathrm{1000}\) iterations were needed for convergence. The resolution is about \({\mathrm{75}}\,{\mathrm{nm}}\). A detailed description of an improved version of the apparatus used to obtain this and other recent results at the Advanced Light Source is given in Beetz et al [20.64].

Fig. 20.1

(a) Soft-x-ray transmission diffraction pattern formed with \({\mathrm{1.7}}\,{\mathrm{nm}}\) x-rays from the set of lithographed letters shown in (b). Image recovered from (b) using a modified form of the HIO algorithm. From [20.38]

Figure 20.2a shows a transmission diffraction pattern obtained using \({\mathrm{600}}\,{\mathrm{eV}}\) monochromatic soft x-rays from clusters of gold balls, \({\mathrm{50}}\,{\mathrm{nm}}\) in diameter, lying on a silicon nitride membrane. The silicon nitride is almost transparent to the x-rays, so the object provides a useful test object for reconstruction. The pattern resembles the Airey's disk-like pattern from one ball, crossed by speckle fringes due to interference between different balls. An image reconstruction series using the shrinkwrap algorithm, in which the HIO algorithm refines the support during iterations, is shown in Fig. 20.2b. The top image is the centrosymmetric autocorrelation function, with the support estimate shown as a mask below. This mask was obtained by Fourier transform of the diffraction pattern intensity (to produce the autocorrelation function shown), followed by the selection of a contour corresponding to a certain threshold of intensity. This thresholding operation is repeated after each HIO–ER iteration cycle to generate a new improved estimate of the object support ( [20.32, 20.33] for details). Intermediate iterations lead to the final converged image of the gold ball clusters (Fig. 20.2f). We note that the inversion symmetry necessarily possessed by the autocorrelation function at Fig. 20.2b is lost at Fig. 20.2d as it changes smoothly into the correctly phased image.

Fig. 20.2

(a) Soft x-ray transmission diffraction pattern from clusters of \({\mathrm{50}}\,{\mathrm{nm}}\) gold balls lying on a silicon nitride membrane. The x-ray wavelength is \({\mathrm{2}}\,{\mathrm{nm}}\) (\({\mathrm{600}}\,{\mathrm{eV}}\)). The resolution at the midpoint of the sides corresponds to a spatial periodicity \(u^{-1}={\mathrm{17.4}}\,{\mathrm{nm}}\) or a Rayleigh resolution of \({\mathrm{8.7}}\,{\mathrm{nm}}\). Image reconstruction series using the shrinkwrap algorithm, in which the HIO algorithm refines the support during iterations. (b) The centrosymmetric autocorrelation function, with the support estimate shown as a mask below. Intermediate iterations (ce) lead to final converged image of gold ball clusters in (f). The inversion symmetry is lost in (d)

Figure 20.3 shows an instructive case, indicating the way in which prepared objects may be used to assist reconstruction. (Full experimental details for CXDI are given in He et al [20.65], from which Fig. 20.3 is taken.) Figure 20.3a shows an SEM image of a set of gold balls lying on a silicon nitride membrane. One ball, at A, is isolated. The autocorrelation function obtained from an experimental soft-x-ray transmission diffraction pattern (not shown) taken from this object is given in Fig. 20.3b. This may be interpreted as the self-convolution of the object with its inverse, or, for a collection of point-like objects, as the set of all inter-point vectors. Some inter-ball vectors are shown in Fig. 20.3a and indicated again in Fig. 20.3b. The convolution of the single isolated ball A with the three balls at B produces in the autocorrelation function in Fig. 20.3b a faithful image of the three balls, blurred by the image of one ball. This process is similar to the heavy-atom method of x-ray crystallography, or the method of Fourier transform holography in optics [20.66]. If such a gold ball, or strong point scatterer can be placed near an unknown object, the autocorrelation function will contain a useful first estimate of the desired image of the unknown, which can also be used to provide a support for further HIO iterations aimed at improving resolution. This process is demonstrated in He et al [20.55], where, following the original suggestion of Stroke, it is found that the resolution in the autocorrelation image may be considerably improved beyond the size of the reference ball by simple deconvolution. By using a larger reference object, or one consisting of a cluster of small balls, the intensity of scattering from the reference object can be increased, but resolution is commonly limited by the size of the reference scatterer.

Fig. 20.3

(a) SEM image of several clusters of gold balls, each \({\mathrm{50}}\,{\mathrm{nm}}\) in diameter. Some inter-ball vectors are indicated. The balls lie on an x-ray transparent substrate. (b) The Fourier transform of the x-ray diffraction pattern. This is the autocorrelation function of the density in (a) and is a map of all interatomic vectors or the self-convolution of the object with its inverse. Because the object in (a) includes a single isolated ball at A, the vector AB leads to a faithful image of the triple-ball cluster at B. (The convolution of one ball with three gives a blurred image of three.) From [20.65], with permission from the IUCr. http://journals.iucr.org/

It has been noted that a randomly placed cluster of point scatterers can provide a high resolution image in Fourier transform holography when used as a reference object [20.55, 20.62, 20.66, 20.67]. A uniformly redundant array has been used as a reference object for x-ray Fourier transform holography  [20.68], giving a resolution of about \({\mathrm{50}}\,{\mathrm{nm}}\). The method has also proven powerful, with an array of reference scatterers, for x-ray imaging of magnetic structures [20.69].

The first successful application of CDI to the electron diffraction patterns provided by a TEM is described in Weierstall et al [20.39], where a complete description of the method can be found. An important asset of the TEM is its ability to provide an image of the same region which contributes to the microdiffraction pattern, so that this image can be used to supply the support. The resolution of the best TEM instruments in direct phase-contrast imaging mode using lenses is now about \({\mathrm{1}}\,{\mathrm{\AA{}}}\). Figure 20.4 shows a remarkable application of CDI to an electron diffraction pattern using a TEM [20.3]. This image is the first atomic-resolution CDI image and possibly the first atomic-resolution image of a nanotube. The image gives us the helicity of the tube, its dimensions, and the number of walls. The double-walled nanotube spans a hole in a thin amorphous carbon film, while the electron beam diameter (about \({\mathrm{50}}\,{\mathrm{nm}}\)) is smaller than the hole, so that there is no background contribution from the carbon film. A conventional TEM image was used to provide the support function for HIO iterations along the edges of the tube, and it is suggested that the boundary of the support across the tube is provided by loss of coherence at the edge of the electron probe due to rapid phase variations arising from the aberrations of the probe-forming lens. (In general, a tight support is desirable for CDI.) The image shows higher resolution detail than conventional TEM images of nanotubes. Resolution is limited perhaps only by the temperature factor or by distortions in electron lenses used to magnify the diffraction pattern. It remains to be seen whether tomographic imaging at atomic resolution is simplest by this method or by direct TEM imaging using lenses. If CDI is used, the difficult problem of supporting a nanoparticle for diffraction over a range of orientations will need to be solved. Radiation damage may be reduced in diffraction mode under some conditions. A serious problem with electrons is scattering from any supporting membrane—it is usually found that even the thinnest (e. g., \({\mathrm{10}}\,{\mathrm{nm}}\) thick) carbon film will scatter too strongly to allow inversion by the HIO algorithm in the region around a particle, so that one does not have a compact support constraint. An approach to this problem of substrate background scattering was suggested by Wu et al [20.70], who attempted to reconstruct an atomic-resolution image of a defect in a thin crystal from the transmission electron microdiffraction pattern, which showed both Bragg reflections and diffuse scattering from the defect. The pattern from a nearby crystalline region of the sample was used to provide an independent estimate of the crystalline region, and was subtracted off during the iterations. The defect image was later found not to be a unique reconstruction.

Fig. 20.4

(a) Electron microdiffraction pattern from a single double-walled nanotube. This consists of a rolled-up sheet of graphite. Fine details arise from the helical structure. (b) Experimental image of the double-walled nanotube reconstructed from the electron diffraction pattern. Right: a corresponding model of the structure. From [20.3]. Reprinted with permission from AAAS

Three-dimensional (tomographic) CDI of inorganic samples has been demonstrated in many papers since the first results appeared [20.63, 20.71, 20.72], using soft x-rays at a resolution of about \({\mathrm{10}}\,{\mathrm{nm}}\). More recently, interest has moved to the method of ptychography, since that method does not require an isolated compact sample of approximately known size (support). This approach is described in Chap.  17. The trade-off between the two methods is as follows. Where the support is approximately known, the CDI method (e. g., using HIO for reconstruction) with approximately plane-wave illumination is insensitive to vibration or sample movement during the exposure, since the far-field diffraction pattern alone is detected, and this is independent of transverse sample motion, which, therefore, does not affect resolution. Instrumentation for ptychography is much more complex, and the scattering detected is sensitive to sample motion, as for a conventional optical microscope (but not as severely, for interesting reasons, being a hybrid real and reciprocal-space method [20.73]). This can, therefore, affect resolution. However, ptychography has the great advantage of allowing the use of an extended sample such as a slab of material much wider than the focused beam. This greatly facilitates sample preparation since an invisible substrate (such as graphene or a silicon nitride membrane) is not needed. Recent examples of soft x-ray ptychography (which can often be accommodated on existing scanning transmission x-ray microscopes ( )) at about \({\mathrm{5}}\,{\mathrm{nm}}\) resolution, can be found in Shapiro et al [20.5] and Saliba et al [20.67].

For one experimental CXDI experiment [20.74], the details were as follows. The x-ray source was a synchrotron and undulator, providing coherent radiation at \({\mathrm{600}}\,{\mathrm{eV}}\). A simple zone plate was used as a monochromator, followed by a beam-defining aperture of about \({\mathrm{10}}\,{\mathrm{\upmu{}m}}\) diameter, coherently filled. A nude soft-x-ray CCD camera, employing \({\mathrm{1024}}\times{\mathrm{1024}}\) \({\mathrm{24}}\,{\mathrm{\upmu{}m}}\) pixels was used. The sample was mounted in the center of a silicon-nitride window fitted to a TEM single-tilt holder, which provides automated rotation about a single axis normal to the x-ray beam. The window was rectangular, with the long axis normal to both the beam and the holder axis. Diffraction patterns were recorded at one-degree rotation increments, with a typical recording time of about \({\mathrm{15}}\,{\mathrm{min}}\) per orientation. The maximum tilt angle was then limited by the thickness of the silicon frame around the window to perhaps \(80^{\circ}\), resulting in a missing wedge of data. In addition, data may be missing around the axial beamstop. The development of software for automated tomographic diffraction data collection and merging is a large undertaking [20.75] and much can be learnt from the prior experience of tomography in biological electron microscopy, where these techniques have been perfected [20.76]. In that case, however, the registration of successive images at different tilts is greatly facilitated by direct observation of image features. The use of shadow images or x-ray zone-plate images for similar purposes has been suggested. With no direct imaging mode, much time is wasted in x-ray work locating the beam on the sample, which, with current CCD detectors will typically be smaller than \({\mathrm{2}}\,{\mathrm{\upmu{}m}}\) in diameter. The final resolution (in one dimension), allowing for an oversampling factor of 2, will then be \({\mathrm{4000}}/{\mathrm{1024}}={\mathrm{3.9}}\,{\mathrm{nm}}\). The camera length (sample to detector distance) of the diffraction camera must then be selected to allow half this spatial frequency to fall at the edge of the CCD camera at \(u_{\text{max}}=\theta_{\text{max}}/\lambda={\mathrm{0.5}}/{\mathrm{3.9}}\,{\mathrm{nm^{-1}}}\), so that the maximum scattering angle is \(\theta_{\text{max}}={\mathrm{0.25}}\,{\mathrm{rad}}\) for \(\lambda={\mathrm{2}}\,{\mathrm{nm}}\). Then the finest periodicity in the object \(({\mathrm{3.9}}/{\mathrm{0.5}}\,{\mathrm{nm}})\) is sampled twice in every period, according to Shannon's requirement (two points are required to define the period and amplitude of a sine wave if aliasing is avoided). For a CCD with linear pixel number \(N\), the ratio of the finest detail to the largest dimension is \(N/2\), so that developments in detector technology limit CDI. The transverse spatial coherence of the beam must exceed \({\mathrm{4}}\,{\mathrm{\upmu{}m}}\), as discussed together with monochromator requirements below.

Tomographic or 3-D imaging can provide the ability to see inside an object, but this requires that the intensity at a point in a projection be proportional to a line integral of some simple property of the object, such as the charge density. Then methods such as filtered backprojection can re-assemble these 2-D projections into a volume density. Contours of equal density may then be isolated and presented to show internal structure. For CXDI, a different approach is used, and some simplifications occur. It is no longer necessary to make the resolution-limiting flat Ewald sphere approximation, since diffraction data collected at one tilt can be assigned to points lying on the curved Ewald sphere in reciprocal space. (This is the momentum and energy-conserving sphere which describes elastic scattering in reciprocal space.) The sample is then rotated through this sphere around a single axis, until all of reciprocal space is filled, out to a given resolution. Three-dimensional interpolation of data points near the sphere is needed, and careful intensity scaling may be needed if several exposures with different times are needed to cover the full dynamic range of the data. It is often found that missing data points in the central region can be treated as adjustable parameters in the HIO iterations. Once a roughly spherical volume has been filled in reciprocal space (perhaps with missing wedge and beamstop region), the 3-D iterations of the HIO algorithm may be applied ((20.8) and (20.9) extended to three dimensions). The computing demands are severe, as outlined below. The converged data will provide a 3-D density map, proportional to the local charge density, if the single-scattering approximation of x-ray diffraction theory applies, and if the spatial variation in attenuation of the beam due to the photoelectric effect can be neglected. Figure 20.5 shows such a tomographic reconstruction, from which 3-D surfaces of constant density may be obtained. These surfaces allow us to see inside materials and may eventually permit maps to be obtained which distinguish regions of different chemical composition.

Fig. 20.5

(a) Tomographic reconstruction from the soft-x-ray diffraction pattern shown in (b). The object consists of gold balls (\({\mathrm{50}}\,{\mathrm{nm}}\) diameter) lying along the edges of a pyramidal-shaped silicon-nitride structure. This is one image from a rotation series. From the complete series, 3-D surfaces of constant density can be constructed

The usefulness of tomographic CXDI in biology remains to be determined—at present, the method appears to have the advantages over electron microscopy of allowing observation of thicker samples under a wider range of environments (for example, in the water window around \({\mathrm{580}}\,{\mathrm{eV}}\) for soft x-rays). By comparison with x-ray zone-plate full-field imaging, the method allows a much larger numerical aperture to be used, and, hence, makes more efficient use of scattered photons, while providing potentially higher resolution. At high resolution the depth of focus \(\lambda/\theta^{2}\) may become less than the sample thickness, which prevents tomographic reconstruction by backprojection methods based on simple projections. Then tomography may best be undertaken using optical sectioning rather than reconstruction from projections. CDI, based on 3-D diffraction data, provides a third alternative. The resolution limit imposed by radiation damage in CXDI, as expressed by the Rose equation [20.77], remains to be determined experimentally, but is unlikely to be less than \({\mathrm{1}}\,{\mathrm{nm}}\), significantly larger than achieved in 3-D single-particle cryoelectron microscopy of proteins, with their new single-electron detectors [20.73]. Here, \({\mathrm{0.2}}\,{\mathrm{nm}}\) resolution has been achieved for the much smaller (\(<{\mathrm{60}}\,{\mathrm{nm}}\)) molecular structures imaged by cyro-EM in thin vitreous ice films, thin enough to avoid multiple elastic electron scattering ([20.48] for analysis). Howells et al [20.78] and Marchesini et al [20.32, 20.33] provide a detailed discussion of this large subject beyond the scope of this review, including a plot of the dose against resolution for various microscopies in biology, as shown in Fig. 20.6 [20.79]. The dose-fractionation theorem of Hegel and Hoppe is also relevant [20.80]. Images of whole yeast cells, for example, have been imaged by CDI [20.81] using the apparatus described by Beetz et al [20.64].

Fig. 20.6

Dose plotted against resolution, using experimental data from various microscopies. Literature experimental values are shown as follows: filled circles are from x-ray crystallography, filled triangles from electron crystallography, open circles from single-particle electron cryoelectron microscopy, and filled squares in the maximum tolerable dose from recent x-ray crystallography work on spot-fading experiments by Holton on the ribosome at the Advanced Light Source. Viable single-molecule microscopies must fall above the Rose equation line (to give a statistically significant image) and below the maximum tolerable damage line. Those below the Rose equation line succeeded by using crystallographic redundancy, and form a periodically averaged image of perhaps \(\mathrm{10^{8}}\) molecules in a crystal. The required imaging dose is calculated for a protein of empirical formula \(\mathrm{H_{50}C_{30}N_{9}O_{10}S_{1}}\) and density \({\mathrm{1.35}}\,{\mathrm{g{\,}cm^{-3}}}\) against a background of water, imaged with \({\mathrm{10}}\,{\mathrm{keV}}\) x-rays (upper Rose line) and \({\mathrm{1}}\,{\mathrm{keV}}\) (lower Rose line)

20.5 Iterated Projections

A breakthrough in understanding the remarkable success of the HIO algorithm occurred in 1984, when Levi and Stark [20.82] (based on earlier work by Youla and Webb [20.27]) showed that the algorithm could be understood as a successive Bregman projection between convex and nonconvex sets . Here, an image is represented as a single vector \(\boldsymbol{R}\) in an \(N\)-dimensional space, with one coordinate for each pixel. The addition of two such vectors adds together two images. Distance between images (vectors) in this space has the form of the familiar \(\chi^{2}\) goodness of fit index, so that similar images are represented by points in Hilbert space near each other. The set of all images subject to a given constraint (e. g., known symmetry, known Fourier modulus, known sign of density, known support) are considered to occupy a volume in this space. The operation of taking a current estimate of the image, performing a Fourier transform , replacing the magnitudes of the diffracted amplitudes with the known values, and inverse transforming, was shown to be a projection onto the set of images subject to the Fourier modulus constraint. One considers vectors \(\boldsymbol{R}\) between the boundaries of two constrained sets of images. If it is shown that all the images within the only overlap between two constrained sets are equivalent solutions, then the phase problem reduces to finding this volume, where \(\boldsymbol{R}=\chi^{2}\) and the ER error metric are a minimum. Constraints may be of two types—convex and nonconvex. For a convex set, all points on any line segment starting and terminating within the set lie inside the set. A set \(\boldsymbol{P}\) is convex if
$$\alpha\boldsymbol{R}+(1-\alpha)\boldsymbol{R}^{\prime}$$
lies within \(\boldsymbol{P}\) for all \(\boldsymbol{R}\) and \(\boldsymbol{R}^{\prime}\). Here, \(0<\alpha<1\) is a scalar defining position along the line. In two dimensions, a kidney-shaped set is nonconvex, and an ellipse is convex. Bregman has shown that iterative projections between convex sets must lead directly to a unique solution if it exists, without stagnation. In this manner, the global optimization problem is solved without exhaustive search for the case where a unique solution and convex constraints are known to exist. For our problem, the Fourier modulus constraint (the known diffraction intensities) is nonconvex, so that this approach has been of limited value. However, it provides a powerful geometric way of thinking about the algorithm as a trajectory in Hilbert space, which is usually drawn in two dimensions for simplicity. The effects of variations in feedback parameter \(\beta\) can be understood, convergence properties studied, and new algorithms proposed. For the ER algorithm , the path is a zig-zag between the boundaries of sets; for the HIO it is a spiral. Some desirable convex constraints include known support, a knowledge of phase rather than amplitude, the sign constraint, symmetry, a known histogram of density levels  [20.83] (such as exists for proteins), entropy minimization , and (for nonoverlapping atoms) atomicity. A list of constraints used in protein crystallography can be found in the relevant section of volume F of the international tables on crystallography. Application of these constraints thus avoids the common problem whereby optimization programs become trapped in local minima.
The application of constraints can be viewed as projections in the \(N\)-dimensional space. Recall that the support \(\boldsymbol{S}\) is defined as the set of points for which the density is nonzero. For example, application of the support constraint \(\boldsymbol{P}_{\mathrm{s}}\) corresponds to setting many pixels to zero, that is, to projecting onto a space of lower dimension. For the Fourier modulus constraint, we note that Parseval's theorem ensures that distances in the \(N\)-dimensional real space are equal to those in a similar \(N\)-dimensional space of Fourier coefficients. Consider an Argand diagram for a particular Fourier component, for which the modulus constraint restricts solutions to a circle, whose radius is given by the measured value of the Fourier modulus. An estimate provided by the algorithm (e. g., outside this circle) must be projected (by operator \(\boldsymbol{P}_{\mathrm{m}}\)) onto the nearest point on the circle, along a line which will pass through the origin. (This corresponds to the numerical process during one iteration of retaining the current phase estimate, but replacing the magnitude with the measured magnitude, in the HIO algorithm.) Since the linear addition of two vectors terminating on the circle does not produce a third which terminates on the circle, the modulus constraint is not convex. Note, however, that the addition of two vectors of arbitrary length but equal phase produces a new complex number with the same phase, so that a knowledge of phase is a convex constraint, and thus more powerful than a knowledge of amplitudes. The identity operation \(\boldsymbol{I}\) is also useful, and a reflector operation \(\boldsymbol{R}_{\mathrm{s}}=2\boldsymbol{P}_{\mathrm{s}}-\boldsymbol{I}\) can be defined, which reverses the sign of the density outside the support \(\boldsymbol{S}\). Using these operators, all the iterative algorithms can be represented simply and analyzed as alternating projections onto convex (and nonconvex) sets ( ). In this context, we may define these more limited projections as projectors , which takes a given vector \(\boldsymbol{R}\) to the nearest point of a nearby constrained set (usually on its boundary). Then:
  1. 1.
    The error-reduction (ER) (Gerchberg–Saxton) algorithm may be written as
    $$\rho^{(n+1)}=\boldsymbol{P}_{\mathrm{s}}\boldsymbol{P}_{\mathrm{m}}\rho^{(n)}$$
     
  2. 2.
    The charge-flipping ( ) algorithm may be written as
    $$\rho^{(n+1)}=\boldsymbol{R}_{\mathrm{s}}\boldsymbol{P}_{\mathrm{m}}\rho^{(n)}$$
     
  3. 3.
    The hybrid input–output (HIO) algorithm may be written as
    $$\begin{aligned}\displaystyle\rho^{(n+1)}(\boldsymbol{r})&\displaystyle=\boldsymbol{P}_{\mathrm{m}}\rho^{(n)}(\boldsymbol{r})&\displaystyle\text{ if }\boldsymbol{r}\in\mathbf{S}&\displaystyle\\ \displaystyle\rho^{(n+1)}(\boldsymbol{r})&\displaystyle=(\boldsymbol{I}-\beta)\boldsymbol{P}_{\mathrm{m}}\rho^{(n)}(\boldsymbol{r})&\displaystyle\text{ if }\boldsymbol{r}\notin\mathbf{S}&\displaystyle\end{aligned}$$
     
  4. 4.
    The averaged successive reflections ( ) algorithm may be written as
    $$\rho^{(n+1)}=0.5(\boldsymbol{R}_{\mathrm{s}}\boldsymbol{R}_{\mathrm{m}}+\boldsymbol{I})\rho^{(n)}\;. $$
     
Similar descriptions of the difference map method  [20.31], the hybrid projection reflection ( ) [20.28], and the relaxed averaged alternating reflectors ( ) [20.84] have been given. For \(\beta=1\), the HIO, HPR, ASR, and RAAR algorithms are identical. A comparison of the performance of all of these, together with the powerful shrinkwrap algorithm (HIO with dynamic support) and simple geometric representations of the trajectory of the error metric for few-dimensional cases, can be found in Marchesini [20.22].

Using this approach, it has also been shown that the HIO algorithm is equivalent to the Douglas–Rachford algorithm and related to classical convex optimization methods [20.28]. The text by Stark [20.21] is recommended as a tutorial introduction to this large subject.

20.6 Coherence Requirements for CDI Resolution

It is readily shown [20.53] that the lateral or spatial coherence requirement for diffractive imaging is, in one dimension, that the coherence width \(X_{\mathrm{c}}\approx\lambda/\theta_{\mathrm{c}}\) be at least equal to twice the largest lateral dimension \(W\) of the object. (This is similar to the requirement in crystallography that \(X_{\mathrm{c}}\) exceed the dimensions of a primitive unit cell in order to avoid overlap of Bragg beams, with beam divergence \(\theta_{\mathrm{c}}\). For phasing by the oversampling method, this cell must be about twice as big as the molecule.) This fixes the incident beam divergence and, hence, the exposure time for a given object size and source. Since at the un-apertured diffraction limit (\(\theta=90^{\circ}\)) the resolution is approximately equal to the wavelength and about two pixels are required per resolution element, a total of about (\(4X_{\mathrm{c}}/\lambda)^{2}\) image pixels would be needed for a coherence width \(X_{\mathrm{c}}\) and oversampling factor 2. Physically, this just means that the coherence patch must include the known region of vacuum (zero density) surrounding the object boundary (support). One must diffract coherently from an area about twice as big as the isolated object of interest.

The temporal coherence length \(L_{\mathrm{c}}\) is also important. For a field of view \(W\) at the object (so that the first oversampling point occurs at scattering angle \(\lambda/W\)) and finest (bandlimited) object spatial frequency \(d^{-1}\), the optical path difference between points on opposite sides of the object and a distant detector point is \(W\sin\theta=W\lambda/d\), which should not exceed the longitudinal coherence length for x-rays \(L_{\mathrm{c}}=\lambda E/\Updelta E\). Hence, the fractional energy spread allowable in the beam to record spatial frequency \(d^{-1}\) is \(E/\Updelta E> W/d=N\), where \(d\) is the sampling interval in the object and \(N\) the linear number of pixels needed to sample the object space in the HIO algorithm. A more detailed calculation, considering the shape of the temporal coherence function, gives the requirement on longitudinal coherence as about
$$\frac{E}{\Updelta E}> \frac{N}{3}\;,$$
which improves on the estimate in Spence et al [20.53]. This determines the quality of the monochromator needed. In practice, values of \(E/\Updelta E={\mathrm{500}}\) have yielded good results in soft x-ray work using CCD detectors with \(N^{2}\) pixels, where \(N={\mathrm{1024}}\). Then the in-line arrangement of a simple zone-plate monochromator can be used [20.85]. For CDI using electron beams, the coherence requirements are easily met for the nanostructures of most interest. We note that the drive for higher resolution, for a fixed number of object pixels, reduces the demand on coherence. The number of pixels, especially in 3-D tomography, is commonly limited by computer processing power to about \(1024^{3}\) in the medium-term future, as discussed below.

It is important to devise a consistent definition of resolution in CDI . Each spatial frequency (assuming small scattering angles) \(u=d^{-1}=\theta/\lambda\) in the object diffracts energy at scattering angle \(\theta\) into the far field. Consider a square area detector used for diffractive imaging for which the largest scattering angle into the midpoint of the side of the detector (not the corner) is \(\theta_{\text{max}}\). Then, this angle defines a cutoff in the transfer function (see below) at \(u_{\text{max}}=\theta_{\text{max}}/\lambda\), and the full period of the corresponding finest periodicity in the object which can be reconstructed is \(d_{\text{min}}=\lambda/\theta_{\text{max}}=1/u_{\text{max}}\). This value of \(d_{\text{min}}\) has frequently been quoted as the resolution limit in CXDI. However, while it is a most important experimental parameter, it fails to consider the accuracy of the phasing process and is not equal to the corresponding Rayleigh resolution limit. There are other considerations, which we now discuss.

The Rayleigh resolution limit \(d_{\mathrm{R}}\) was intended for spectroscopy and the imaging of binary stars, which are incoherent point sources, unlike the phase contrast usually important for CXDI . If we do, nevertheless, wish to apply the Rayleigh condition to this situation, we may consider that the CXDI area detector in the far field plays the role of a square aperture in the back-focal plane of an ideal lens. The numerical aperture imposed on the reconstruction is then \(\theta_{\text{max}}=\lambda u_{\text{max}}\), and, if the reconstruction is perfect (no errors in phasing), the image will be given by the ideal object charge-density convoluted with a sinc function amplitude (the impulse response for linear imaging), whose full width at half maximum is \(d_{\mathrm{R}}=d_{\text{min}}/2=0.5/u_{\text{max}}\). (The distance between first minima is \(d_{\text{min}}\). For a circular area detector the factor \(\mathrm{0.5}\) becomes \(\mathrm{0.61}\)—the square detector does slightly better because of contributions from the corners.) Thus our adapted Rayleigh resolution is half the finest spatial periodicity , and is, therefore, equal to the sampling interval needed in real space for linear phase-contrast imaging, in order to avoid loss of information. Two samples are needed for every full periodicity in the object. (For an incoherent imaging model, the impulse response becomes a sinc squared function, for which a sampling interval of \(d_{\text{min}}/4\) is needed, since the autocorrelation of the transform of the sinc squared function is a triangular function, doubling the bandwidth and resolution , as pointed out in Rayleigh's original paper.) For a lens-based system with aberrations, the factor \(\mathrm{0.5}\) depends on the aberrations of the lens, but takes its minimum value for the diffraction-limited CXDI case.

There are two further considerations. For phase contrast, the ability to distinguish adjacent small objects will depend on the phase shift each introduces, and thus the resolution becomes a property of the sample, not only of the instrument. It is then not possible to define resolution in a meaningful sample-independent manner. (In fact, the resolution limits under coherent and incoherent illumination conditions are equal if neighboring points in the object scatter with a \(90^{\circ}\) phase shift. Incoherent illumination does better if they are in phase, and worse if they are in antiphase). In coherent optics, this problem is partly addressed by introducing the concept of a lens-coherent transfer function ( ). Such a function has been introduced in a way which also tests the reliability of the phasing process in a proposal by V. Elser for a resolution definition for CXDI [20.86]. The algorithm is repeatedly run to convergence, each time starting with a different set of random phases. The results for image contrast are plotted as a function of spatial frequency , showing, if noise is not too severe, a relatively smooth curve which falls to zero at some \(u_{\text{max}}\). (If the phasing process fails, the average of these many runs will be zero at each spatial frequency.) The resolution is then \(0.5/u_{\text{max}}\), if a square detector is used. This appears to be the best current definition of resolution for CXDI, however, it ignores the dependence of resolution on sample properties for phase contrast.

For a known object, the faithfulness of the reconstruction may be indicated by a cross-correlation function between the reconstructed estimate and the known object, or crystallographic \(R\)-factor, as discussed in detail elsewhere [20.61]. For an unknown object, the ER error metric \(\varepsilon\) defined in (20.10) has been shown by computational trials against known objects to vary monotonically with a cross-correlation function [20.18]. The best resolution achieved in CXDI is currently about \({\mathrm{8}}\,{\mathrm{nm}}\).

20.7 Single-Particle Image Reconstruction from XFEL Data

The invention of the hard x-ray (free-electron) laser ( ), the first of which began operation at the US Department of Energy SLAC laboratory near Stanford in 2009, has created many new opportunities for imaging, especially fast imaging with femtosecond time resolution. More importantly, for structural biology, it has allowed time-resolved imaging to be undertaken while avoiding most effects of radiation damage . The diffract-then-destroy method used outruns damage, using, for example, x-ray pulses of \({\mathrm{10}}\,{\mathrm{fs}}\) duration to produce an x-ray diffraction before the onset of damage from the later cascade of photoelectrons, which subsequently destroy the sample [20.72, 20.87]. This has allowed molecular movies to be made under room-temperature conditions (without the need for cooling to avoid damage) and controlled chemical conditions in the correct thermal bath [20.88]. This is possible using microcrystal samples (which provide Bragg scattering to atomic resolution) where the atomic motions within each molecule of the crystal do not destroy crystallinity. A continuous supply of identical samples is, therefore, needed, whose scattering patterns must be merged. A typical pulse may contain about \(\mathrm{10^{11}}\) photons (at \({\mathrm{8}}\,{\mathrm{kV}}\)), which will scatter more than a million photons from a single large virus. X-ray beam diameters can be as small as \({\mathrm{0.1}}\,{\mathrm{\upmu{}m}}\) in diameter, with a fractional energy spread in the beam of \(\mathrm{10^{-3}}\), and a repetition rate for x-ray pulses of \({\mathrm{120}}\,{\mathrm{Hz}}\). This is the rate at which diffraction patterns are read out from the area detector. The field of biology with x-ray lasers , including a simple description of the functioning of an XFEL, is reviewed in Spence [20.89] and Bosted et al [20.90], and is supported by a consortium of US universities which provides assistance to interested researchers (https://www.bioxfel.org). Whereas scattering from individual viruses has been detected at \({\mathrm{0.5}}\,{\mathrm{nm}}\), the highest-resolution 3-D reconstruction from a virus single particle is currently about \({\mathrm{9}}\,{\mathrm{nm}}\), whereas Bragg reflections are routinely detected from protein microcrystals at \({\mathrm{0.2}}\,{\mathrm{nm}}\) resolution. This section is limited to single-particle ( ) data analysis methods in biology (with one particle, such as a virus, per shot). For reviews of the atomic-resolution XFEL work on static and time-resolved imaging of proteins in crystals, see Schlichting [20.91] and Spence et al [20.92].

For SP data analysis, with one particle, such as a virus, per shot, the methods of coherent diffractive imaging (CXDI) discussed in previous sections have been adapted for XFEL data. Unlike the CXDI problem, the orientational relationship between successive diffraction patterns must first be determined from the randomly oriented particles of unknown structure. This process requires a certain minimum number of detected photons. The accuracy of the orientation determination may then limit resolution. This determination must be done prior to solution of the phase problem. Approaches to the orientation determination problem include manifold embedding  [20.93, 20.94] and the expectation maximization and compression ( ) algorithm; [20.6, 20.95] and references therein also used in cryo-EM.

As a model sample, in order to get an estimate of scattered intensity per shot, we first consider the scattering from a dielectric sphere of uniform electron density. The far-field coherent diffraction pattern of a sphere of radius \(R\), embedded in a medium such as water, is given by [20.49]
$$I(q)=I_{0}r^{2}_{\mathrm{e}}\Updelta\Omega|\Updelta\rho|^{2}\left(4\uppi\frac{\sin Rq-Rq\cos Rq}{q^{3}}\right)^{2},$$
where \(I_{0}\) is the incident fluence, \(\Updelta\rho\) is the difference between the complex electron density of the sphere and the medium, and \(q=(4\uppi/\lambda)\sin(\theta)\) is the photon momentum transfer for a scattering angle \(2\theta\). The term in brackets is the 3-D Fourier transform of a uniform sphere. The difference electron density can be written in terms of a complex refractive index \(n\) as
$$\begin{aligned}\displaystyle r^{2}_{\mathrm{e}}|\Updelta\rho|^{2}&\displaystyle=\left(\frac{2\uppi}{\lambda^{2}}\right)^{2}|\Updelta n|^{2}\;,\\ \displaystyle|\Updelta n|^{2}&\displaystyle=|n_{\text{prot}}-n_{\text{water}}|^{2}\\ \displaystyle&\displaystyle=(\delta_{\text{prot}}-\delta_{\text{water}})^{2}+(\beta_{\text{prot}}-\beta_{\text{water}})^{2}\;,\end{aligned}$$
where \(\delta\) and \(\beta\) are the optical constants, and the subscripts refer to the protein and the water medium. Far from absorption edges, \(\Updelta\rho\) is independent of wavelength.
The intensity pattern \(I(q)\) consists of circular rings (similar to Airy rings; Fig. 20.7) with minima spaced in \(q\) approximately by \(\uppi/R\). The maximum intensity of the rings is given by
$$I_{\text{max}}(q)=I_{0}r^{2}_{\mathrm{e}}\Updelta\Omega|\Updelta\rho|^{2}\frac{16\uppi^{2}R^{2}}{q^{4}}\;.$$
This \(q^{-4}\) dependence leads to the familiar problem in coherent diffractive imaging of having to simultaneously record strong intensity at low angles and much weaker intensities at higher angles, with the range of intensities often exceeding the dynamic range of the detector.
Fig. 20.7

XFEL diffraction pattern from a single PBCV icosahedral virus (Paramecium bursaria Chlorella virus 1) of \({\mathrm{190}}\,{\mathrm{nm}}\) diameter. Single LCLS shot using \({\mathrm{1.8}}\,{\mathrm{kV}}\) x-rays showing a \({\mathrm{12}}\,{\mathrm{nm}}\) resolution on a pnCCD detector. Reprinted from [20.89], with permission from Elsevier

The fastest variation of diffracted intensity across the detector will be due to interference between x-rays scattered by points in the object located furthest from each other: the sphere diameter. Measuring this maximum fringe spatial frequency requires at least two detector pixels per period, or a so-called Shannon sampling of \(\Updelta q=\uppi/(2R)\). In this case, the pixel solid angle is \(\Updelta\Omega=(\lambda/(2\uppi))^{2}\Updelta q^{2}\), so that the previous equation becomes
$$I_{\text{max}}(q)=I_{0}r^{2}_{\mathrm{e}}|\Updelta\rho|^{2}\uppi^{2}\lambda^{2}q^{-4}\;,$$
which is independent of sphere radius.

The most commonly studied single particles at XFELs are icosahedral viruses , containing either DNA or RNA. The first experimental requirement then is that the individual shots extend to sufficiently high resolution to distinguish such a sphere from an icosahedron (Fig. 20.7). It may seem that for the merging of thousands of diffraction patterns from similar randomly-oriented single particles (such as a virus), the same methods as used in the cryoelectron microscopy (cryo-EM) community could be used. Here, noisy low-dose projection images of many copies of a particle, lying in many random orientations, are recorded within the field-of-view of each image, and must be merged to produce a 3-D image [20.73]. However, the XFEL diffraction patterns also require solution of the phase problem, and, unlike real-space cryo-EM images, there is no requirement for correction of electron lens aberrations, while an enantiomorphous ambiguity arises from the Friedel symmetry of low-resolution patterns, not present for real-space images. In addition, diffraction patterns have an origin, unlike images, and the background due to ice in cryo-EM images must be treated differently from the background in an x-ray diffraction pattern due to diffraction from a water jacket surrounding the particle. Building on previous work on iterative phasing of continuous diffraction patterns, two main approaches have been developed for the reconstruction of a 3-D image (density map) from many randomly oriented snapshot single-particle x-ray diffraction patterns, and for dealing with the associated problems of particle inhomogeneity. We will give here only a brief outline of the general principles of these methods, focusing on key issues.

The manifold embedding approach [20.93] is illustrated in Fig. 20.8a,b, simplified for the case of a three-pixel (\(x,y,z\)) detector and single-axis rotation of a particle, in order to illustrate the principle of the method. With this simplification, a snapshot diffraction pattern can be represented as a 3-D vector, with each component representing the scattered intensity value at a pixel. Rotation of a particle traces out a loop (a 1-D manifold) in this 3-D space of intensities. Determining this manifold allows one to assign an orientation to each snapshot. In general, the detector has \(N\) pixels, and particle rotation about three axes generates a 3-D manifold in the \(N\)-dimensional Hilbert space of pixel intensities. The manifold is seen to be parameterized by a 3-D latent space defined by the three Euler angles defining the particle orientation. If the experimental diffraction patterns are recorded in random orientations and assigned to points in this space, the identification of the loop will then allow them to be sequenced for a movie. Many practical difficulties arise, including the transformation from angular increment to coordinate change in \(N\) dimensions, and the effects of noise and conformational changes. In the simplest case, a second conformation would define a second distinct loop, however the effects of noise thicken the manifolds, so that they may overlap. The key issue of distinguishing changes in particle orientation from conformational changes (essential in order to show a smooth representation of conformation change, as a kind of movie) is resolved using the fact that the operations associated with conformational change commute, while those associated with the rotation group do not. Conformational changes alter the internal structure of a particle, unlike rotations. An important feature of this approach is that all the data is used for all the analysis, rather than selecting subclasses (e. g., of orientation or conformation) for successive analysis. However, even in the absence of noise, a minimum number of scattered counts is needed to identify a particular orientation, proportional to the number of distinct orientations sought. The computational demands of this approach are considerable and set the limit on the size of the largest molecule, which can be analyzed. This method has been used to reconstruct 3-D images of the PR772 icosohedral virus at \({\mathrm{9}}\,{\mathrm{nm}}\) resolution, using snapshot diffraction data collected at the LCLS XFEL, as shown in Fig. 20.9 [20.94]. By using correlation methods to sequence 3-D images by similarity, obtained from an ensemble of viruses in equilibrium, it was possible to map out the conformational energy landscape and perhaps show the emergence of the viral genome from a portal vertex. It is suggested that the large amount of data which can be collected from a single particle at an XFEL (at high repetition rate) will provide access to a wider range of conformational change than is possible by cryo-EM, since, for a statistical ensemble in equilibrium, the fractions of each conformational class are related by a Boltzmann exponential factor involving their energy difference.

Fig. 20.8a,b

Simplified manifold embedding approach for a sample which can rotate only about one axis and a three-pixel detector. One vector in this 3-D space represents a diffraction pattern, each axis is a pixel, and each coordinate value is an intensity for that pixel. Rotation of the molecule causes the vector to trace out a loop as the particle returns to its original orientation, while neighboring points on the loop represent similar diffraction patterns (allowing them to be ordered for a movie) with small vectors \(\chi\) (the least-squares difference, Euclidean metric) between their ends

Fig. 20.9

Reconstructed 3-D images of colliphage PR772 icosahedral virus made from \(\mathrm{37550}\) XFEL snapshot diffraction patterns at \({\mathrm{1.6}}\,{\mathrm{kV}}\), each with one virus per shot. The resolution is \({\mathrm{9}}\,{\mathrm{nm}}\) and the particle diameter about \({\mathrm{70}}\,{\mathrm{nm}}\). Ranked by similarity, the last suggests emergence of the genome. Density is seen to decrease in the center of the virus. From [20.94]

A second approach has been based on the principle of expectation maximization  [20.95, 20.96, 20.97]. The method, known a Expand, Maximize and Compress (EMC) can be explained for the simple case of a set of noisy 2-D pictures or images \(I_{k}(i)\) (where \(i\) denotes a pixel) of the same nonsymmetric 2-D object. Assume that this object is known to lie in any one of four orientations \(\theta_{j}\) differing by \(90^{\circ}\) rotations about the normal to the object plane. We wish to merge the information from all these images in unknown orientations into a single erect image with reduced noise. A model \(M(i)\) of the picture intensity is first assumed. (This may initially consist of random numbers). Assuming Poisson statistics to identify the background, the probability \(P_{k}(\theta_{j})\) is calculated that each experimental image \(I_{k}(i)\) came from the model in each of the four orientations \(\theta_{j}\). Each image \(I_{k}(i)\) then has these four probabilities associated with it. The probabilities are nomalized to unity. The pixels \(i\) of the model \(M(i)\) are then updated to \(M^{\prime}(i)\) by first summing a pixel in an image \(I_{k}(i)\) over the values it would have in each orientation, weighted by the probability \(P_{k}(\theta_{j})\) of that orientation occurring. This process is repeated for all the pixels and images, and the resulting images are then summed to form the updated model \(M^{\prime}(i)\) for the next iteration. The update rule is therefore
$$M^{\prime}(i)=\Sigma_{k}\Sigma_{j=1,4}P_{k}(\theta_{j})I_{k}(i-\theta_{j})$$
where \(I_{k}(i-\theta_{j})\) is the intensity of the pixel at \(i\) after rotation of the image by \(\theta_{j}\) about the normal to the image.

This approach can be generalized to the problem of orientation and merging single-particle diffraction data, where the scattering from successive similar particles in random orientations is limited to the Ewald sphere, and three Euler angles must be assigned to fix the orientation of the scattering on the sphere into the three-dimensional diffraction volume. The angles must be quantized into discrete increments to limit the computational expense. We then work with the scattered intensities rather than the image intensities above. The method then reconstructs the best set of diffracted intensities for the 3-D diffraction volume which are most consistent with all the diffraction data. An experimental demonstration of the method using low-resolution 2-D x-ray shadow images has been demonstrated using only a few photons per image [20.98].

The method has some similarity to cryo-EM methods based on cross-correlation between experimental patterns to find similar orientations but has the advantage that cross-correlations are computed between experimental patterns and a model, rather than between every possible experimental pair, so that the computation time is linear, rather than quadratic, in the number of patterns. Like the GTM method , this approach uses the entire body of data in each update of the model parameters, however, the GTM method does not constrain the data to fit a 3-D model density. At least a thousand counts per image are needed to assign patterns to a particular orientation class.

In both these methods, solution of the noncrystallographic phase problem may be integrated with the problem of orientation determination. Particle inhomogeneity (which increases with particle size) is the most important problem for single-particle XFEL imaging and may be solved in principle by the ability of the above methods to distinguish conformations, if sufficient high-quality data is available. A method for obtaining a 3-D reconstruction from a single shot is described in [20.99], using multiple incident beams from a beamsplitter. Several authors have pointed out that the curvature of the Ewald sphere provides limited 3-D information from a single shot, while Bergh et al [20.100] describe other possibilities for extracting 3-D information from a single shot, such as Laue diffraction using harmonics, coherent-convergent beam diffraction, and multiple-pinhole Fourier transform holography. Figure 20.7 shows the diffraction pattern obtained from a single Chlorella virus (one particle per shot). The rings extend clearly to a resolution of \({\mathrm{12}}\,{\mathrm{nm}}\). A 3-D reconstruction of the Mimivirus obtained from XFEL scattering is described in Ekeberg et al [20.6] at \({\mathrm{125}}\,{\mathrm{nm}}\) resolution. Here the EMC method, based on model diffraction patterns, was used for orientation determination and a modified form \(f\) the HIO algorithm used for phasing. Figure 20.10 shows the diffraction patterns (one particle per shot) obtained from Mimivirus particles and the reconstructed 3-D image of the virus obtained using the EMC algorithm  [20.6].

Fig. 20.10

(a) 3-D reconstruction of Mimivirus (\({\mathrm{450}}\,{\mathrm{nm}}\) diameter capsid) density at \({\mathrm{125}}\,{\mathrm{nm}}\) resolution obtained using Hawk software (EMC algorithm) from \(\mathrm{198}\) single-shot diffraction patterns obtained at LCLS (AMO, pnCCD detector); (b\({\mathrm{70}}\,{\mathrm{fs}}\) pulses, \({\mathrm{1.2\times 10^{12}}}\,{\mathrm{ph/pulse}}\), (\({\mathrm{0.24}}\,{\mathrm{mJ}}\)), \({\mathrm{1.2}}\,{\mathrm{keV}}\) x-rays. Reprinted with permission from [20.6]. Copyright 2015 by the American Physical Society

The most important experimental problems in this field are the hit-rate of the sample-delivery devices used. These typically consist of an electrospray or gas-dynamic virtual nozzle  [20.101] feeding an aerodynamic lens stack [20.6]. Since the hydrated virus particles are not synchronized with the XFEL pulses, hit rates are typically about \({\mathrm{1}}\%\), producing much lower volumes of data than work with microcrystals. Plating-out of salts, which concentrate as droplets evaporate, may also lead to apparent differences in particle sizes. In general, heterogeneity of the particles creates challenging problems, however this is likely to be addressed in the same way as in Cryo-EM, which particles quenched from an equilibrium ensemble are sorted in software by conformation. It has been pointed out that the much larger volume of data obtainable by XFEL SP methods should permit larger conformational changes to be imaged, and data volumes will increase with the new XFELs planned which operate at much higher repetition rates. Data aquisition is limited by detector read-out speed, which is expected to increase from \({\mathrm{120}}\,{\mathrm{Hz}}\) to about \({\mathrm{20}}\,{\mathrm{kH}}\) in the near future.

A database for SFX and SP data has been established at http://cxidb.org/index.html, where published data can be found and used to evaluate new algorithms. A special issue of the Journal of Applied Crystallography has appeared, devoted to software and algorithms for single-particle (and serial crystallography) analysis [20.102], and experimental XFEL single-particle data sets have been published for analysis by any interested scientists (Munke et al [20.103] for rice dwarf virus, and Reddy et al [20.104] for the icosoheral coliphage virus PR772, both collected at the LCLS).

20.8 Summary

The past two decades have been exciting times for coherent diffractive imaging ( ) and lensless imaging . For static structures, there have been many successes [20.105], as reported in the continuing series of coherence conferences, and there is every reason to suppose that the pursuit by tomographic CXDI imaging of these types of materials to higher resolution will be possible and of great interest to scientists. Using medium-energy x-rays, the imaging of much thicker material should be possible than that studied by tomographic electron microscopy in materials science [20.106]. However, in view of the difficulty of preparing samples with compact support for inversion by the HIO algorithm (and its variants), much recent x-ray work has been devoted to ptychography [20.107, 20.5]. For fast time-resolved imaging, however, where the focused scanned beam of ptychography cannot be used, CDI remains the method of choice for single particles, and there has been a creative explosion of ideas on how to improve the original HIO algorithm. In addition, the advent of the x-ray laser now provides biologists the possibility of minimizing radiation damage by outrunning it with femtosecond pulses.

Figure 20.6 shows a plot of dose against resolution, with the domain of applicability for a variety of microscopies indicated, indicating the niche for CXDI. Whole-cell imaging has been pursued with both the zone-plate x-ray microscope and by cryoelectron microscopy, where, using the latter technique, a resolution of about \({\mathrm{2}}\,{\mathrm{nm}}\) is possible in samples up to \({\mathrm{100}}\,{\mathrm{nm}}\) thick or more. It seems likely that radiation damage will prevent competitive performance by CXDI, however, the method may provide useful images at perhaps \({\mathrm{5}}\,{\mathrm{nm}}\) resolution in much thicker samples using medium-energy x-rays in an environment of vitreous ice , and the resolution limit when using XFEL pulses, which can out-run radiation damage , has yet to be fully explored. The publication of the first conformational movie from XFEL single-particle data is an important development, [20.94], as shown in Fig. 20.9. The optimum choice of x-ray energy involves many issues, including the variation of synchrotron undulator brightness with beam energy and the variation in phase contrast with beam energy. While the coherent flux \(B\) available for a given synchrotron source brightness varies as \(\lambda^{2}\), the required fluence (from the x-ray cross-section) \(A\) scales as \(\lambda^{-2}\), so that the recording time \(A/B\) varies as \(\lambda^{-4}\) in a most unfavorable manner as the x-ray beam energy increases. The dose in grays needed to scatter a given number of photons into a voxel varies inversely as the fourth power of the resolution in tomography. An analysis of the variation of dose against resolution for several microscopies, including CDI, can be found in Marchesini et al [20.32, 20.33]. Here, the statistical demands of good imaging (based on the Rose equation) are compared with the maximum tolerable dose for a given resolution for single-particle imaging.

In summary, the demand for higher-resolution 3-D noninvasive imaging with old and new radiation sources continues unabated in both materials science and biology. New convex constraints are steadily being discovered, leading to steady incremental advances. Looking back, it seems clear now that diffractive imaging has made a decisive contribution to imaging science, with exciting possibilities for further development.

Notes

Acknowledgements

This chapter has summarized the work of many groups and many of the author's collaborators, as indicated in the References. The author is particularly grateful for the help of Malcolm Howells, Uwe Weierstall, Rick Millane, Anton Barty, Nadia Zatsepin, and Rick Kirian during the preparation of this review. The work was supported by NSF STC award 1231306 and NSF ABI 1565180.

References

  1. S. Grandl, K. Scherer, A. Sztrókay-Gaul, L. Birnbacher, K. Willer, M. Chabior, J. Herzen, D. Mayr, S. Auweter, F. Pfeiffer, F. Bamberg, K. Hellerhoff: Improved visualization of breast cancer features in multifocal carcinoma using phase-contrast and dark-field mammography: An ex vivo study, Eur. Radiol. 25, 3659–3668 (2015)CrossRefGoogle Scholar
  2. D. Paganin, K. Nugent: Noninterferometric phase imaging with partially coherent light, Phys. Rev. Lett. 80, 2586 (1998)CrossRefGoogle Scholar
  3. J.M. Zuo, I.A. Vartanyants, M. Gao, M. Zhang, L.A. Nagahara: Atomic resolution imaging of a carbon nanotube from diffraction intensities, Science 300, 1419 (2003)CrossRefGoogle Scholar
  4. H. Chapman, A. Barty, S. Marchesini, A. Noy, S. Hau-Riege, C. Cui, M. Howells, R. Rosen, H. He, J.C.H. Spence, U. Weierstall, T. Beetz, C. Jacobsen, D. Shapiro: High-resolution ab initio three-dimensional x-ray diffraction microscopy, J. Opt. Soc. Am. A 23, 1179 (2006)CrossRefGoogle Scholar
  5. D.A. Shapiro, Y.S. Yu, T. Tyliszczak, J. Cabana, R. Celestre, W. Chao, K. Kaznatcheev, A.D. Kilcoyne, F. Maia, S. Marchesini, Y.S. Meng: Chemical composition mapping with nanometre resolution by soft x-ray microscopy, Nat. Photonics 8, 765 (2014)CrossRefGoogle Scholar
  6. T. Ekeberg, M. Svenda, C. Abergel, F.R.N.C. Maia, V. Seltzer, J.M. Claverie, M. Hantke, O. Jönssin, C. Nettelblad, G. van der Schot, M. Liang, D.P. DePonte, A. Barty, M.M. Seibert, B. Iwan, I. Andersson, N.D. Loh, A.V. Martin, H. Chapman, C. Bostedt, J.D. Bozek, K.R. Ferguson, J. Krzywinski, S.W. Epp, D. Rolles, A. Rudenko, R. Hartman, N. Kommel, J. Hajdu: Three-dimensional reconstruction of the giant Mimivirus particle with an x-ray free-electron laser, Phys. Rev. Lett. 114(9), 098102 (2015)CrossRefGoogle Scholar
  7. Lord Rayleigh: XLVII. On the interference bands of approximately homogeneous light; in a letter to Prof. A. Michelson, Philos. Mag. 34, 407 (1892)CrossRefGoogle Scholar
  8. R.P. Millane: Phase retrieval in crystallography and optics, J. Opt. Soc. Am. A 7, 394 (1990)CrossRefGoogle Scholar
  9. R.P. Millane, V.L. Lo: Iterative projection algorithms in protein crystallography. I. Theory, Acta Crystallogr. A 69, 517 (2013)CrossRefGoogle Scholar
  10. D. Sayre: Some implications of a theorem due to Shannon, Acta Crystallogr. 5, 843 (1952)CrossRefGoogle Scholar
  11. H. He, W. Su: Direct phasing of protein crystals with high solvent content, Acta Crystallogr. A 71, 92–98 (2015)CrossRefGoogle Scholar
  12. R.W. Gerchberg, W.O. Saxton: Phase determination from image and diffraction plane pictures in the electron microscope, Optik 34, 275 (1971)Google Scholar
  13. R.W. Gerchberg, W.O. Saxton: A practical algorithm for the determination of the phase from image and diffraction plane pictures, Optik 35, 237 (1972)Google Scholar
  14. D. Sayre: Prospects for long-wavelength x-ray microscopy and diffraction. In: Image Processing and Coherence in Physics, Springer Lecture Notes in Physics, Vol. 112, ed. by M. Schlenker (Springer, New York 1980) p. 229CrossRefGoogle Scholar
  15. R. Bates, M. McDonnell: Image Restoration and Reconstruction (Oxford Univ. Press, Oxford 1989)Google Scholar
  16. C. Liao, M. Fiddy, C. Byrne: Imaging from the zero locations of far-field-intensity data, J. Opt. Soc. Am. A 14, 3155 (1997)CrossRefGoogle Scholar
  17. J.R. Fienup: Phase retrieval algorithms: A comparison, Appl. Opt. 21, 2758 (1982)CrossRefGoogle Scholar
  18. J.R. Fienup: Invariant error metrics for image reconstruction, Appl. Opt. 36, 8352 (1997)CrossRefGoogle Scholar
  19. Y. Bruck, L. Sodin: On the ambiguity of the image reconstruction problem, Opt. Commun. 30, 304 (1979)CrossRefGoogle Scholar
  20. R. Barakat, G. Newsam: Necessary conditions for a unique solution to two-dimensional phase recovery, J. Math. Phys. 25, 3190 (1984)CrossRefGoogle Scholar
  21. H. Stark: Image Recovery: Theory and Applications (Academic Press, New York 1987)Google Scholar
  22. S. Marchesini: Invited article: A unified evaluation of iterative projection algorithms for phase retrieval, Rev. Sci. Instrum. 78, 011301 (2007)CrossRefGoogle Scholar
  23. B.-C. Wang: Resolution of phase ambiguity in macromolecular crystallography, Methods Enzymol. 115, 90 (1985)CrossRefGoogle Scholar
  24. R. Fienup: Analytic design of optimum holographic optical elements, J. Opt. Soc. Am. A 4, 118 (1987)CrossRefGoogle Scholar
  25. D. Sayre, H. Chapman, J. Miao: On the extendibility of x-ray crystallography to noncrystals, Acta Crystallogr. A 54, 232 (1998)CrossRefGoogle Scholar
  26. J. Miao, D. Sayre, H. Chapman: Phase retrieval from the magnitude of the Fourier transforms of nonperiodic objects, J. Opt. Soc. Am. A 15, 1662 (1998)CrossRefGoogle Scholar
  27. D.C. Youla, H. Webb: Image restoration by the method of convex projections: Part 1 Theory, IEEE Trans. Med. Imaging 1(2), 81–94 (1982)CrossRefGoogle Scholar
  28. H. Bauschke, P.I. Combettes, D.L. Luke: Phase retrieval, error reduction algorithm, and Fienup variants: A view from convex optimization, J. Opt. Soc. Am. A 19, 1334 (2002)CrossRefGoogle Scholar
  29. R.P. Millane: Multidimensional phase problems, J. Opt. Soc. Am. A 13, 725 (1995)CrossRefGoogle Scholar
  30. H. Bauschke, P.I. Combettes, D.R. Luke: Hybrid projection–reflection method for phase retrieval, J. Opt. Soc. Am. A 20, 1025 (2003)CrossRefGoogle Scholar
  31. V. Elser: Phase retrieval by iterated projections, J. Opt. Soc. Am. A 20, 40 (2003)CrossRefGoogle Scholar
  32. S. Marchesini, H.N. Chapman, S.P. Hau-Riege, R.A. London, A. Szoke, H. He, M.R. Howells, H. Padmore, R. Rosen, J.C.H. Spence, U. Weierstall: Coherent x-ray diffractive imaging: Applications and limitations, Opt. Express 11, 2344 (2003)CrossRefGoogle Scholar
  33. S. Marchesini, H. He, H. Chapman, S. Hau-Riege, M. Noy, M. Howells, U. Weierstall, J.C.H. Spence: X-ray image reconstruction from a diffraction pattern alone, Phys. Rev. B 140101(R), 68 (2003)Google Scholar
  34. I.K. Robinson, I.A. Vartanyants, G. Williams, M. Pfeifer, J. Pitney: Reconstruction of the shapes of gold nanocrystals using coherent x-ray diffraction, Phys. Rev. Lett. 87, 195505 (2001)CrossRefGoogle Scholar
  35. J.C.H. Spence, M. Howells, L.D. Marks, J. Miao: Lensless imaging: A workshop on ‘‘new approaches to the phase problem for nonperiodic objects'', Ultramicroscopy 90, 1 (2001)CrossRefGoogle Scholar
  36. J.N. Cederquist, J.R. Fienup, J.C. Marron, R.G. Paxman: Phase retrieval from experimental far-field speckle data, Opt. Lett. 13, 619 (1988)CrossRefGoogle Scholar
  37. J.C.H. Spence, U. Weierstall, M. Howells: Phase recovery and lensless imaging by iterative methods in optical, x-ray and electron diffraction, Philos. Trans. R. Soc. A 360, 875 (2002)CrossRefGoogle Scholar
  38. J. Miao, C. Charalambous, J. Kirz, D. Sayre: Extending the methodology of x-ray crystallography to allow imaging of micrometre-sized noncrystalline specimens, Nature 400, 342 (1999)CrossRefGoogle Scholar
  39. U. Weierstall, Q. Chen, J.C.H. Spence, M. Howells, M. Isaacson, R. Panepucci: Image reconstruction from electron and x-ray diffraction patterns using iterative algorithms: Experiment and simulation, Ultramicroscopy 90, 171 (2001)CrossRefGoogle Scholar
  40. J.M. Zuo, J.C.H. Spence: Advanced Electron Microsscopy (Springer, New York 2017)Google Scholar
  41. J. Kirz, C. Jacobsen, M. Howells: Soft x-ray microscopes and their biological applications, Q. Rev. Biophys. 28, 33 (1995)CrossRefGoogle Scholar
  42. G. Radi: Complex lattice potentials in electron diffraction calculated for a number of crystals, Acta Crystallogr. A 26, 41 (1970)CrossRefGoogle Scholar
  43. A. Howie, R.M. Stern: The optical potential in electron diffraction, Z. Naturforsch. A 27, 382 (1972)CrossRefGoogle Scholar
  44. J.C.H. Spence: On the accurate measurement of structure factor amplitudes and phases by electron diffraction, Acta Crystallogr. A 49, 231 (1993)CrossRefGoogle Scholar
  45. M.A. O'Keeffe, J.C.H. Spence: On the average Coulomb potential (\(\Phi_{0}\)) and constraints on the electron density in crystals, Acta Crystallogr. A 50, 33 (1993)CrossRefGoogle Scholar
  46. K.H. Downing, R.M. Glaeser: Estimating the effect of finite depth of field in single-particle Cryo-EM, Ultramicroscopy 184, 94 (2017)CrossRefGoogle Scholar
  47. G. Subramanian, S. Basu, H. Liu, J.M. Zuo, J.C.H. Spence: Solving protein nanocrystals by cryo-EM diffraction: Multiple scattering artifacts, Ultramicroscopy 148, 87 (2014)CrossRefGoogle Scholar
  48. D. Starodub, P. Rez, G. Hembree, M. Howells, D. Shapiro, H.N. Chapman, P. Fromme, K. Schmidt, U. Weierstall, R.B. Doak, J.C.H. Spence: Dose, exposure time and resolution in serial x-ray crystallography, J. Synchrotron Radiat. 15, 62 (2008)CrossRefGoogle Scholar
  49. R.P. Millane, W.J. Stroud: Reconstructing symmetric images from their undersampled Fourier intensities, J. Opt. Soc. Am. A 14, 568 (1997)CrossRefGoogle Scholar
  50. V. Elser, R.P. Millane: Reconstruction of an object from its symmetry-averaged diffraction pattern, Acta Crystallogr. A 64(2), 273 (2008)CrossRefGoogle Scholar
  51. R.P. Millane, R.D. Arnal: Uniqueness of the macromolecular crystallographic phase problem, Acta Crystallogr. A 71, 592 (2015)CrossRefGoogle Scholar
  52. J.C.H. Spence, U. Weierstall, M. Howells: Coherence and sampling requirements for diffractive imaging, Ultramicroscopy 101, 149 (2004)CrossRefGoogle Scholar
  53. W. McBride, N. O'Leary, L. Allen: Retrieval of a complex-valued object from its diffraction pattern, Phys. Rev. Lett. 93, 233902 (2004)CrossRefGoogle Scholar
  54. H. He, M. Howells, S. Marchesini, H. Chapman, U. Weierstall, H. Padmore, J. Spence: Use of extended and prepared reference objects in experimental Fourier transform x-ray holography, Appl. Phys. Lett. 85, 2454 (2004)CrossRefGoogle Scholar
  55. A. Szöke: Holographic microscopy with a complicated reference, J. Imaging Sci. Technol. 41, 332 (1997)Google Scholar
  56. G. Oszlanyi, A. Suto: Ab initio structure solution by charge flipping, Acta Crystallogr. A 60, 134 (2004)CrossRefGoogle Scholar
  57. J. Wu, U. Weierstall, J.C.H. Spence, C. Koch: Iterative phase retrieval without support, Opt. Lett. 29, 2737 (2004)CrossRefGoogle Scholar
  58. J. Wu, J.C.H. Spence, M. O'Keeffe, T. Groy: Application of a modifed Oszlányi and Süto ab initio charge flipping algorithm to experimental data, Acta Crystallogr. A 60, 326 (2004)CrossRefGoogle Scholar
  59. S. Hau-Riege, A. Szoke, H. Chapman, S. Marchesini, A. Noy, H. He, M. Howells, U. Weierstall, J. Spence: SPEDEN: Reconstructing single particles from their diffraction patterns, Acta Crystallogr. A 60, 294 (2004)CrossRefGoogle Scholar
  60. J.C.H. Spence, U. Weierstall, T.T. Fricke, R.M. Glaeser, K.H. Downing: Three-dimensional diffractive imaging for crystalline monolayers with one-dimensional compact support, J. Struct. Biol. 144, 209 (2003)CrossRefGoogle Scholar
  61. S. Eisebitt, M. Lorgen, W. Eberhardt: Scalable approach for lensless imaging at x-ray wavelengths, App. Phys. Lett. 84, 3373 (2004)CrossRefGoogle Scholar
  62. G.J. Williams, M.A. Pfeifer, I.A. Vartanyants, I. Robinson: Three-dimensional imaging of microstructure in Au nanocrystals, Phys. Rev. Lett. 90, 175501 (2003)CrossRefGoogle Scholar
  63. T. Beetz, M. Howells, C. Jacobsen, C. Koa, J. Kirz, E. Lima, T. Mentes, H. Miao, C. Sanchez-Hanke, D. Sayre, D. Shapiro: Apparatus for x-ray diffraction microscopy and tomography of cryo specimens, Nucl. Instrum. Methods Phys. Res. A 545, 459 (2005)CrossRefGoogle Scholar
  64. H. He, S. Marchesini, M. Howells, U. Weierstall, G. Hembree, J.C.H. Spence: Experimental lensless soft-x-ray imaging using iterative algorithms: Phasing diffuse scattering, Acta Crystallogr. A 59, 143 (2003)CrossRefGoogle Scholar
  65. R. Collier, C.B. Burkhadt, L.H. Lin: Optical Holography (Academic Press, New York 1971)Google Scholar
  66. M. Saliba, J. Bosgra, A.D. Parsons, U.H. Wagner, C. Rau, P. Thibault: Novel methods for hard x-ray holographic lensless imaging, Microsc. Microanal. 22(S3), 110–111 (2016)CrossRefGoogle Scholar
  67. S. Marchesini, S. Boutet, A. Sakdinawat, M. Bogan, S. Bajt, A. Barty, J.C.H. Spence, H.N. Chapman: Massively parallel x-ray holography, Nat. Photonics 2, 560 (2008)CrossRefGoogle Scholar
  68. T. Wang, D. Zhu, B. Wu, C. Graves, S. Schaffert, T. Rander, L. Müller, B. Vodungbo, C. Baumier, D.P. Bernstein, B. Bräuer, V. Cros, S. de Jong, R. Delaunay, A. Fognini, R. Kukreja, S. Lee, V. López-Flores, J. Mohanty, B. Pfau, H. Popescu, M. Sacchi, A.B. Sardinha, F. Sirotti, P. Zeitoun, M. Messerschmidt, J.J. Turner, W.F. Schlotter, O. Hellwig, R. Mattana, N. Jaouen, F. Fortuna, Y. Acremann, C. Gutt, H.A. Dürr, E. Beaurepaire, C. Boeglin, S. Eisebitt, G. Grübel, J. Lüning, J. Stöhr, A.O. Scherz: Femtosecond single-shot imaging of nanoscale ferromagnetic order in Co/Pd multilayers using resonant x-ray holography, Phys. Rev. Lett. 108, 267403 (2008)CrossRefGoogle Scholar
  69. J. Wu, U. Weierstall, J.C.H. Spence: Diffractive electron imaging of nanoparticles on a substrate, Nat. Mater. 4, 912 (2005)CrossRefGoogle Scholar
  70. J. Miao, T. Ishikawa, E. Johnson, B. Lai, K. Hodgson: High resolution 3-D x-ray diffraction microscopy, Phys. Rev. Lett. 89, 088303 (2002)CrossRefGoogle Scholar
  71. H. Chapman, A. Barty, M. Bogan, S. Boutet, M. Frank, S. Hau-Riege, S. Marchesini, B. Woods, S. Bajt, W. Benner, R. London: Femtosecond diffractive imaging with a soft-X-ray free-electron laser, Nat. Phys. 2, 839 (2006)CrossRefGoogle Scholar
  72. J.C.H. Spence: High Resolution Electron Microscopy, 4th edn. (Oxford Univ. Press, New York 2017)Google Scholar
  73. H. He, U. Weierstall, J. Spence, M. Howells, H. Padmore, S. Marchesini, H. Chapman: Use of extended and prepared reference objects in experimental Fourier transform x-ray holography, App. Phys. Lett. 85, 2454 (2004)CrossRefGoogle Scholar
  74. J. Frank, M. Radermacher, P. Penczek, J. Zhu, Y. Li, M. Ladjadj, A. Leith: SPIDER and WEB: Processing and visualization of images in 3-D electron microscopy and related fields, J. Struct. Biol. 116, 190 (1996)CrossRefGoogle Scholar
  75. J. Frank: Electron Tomography (Plenum, New York 1992)CrossRefGoogle Scholar
  76. A. Rose: Television pickup tubes and the problem of noise, Adv. Electron. 1, 131 (1948)Google Scholar
  77. M. Howells, T. Beetz, H. Chapman, C. Cui, J. Holton, C. Jacobsen, J. Kirz, E. Lima, S. Marchesini, H. Miao, D. Sayre, D. Shapiro, J.C.H. Spence: An assessment of the resolution limitation due to radiation-damage in x-ray diffraction microscopy, J. Electron Spectrosc. Relat. Phenom. 170, 4 (2009)CrossRefGoogle Scholar
  78. R. Henderson: The potential and limitations of neutrons, electrons and x-rays for atomic resolution microscopy of unstained biological molecules, Q. Rev. Biophys. 28, 171 (1995)CrossRefGoogle Scholar
  79. L. Reimer: Transmission Electron Microscopy (Springer, New York 1989)CrossRefGoogle Scholar
  80. D. Shapiro, P. Thibault, T. Beetz, V. Elser, M. Howells, C. Jacobsen, J. Kirz, E. Lima, H. Miao, A.M. Neiman, D. Sayre: Biological imaging by soft x-ray diffraction microscopy, Proc. Natl. Acad. Sci. U.S.A. 105, 15343 (2005)CrossRefGoogle Scholar
  81. A. Levi, H. Stark: Image restoration by the method of generalized projections with application to restoration from magnitude, J. Opt. Soc. Am. A 1, 932 (1984)CrossRefGoogle Scholar
  82. K.Y.J. Zhang, P. Main: Histogram matching as a new density modification technique for phase refinement and extension of protein molecules, Acta Crystallogr. A 46, 41 (1990)CrossRefGoogle Scholar
  83. R. Luke: Relaxed averaged alternating reflections for diffraction imaging, Inverse Probl. 21, 37–50 (2005)CrossRefGoogle Scholar
  84. M. Howells, P. Charalambous, H. He, S. Marchesini, J.C.H. Spence: An off-axis zone-plate monochromator for high-power undulator radiation. In: Design and Microfabrication of Novel X-ray Optics, Proc. SPIE, Vol. 4783, ed. by D. Mancini (SPIE, Bellingham 2002), https://doi.org/10.1117/12.453809CrossRefGoogle Scholar
  85. S. Marchesini, H.N. Chapman, A. Barty, C. Cui, M.R. Howells, J.C.H. Spence, U. Weierstall, A.M. Minor: Phase aberrations in diffraction microscopy, arXiv:physics/0510033 [physics.optics] (2005)Google Scholar
  86. R. Neutze, R. Wouts, D. van der Spoel, E. Weckert, J. Hajdu: Potential for biomolecular imaging with femtosecond x-ray pulses, Nature 406, 752 (2000)CrossRefGoogle Scholar
  87. K. Pande, C.D.M. Hutchison, G. Groenhof, A. Aquila, J.S. Robinson, J. Tenboer, S. Basu, S. Boutet, D.P. DePonte, M. Liang, T.A. White, N.A. Zatsepin, O. Yefanov, D. Morozov, D. Oberthuer, C. Gati, G. Subramanian, D. James, Y. Zhao, J. Koralek, J. Brayshaw, C. Kupitz, C. Conrad, S. Roy-Chowdhury, J.D. Coe, M. Metz, P.L. Xavier, T.D. Grant, J.E. Koglin, G. Ketawala, R. Fromme, V. Šrajer, R. Henning, J.C.H. Spence, A. Ourmazd, P. Schwander, U. Weierstall, M. Frank, P. Fromme, A. Barty, H.N. Chapman, K. Moffat, J.J. van Thor, M. Schmidt: Femtosecond structural dynamics drives the trans/cis isomerization in photoactive yellow protein, Science 352, 725 (2016)CrossRefGoogle Scholar
  88. J.C.H. Spence: X-ray lasers in biology: Structure and dynamics. In: Advances in Imaging and Electron Physics, Vol. 200, ed. by P. Hawkes (Elsevier, New York 2017) pp. 103–152Google Scholar
  89. C. Bostedt, S. Boutet, D.M. Fritz, Z.R. Huang, H.J. Lee, H.T. Lemke, A. Robert, W.F. Schlotter, J.J. Turner, G.J. Williams: Linac coherent light source: The first five years, Rev. Mod. Phys. 88(1), 015007 (2016)CrossRefGoogle Scholar
  90. I. Schlichting: Serial femtosecond crystallography: The first five years, IUCrJ 2, 246 (2015)CrossRefGoogle Scholar
  91. J.C.H. Spence: XFELs for structure and dynamics in biology, IUCrJ 4, 322 (2017)CrossRefGoogle Scholar
  92. C.H. Yoon, P. Schwander, C. Abergel, I. Andersson, J. Andreasson, A. Aquila, S. Bajt, M. Barthelmess, A. Barty, M.J. Bogan, C. Bostedt, J. Bozek, H.N. Chapman, J.-M. Claverie, N. Coppola, D.P. DePonte, T. Ekeberg, S.W. Epp, B. Erk, H. Fleckenstein, L. Foucar, H. Graafsma, L. Gumprecht, J. Hajdu, C.Y. Hampton, A. Hartmann, E. Hartmann, R. Hartmann, G. Hauser, H. Hirsemann, P. Holl, S. Kassemeyer, N. Kimmel, M. Kiskinova, M. Liang, N.D. Loh, L. Lomb, F.R.N.C. Maia, A.V. Martin, K. Nass, E. Pedersoli, C. Reich, D. Rolles, B. Rudek, A. Rudenko, I. Schlichting, J. Schulz, M. Seibert, V. Seltzer, R.L. Shoeman, R.G. Sierra, H. Soltau, D. Starodub, J. Steinbrener, G. Stier, L. Strüder, M. Svenda, J. Ullrich, G. Weidenspointner, T.A. White, C. Wunderer, A. Ourmazd: Unsupervised classification of single-particle x-ray diffraction snapshots by spectral clustering, Opt. Express 19, 16542–16549 (2011)CrossRefGoogle Scholar
  93. A. Hosseinizadeh, G. Mashayekhi, J. Copperman, P. Schwander, A. Dashti, R. Sepehr, R. Fung, M. Schmidt, C.H. Yoon, B.G. Hogue, G. Williams, A. Aquila, A. Ourmazd: Conformational landscape of a virus by single-particle x-ray scattering, Nat. Methods 14, 877 (2017)CrossRefGoogle Scholar
  94. N.T.D. Loh, V. Elser: Reconstruction algorithm for single-particle diffraction imaging experiments, Phys. Rev. E 80(2), 026705 (2009)CrossRefGoogle Scholar
  95. F.J. Sigworth: Maximum-likelihood approach to the particle alignment problem, Biophys. J. 74, A226 (1998)Google Scholar
  96. K. Ayyer, T.-Y. Lan, V. Elser, N.D. Loh: Dragonfly: An implementation of the expand–maximize–compress algorithm for single-particle imaging, J. Appl. Crystallogr. 49(4), 1320 (2016)CrossRefGoogle Scholar
  97. K. Ayyer, H.T. Philipp, M.W. Tate, V. Elser, S.M. Gruner: Real-space x-ray tomographic reconstruction of randomly oriented objects with sparse data frames, Opt. Express 22, 2403 (2014)CrossRefGoogle Scholar
  98. K.E. Schmidt, J.C.H. Spence, U. Weierstall, R. Kirian, X. Wang, D. Starodub, H.N. Chapman, M.R. Howells, R.B. Doak: Tomographic femtosecond x-ray diffractive imaging, Phys. Rev. Lett. 101, 115507 (2008)CrossRefGoogle Scholar
  99. M. Bergh, G. Huldt, N. Tîmneanu, F.R.N.C. Maia, J. Hajdu: Feasibility of imaging living cells at subnanometer resolutions by ultrafast x-ray diffraction, Q. Rev. Biophys. 41, 181 (2008)CrossRefGoogle Scholar
  100. U. Weierstall, J.C.H. Spence, R.B. Doak: Injector for scattering measurements on fully solvated biospecies, Rev. Sci. Instrum. 83, 035108 (2012)CrossRefGoogle Scholar
  101. F.R.N.C. Maia, T.A. White, N.D. Loh, J. Hajdu: CCP-FEL: A collection of computer programs for free-electron laser research, J. Appl. Crystallogr. 49(4), 1117–1120 (2016)CrossRefGoogle Scholar
  102. A. Munke, J. Andreasson, A. Aquila, S. Awel, K. Ayyer, A. Barty, R.J. Bean, P. Berntsen, J. Bielecki, S. Boutet, M. Bucher, H. Chapman, B.J. Daurer, H. DeMirci, V. Elser, P. Fromme, J. Hajdu, M. Hantke, A. Higashiura, B.G. Hogue, A. Hosseinizadeh, Y. Kim, R.A. Kirian, H.K.N. Reddy, T.-Y. Lan, D.S.D. Larsson, H. Liu, N.D. Loh, F.R.N.C. Maia, A.P. Mancuso, K. Mühlig, A. Nakagawa, D. Nam, G. Nelson, C. Nettelblad, K. Okamoto, A. Ourmazd, M. Rose, G. van der Schot, P. Schwander, M.M. Seibert, J.A. Sellberg, R.G. Sierra, C. Song, M. Svenda, N. Timneanu, I.A. Vartanyants, D. Westphal, M.O. Wiedorn, G.J. Williams, P.L. Xavier, C.H. Yoon, J. Zook: Coherent diffraction of single Rice Dwarf virus particles using hard x-rays at the Linac Coherent Light Source, Sci. Data 3, 160064 (2016)CrossRefGoogle Scholar
  103. H.K.N. Reddy, C.H. Yoon, A. Aquila, S. Awel, K. Ayyer, A. Barty, P. Berntsen, J. Bielecki, S. Bobkov, M. Bucher, G.A. Carini, S. Carron, H. Chapman, B. Daurer, H. DeMirci, T. Ekeberg, P. Fromme, J. Hajdu, M.F. Hanke, P. Hart, B.G. Hogue, A. Hosseinizadeh, Y. Kim, R.A. Kirian, R.P. Kurta, D.S.D. Larsson, N.D. Loh, F.R.N.C. Maia, A.P. Mancuso, K. Mühlig, A. Munke, D. Nam, C. Nettelblad, A. Ourmazd, M. Rose, P. Schwander, M. Seibert, J.A. Sellberg, C. Song, J.C.H. Spence, M. Svenda, G. van der Schot, I.A. Vartanyants, G.J. Williams, P.L. Xavier: Coherent soft x-ray diffraction imaging of coliphage PR772 at the Linac coherent light source, Sci. Data 4, 170079 (2017)CrossRefGoogle Scholar
  104. I.A. Vartanyants, I.K. Robinson, I. McNulty, C. David, P. Wochner, T. Tschentscher: Coherent x-ray scattering and lensless imaging at the European XFEL facility, J. Synchrotron Radiat. 14(6), 453–470 (2007)CrossRefGoogle Scholar
  105. M. Weyland, P. Midgley: Electron tomography. In: Transmission Electron Microscopy (Springer, Cham 2016) pp. 343–376CrossRefGoogle Scholar
  106. B. Enders, P. Thibault: A computational framework for ptychographic reconstructions, Proc. R. Soc. A 472(2196), 20160640 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Dept. of PhysicsArizona State UniversityTempe, AZUSA

Personalised recommendations