Ptychography

Ptychography (/t(ʌ)ɪˈkogræfi/ t(a)i-KO-graf-ee)^{[citation needed]} is a computational method of microscopic imaging.^[1] It generates images by processing many coherent interference patterns that have been scattered from an object of interest. Its defining characteristic is translational invariance, which means that the interference patterns are generated by one constant function (e.g. a field of illumination or an aperture stop) moving laterally by a known amount with respect to another constant function (the specimen itself or a wave field). The interference patterns occur some distance away from these two components, so that the scattered waves spread out and "fold" (Ancient Greek: πτύξ is 'fold'^[2]) into one another as shown in the figure.

Ptychography can be used with visible light, X-rays, extreme ultraviolet (EUV) or electrons. Unlike conventional lens imaging, ptychography is unaffected by lens-induced aberrations or diffraction effects caused by limited numerical aperture^{[citation needed]}. This is particularly important for atomic-scale wavelength imaging, where it is difficult and expensive to make good-quality lenses with high numerical aperture. Another important advantage of the technique is that it allows transparent objects to be seen very clearly. This is because it is sensitive to the phase of the radiation that has passed through a specimen, and so it does not rely on the object absorbing radiation. In the case of visible-light biological microscopy, this means that cells do not need to be stained or labelled to create contrast.

Although the interference patterns used in ptychography can only be measured in intensity, the mathematical constraints provided by the translational invariance of the two functions (illumination and object), together with the known shifts between them, means that the phase of the wavefield can be recovered by an inverse computation. Ptychography thus provides a comprehensive solution to the so-called "phase problem". Once this is achieved, all the information relating to the scattered wave (modulus and phase) has been recovered, and so virtually perfect images of the object can be obtained. There are various strategies for performing this inverse phase-retrieval calculation, including direct Wigner distribution deconvolution (WDD)^[3] and iterative methods.^{[4][5][6][7][8]} The difference map algorithm developed by Thibault and co-workers^[7] is available in a downloadable package called PtyPy.^[9]

There are many optical configurations for ptychography: mathematically, it requires two invariant functions that move across one another while an interference pattern generated by the product of the two functions is measured. The interference pattern can be a diffraction pattern, a Fresnel diffraction pattern or, in the case of Fourier ptychography, an image. The "ptycho" convolution in a Fourier ptychographic image derived from the impulse response function of the lens.

This is conceptually the simplest ptychographical arrangement.^[10] The detector can either be a long way from the object (i.e. in the Fraunhofer diffraction plane), or closer by, in the Fresnel regime. An advantage of the Fresnel regime is that there is no longer a very high-intensity beam at the centre of the diffraction pattern, which can otherwise saturate the detector pixels there.

A lens is used to form a tight crossover of the illuminating beam at the plane of the specimen. The configuration is used in the scanning transmission electron microscope (STEM),^[11][12] and often in high-resolution X-ray ptychography. The specimen is sometimes shifted up or downstream of the probe crossover so as to allow the size of the patch of illumination to be increased, thus requiring fewer diffraction patterns to scan a wide field of view.

This uses a wide field of illumination. To provide magnification, a diverging beam is incident on the specimen. An out-of-focus image, which appears as a Fresnel interference pattern, is projected onto the detector. The illumination must have phase distortions in it, often provided by a diffuser that scrambles the phase of the incident wave before it reaches the specimen, otherwise the image remains constant as the specimen is moved, so there is no new ptychographical information from one position to the next.^[13] In the electron microscope, a lens can be used to map the magnified Fresnel image onto the detector.

A conventional microscope is used with a relatively small numerical aperture objective lens. The specimen is illuminated from a series of different angles. Parallel beams coming out of the specimen are brought to a focus in the back focal plane of the objective lens, which is therefore a Fraunhofer diffraction pattern of the specimen exit wave (Abbe’s theorem). Tilting the illumination has the effect of shifting the diffraction pattern across the objective aperture (which also lies in the back focal plane). Now the standard ptychographical shift invariance principle applies, except that the diffraction pattern is acting as the object and the back focal plane stop is acting like the illumination function in conventional ptychography. The image is in the Fraunhofer diffraction plane of these two functions (another consequence of Abbe's theorem), just like in conventional ptychography. The only difference is that the method reconstructs the diffraction pattern, which is much wider than the aperture stop limitation. A final Fourier transform must be undertaken to produce the high-resolution image. All the reconstruction algorithms used in conventional ptychography apply to Fourier ptychography, and indeed nearly all the diverse extensions of conventional ptychography have been used in Fourier ptychography.^[14]

A lens is used to make a conventional image. An aperture in the image plane acts equivalently to the illumination in conventional ptychography, while the image corresponds to the specimen. The detector lies in the Fraunhofer or Fresnel diffraction plane downstream of the image and aperture.^[15]

This geometry can be used either to map surface features or to measure strain in crystalline specimens. Shifts in the specimen surface, or the atomic Bragg planes perpendicular to the surface, appear in the phase of the ptychographic image.^[16][17][18]

Vectorial ptychography needs to be invoked when the multiplicative model of the interaction between the probe and the specimen cannot be described by scalar quantities.^[19] This happens typically when polarized light probes an anisotropic specimen, and when this interaction modifies the state of polarization of light. In that case, the interaction needs to be described by the Jones formalism,^[20] where field and object are described by a two-component complex vector and a 2×2 complex matrix respectively. The optical configuration for vectorial ptychography is similar to that of classical (scalar) ptychography, although a control of light polarization (before and after the specimen) needs to be implemented in the setup. Jones maps of the specimens can be retrieved, allowing the quantification of a wide range of optical properties (phase, birefringence, orientation of neutral axes, diattenuation, etc.).^[21] Similarly to scalar ptychography, the probes used for the measurement can be jointly estimated together with the specimen.^[22] As a consequence, vectorial ptychography is also an elegant approach for quantitative imaging of coherent vectorial light beams (mixing wavefront and polarization features).^[23]

Ptychography can be undertaken without using any lenses at all,^[10][13] although most implementations use a lens of some type, if only to condense radiation onto the specimen. The detector can measure high angles of scatter, which do not need to pass through a lens. The resolution is therefore only limited by the maximal angle of scatter that reaches the detector, and so avoids the effects of diffraction broadening due to a lens of small numerical aperture or aberrations within the lens. This is key in X-ray, electron and EUV ptychography, where conventional lenses are difficult and expensive to make.

Ptychography solves for the phase induced by the real part of the refractive index of the specimen, as well as absorption (the imaginary part of the refractive index). This is crucial for seeing transparent specimens that do not have significant natural absorption contrast, for example biological cells (at visible light wavelengths),^[24] thin high-resolution electron microscopy specimens,^[25] and almost all materials at hard X-ray wavelengths. In the latter case, the (linear) phase signal is also ideal for high-resolution X-ray ptychographic tomography.^[26] The strength and contrast of the phase signal also means that far fewer photon or electron counts are needed to make an image: this is very important in electron ptychography, where damage to the specimen is a major issue that must be avoided at all costs.^[27]

Unlike holography, ptychography uses the object itself as an interferometer. It does not require a reference beam. Although holography can solve the image phase problem, it is very difficult to implement in the electron microscope, where the reference beam is extremely sensitive to magnetic interference or other sources of instability. This is why ptychography is not limited by the conventional "information limit" in conventional electron imaging.^[28] Furthermore, ptychographical data is sufficiently diverse to remove the effects of partial coherence that would otherwise affect the reconstructed image.^[3][29]

The ptychographical data set can be posed as a blind deconvolution problem.^[7][8][30] It has sufficient diversity to solve for both the moving functions (illumination and object), which appear symmetrically in the mathematics of the inversion process. This is now routinely done in any ptychographical experiment, even if the illumination optics have been previously well characterised. Diversity can also be used to solve retrospectively for errors in the offsets of the two functions, blurring in the scan, detector faults, like missing pixels, etc.

In conventional imaging, multiple scattering in a thick sample can seriously complicate, or even entirely invalidate, simple interpretation of an image. This is especially true in electron imaging (where multiple scattering is called "dynamical scattering"). Conversely, ptychography generates estimates of hundreds or thousands of exit waves, each of which contains different scattering information. This can be used to retrospectively remove multiple scattering effects.^[31]

The number counts required for a ptychography experiment is the same as for a conventional image, even though the counts are distributed over very many diffraction patterns. This is because dose fractionation applies to ptychography. Maximum-likelihood methods can be employed to reduce the effects of Poisson noise.^[32]

Applications of ptychography are diverse because it can be used with any type of radiation that can be prepared as a quasi-monochromatic propagating wave.

Ptychographic imaging, along with advances in detectors and computing, has resulted in the development of X-ray microscopes.^[33][34] Coherent beams are required in order to obtain far-field diffraction patterns with speckle patterns. Coherent X-ray beams can be produced by modern synchrotron radiation sources, free-electron lasers and high-harmonic sources. In terms of routine analysis, X-ray ptychotomography^[26] is today the most commonly used technique. It has been applied to many materials problems including, for example, the study of paint,^[35] imaging battery chemistry,^[36] imaging stacked layers of tandem solar cells,^[37] and the dynamics of fracture.^[38] In the X-ray regime, ptychography has also been used to obtain a 3D mapping of the disordered structure in the white Cyphochilus beetle,^[39] and a 2D imaging of the domain structure in a bulk heterojunction for polymer solar cells.^[40]

Visible-light ptychography has been used for imaging live biological cells and studying their growth, reproduction and motility.^[41] In its vectorial version, it can also be used for mapping quantitative optical properties of anisotropic materials such as biominerals^[21] or metasurfaces^[42]

Electron ptychography is uniquely (amongst other electron imaging modes) sensitive to both heavy and light atoms simultaneously. It has been used, for example, in the study of nanostructure drug-delivery mechanisms by looking at drug molecules stained by heavy atoms within light carbon nanotubes cages.^[12] With electron beams, shorter-wavelength, higher-energy electrons used for higher-resolution imaging can cause damage to the sample by ionising it and breaking bonds, but electron-beam ptychography has now produced record-breaking images of molybdenum disulphide with a resolution of 0.039 nm using a lower-energy electron beam and detectors that are able to detect single electrons, so atoms can be located with more precision.^[27][43]

Ptychography has several applications in the semiconductor industry, including imaging their surfaces using EUV,^[44] their 3D bulk structure using X-rays,^[45] and mapping strain fields by Bragg ptychography, for example, in nanowires.^[46]

The name "ptychography" was coined by Hegerl and Hoppe in 1970^[48] to describe a solution to the crystallographic phase problem first suggested by Hoppe in 1969.^[49] The idea required the specimen to be highly ordered (a crystal) and to be illuminated by a precisely engineered wave so that only two pairs of diffraction peaks interfere with one another at a time. A shift in the illumination changes the interference condition (by the Fourier shift theorem). The two measurements can be used to solve for the relative phase between the two diffraction peaks by breaking a complex-conjugate ambiguity that would otherwise exist.^[50] Although the idea encapsulates the underlying concept of interference by convolution (ptycho) and translational invariance, crystalline ptychography cannot be used for imaging of continuous objects, because very many (up to millions) of beams interfere at the same time, and so the phase differences are inseparable. Hoppe abandoned his concept of ptychography in 1973.

Between 1989 and 2007 Rodenburg and co-workers developed various inversion methods for the general imaging ptychographic phase problem, including Wigner-distribution deconvolution (WDD),^[3] SSB,^[11] the "PIE" iterative method^[4] (a precursor of the "ePIE" algorithm^[8]), demonstrating proof-of-principles at various wavelengths.^[11][51][52] Chapman used the WDD inversion method to demonstrate the first implementation of X-ray ptychography in 1996.^[53] The smallness of computers and poor quality of detectors at that time may account for the fact that ptychography was not at first taken up by other workers.

Widespread interest in ptychography only started after the first demonstration of iterative phase-retrieval X-ray ptychography in 2007 at the Swiss Light Source (SLS).^[52] Progress at X-ray wavelengths was then quick. By 2010, the SLS had developed X-ray ptychotomography,^[26] now a major application of the technique. Thibault, also working at the SLS, developed the difference-map (DM) iterative inversion algorithm and mixed-state ptychography.^[7][29] Since 2010, several groups have developed the capabilities of ptychography to characterize and improve reflective^[54] and refractive X-ray optics.^[55][56] Bragg ptychography, for measuring strain in crystals, was demonstrated by Hruszkewycz in 2012.^[16] In 2012 it was also shown that electron ptychography could improve on the resolution of an electron lens by a factor of five,^[57] a method which was used in 2018 to provide the highest-resolution transmission image ever obtained^[27] earning a Guinness world record,^[58] and once again in 2021 to achieve an even better resolution.^[59][60][61] Real-space light ptychography became available in a commercial system for live-cell imaging in 2013.^[24] Fourier ptychography using iterative methods was also demonstrated by Zheng et al.^[14] in 2013, a field which is growing rapidly. The group of Margaret Murnane and Henry Kapteyn at JILA, CU Boulder demonstrated EUV reflection ptychographic imaging in 2014.^[17]

• Phase retrieval
• Computational imaging
• Fourier ptychography

• Cornell researchers see atoms at record resolution, cornell.edu at 20 May 2021