Abstract
Large amount of data being generated at large scale facilities like European X-ray Free- Electron Laser (XFEL) requires new approaches for data processing and analysis. One of the most computationally challenging experiments at an XFEL is single-particle structure determination. In this paper we propose a new design for an integrated software platform which combines well-established techniques for XFEL data analysis with High Performance Data Analysis (HPDA) methods. In our software platform we use streaming data analysis algorithms with high performance computing solutions. This approach should allow analysis of the experimental dataflow in quasi-online regime.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. Osseyran and M. Giles, Industrial Applications of High-Performance Computing: Best Global Practices (Chapman and Hall/CRC, Boca Raton, 2015).
K. J. Gaffney and H. N. Chapman, “Imaging atomic structure and dynamics with ultrafast X-ray scattering,” Science (Washington, DC, U. S.) 316 (5830), 1444–1448 (2007).
R. Neutze, R. Wouts, D. van der Spoel, E. Weckert, and J. Hajdu, “Potential for biomolecular imaging with femtosecond X-ray pulses,” Nature (London, U. K.) 406 (6797), 752 (2000).
M. Altarelli, R. Brinkmann, M. Chergui, W. Decking, B. Dobson, S. Düsterer, G. Grübel, W. Graeff, H. Graafsma, J. Hajdu, et al., in The European X-Ray Free-Electron Laser, Technical Design Report, No. 97, 1–26 (2006).
A. Aquila, A. Barty, C. Bostedt, S. Boutet, G. Carini, D. DePonte, P. Drell, et al., “The linac coherent light source single particle imaging road map,” Struct. Dyn. 2, 041701 (2015).
H. N. Chapman, A. Barty, M. J. Bogan, S. Boutet, M. Frank, S. P. Hau-Riege, S. Marchesini, B. W. Woods, S. Bajt, W. H. Benner, et al., “Femtosecond diffractive imaging with a soft-X-ray free-electron laser,” Nat. Phys. 2, 839 (2006).
M. M. Seibert, T. Ekeberg, F. Maia, M. Svenda, J. Andreasson, O. Jönsson, D. Odić, B. Iwan, A. Rocker, D. Westphal, et al., “Single mimivirus particles intercepted and imaged with an X-ray laser,” Nature (London, U. K.) 470 (7332), 78 (2011).
M. F. Hantke, D. Hasse, F. R. N. C. Maia, T. Ekeberg, K. John, M. Svenda, N. Duane Loh, A. V. Martin, N. Timneanu, D. S. D. Larsson, et al., “High-throughput imaging of heterogeneous cell organelles with an X-ray laser,” Nat. Photon. 8, 943 (2014).
T. Ekeberg, M. Svenda, C. Abergel, F. Maia, V. Seltzer, J. Claverie, M. Hantke, O. Jönsson, C. Nettelblad, G. Van Der Schot, et al., “Three-dimensional reconstruction of the giant mimivirus particle with an x-ray free-electron laser,” Phys. Rev. Lett. 114, 098102 (2015).
R. P. Kurta, J. J. Donatelli, C. H. Yoon, P. Berntsen, J. Bielecki, B. J. Daurer, H. DeMirci, P. Fromme, M. Hantke, F. Maia, et al., “Correlations in scattered x-ray laser pulses reveal nanoscale structural features of viruses,” Phys. Rev. Lett. 119, 158102 (2017).
A. Barty, R. A. Kirian, F. R. N. C. Maia,M. Hantke, C. H. Yoon, T. A. White, and H. Chapman, “Cheetah: software for high-throughput reduction and analysis of serial femtosecond X-ray diffraction data,” J. Appl. Cryst. 37, 1118 (2014).
K. Ayyer, T. Lan, V. Elser, and N. D. Loh, “Dragonfly: an implementation of the expand–maximize–compress algorithm for single-particle imaging,” Int. J. Appl. Cryptogr. 49, 1320–1335 (2016).
F. R. N. C. Maia, T. Ekeberg, D. van der Spoel, and J. Hajdu, “Hawk: the image reconstruction package for coherent x-ray diffractive imaging,” J. Appl. Cryst. 43, 1535 (2010).
B. Henrich, J. Becker, R. Dinapoli, P. Goettlicher, H. Graafsma, H. Hirsemann, R. Klanner, H. Krueger, R. Mazzocco, A. Mozzanica, et al., “The adaptive gain integrating pixel detector AGIPD a detector for the European XFEL,” Nucl. Instrum. Methods Phys. Res., Sect. A 633, S11–S14 (2011).
S. A. Bobkov, “Comparison study of different approaches to classification of diffraction images of biological particles obtained in coherent X-ray diffractive imaging experiments,” Math. Biol. Bioinform. 12, 411–434 (2017).
F. Maia, “The coherent X-ray imaging data bank,” Nat. Methods 9, 854–855 (2012).
B. J. Daurer, K. Okamoto, J. Bielecki, F. Maia, K. Mühlig, M. Seibert, M. Hantke, C. Nettelblad, W. H. Benner, M. Svenda, et al., “Experimental strategies for imaging bioparticles with femtosecond hard X-ray pulses,” Int. Union Cryst. J. 4, 854–855 (2017).
J. Andreasson, A. V. Martin, M. Liang, N. Timneanu, A. Aquila, F. Wang, B. Iwan, M. Svenda, T. Ekeberg, M. Hantke, et al., “Automated identification and classification of single particle serial femtosecond X-ray diffraction data,” Opt. Express 22, 2497–2510 (2014).
S. A. Bobkov, A. B. Teslyuk, R. P. Kurta, O. Yu. Gorobtsov, O. M. Yefanov, V. A. Ilyin, R. A. Senin, and I. A. Vartanyants, “Sorting algorithms for single-particle imaging experiments at X-ray free-electron lasers,” J. Synchrotr. Rad. 22, 1345–1352 (2015).
R. P. Kurta, M. Altarelli, and I. A. Vartanyants, “Structural analysis by x-ray intensity angular cross correlations,” Adv. Chem. Phys. 161, 1–39 (2016).
N. D. Loh and V. Elser, “Reconstruction algorithm for single-particle diffraction imaging experiments,” Phys. Rev.. 80, 026705 (2009).
A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc., Ser. B, 1–38 (1977).
M. Kevin, Machine Learning: A Probabilistic Perspective (MIT Press, Cambridge,MA, 2012).
R. M. Neal and G. E. Hinton, “A view of the EM algorithm that justifies incremental, sparse, and other variants,” in Learning in GraphicalModels (MIT Press, Cambridge,MA, 1998).
M. A. Sato and S. Ishii, “On-line EMalgorithm for the normalized Gaussian network,” Neural Comput. 12, 407–432 (2000).
O. Cappeánd E. Moulines, “On-line expectation–maximization algorithmfor latent datamodels,” J. R. Stat. Soc., Ser. B 71, 593–613 (2009).
G. Celeux, D. Chauveau, and J. Diebolt, “On stochastic versions of the EM algorithm,” Research Report RR-2514 (INRIA, 1995).
S. F. Nielsen et al., “The stochastic EMalgorithm: estimation and asymptotic results,” Bernoulli 6, 457–489 (2000).
G. Wei and M. A. Tanner, “A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms,” J. Am. Statist. Assoc. 85 (411), 699–704 (1990).
J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt. 21, 2758 (1982).
S. Marchesini, “A unified evaluation of iterative projection algorithms for phase retrieval,” Rev. Sci. Instrum. 78, 011301 (2007).
G. Behrmann et al., “A distributed storage system with dCache,” J. Phys.: Conf. Ser. 119, 062014 (2008).
F. Maia et al., “Hawk: the image reconstruction package for coherent X-ray,” J. Appl. Crystallogr. 43, 1535–1539 (2010).
Author information
Authors and Affiliations
Corresponding author
Additional information
(Submitted by A. V. Lapin)
Rights and permissions
About this article
Cite this article
Bobkov, S.A., Teslyuk, A.B., Zolotarev, S.I. et al. Software Platform for European XFEL: Towards Online Experimental Data Analysis. Lobachevskii J Math 39, 1170–1178 (2018). https://doi.org/10.1134/S1995080218090093
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1995080218090093