Training Auto-Encoder-Based Optimizers for Terahertz Image Reconstruction

  • Tak Ming WongEmail author
  • Matthias Kahl
  • Peter Haring-Bolívar
  • Andreas Kolb
  • Michael Möller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11824)


Terahertz (THz) sensing is a promising imaging technology for a wide variety of different applications. Extracting the interpretable and physically meaningful parameters for such applications, however, requires solving an inverse problem in which a model function determined by these parameters needs to be fitted to the measured data. Since the underlying optimization problem is nonconvex and very costly to solve, we propose learning the prediction of suitable parameters from the measured data directly. More precisely, we develop a model-based autoencoder in which the encoder network predicts suitable parameters and the decoder is fixed to a physically meaningful model function, such that we can train the encoding network in an unsupervised way. We illustrate numerically that the resulting network is more than 140 times faster than classical optimization techniques while making predictions with only slightly higher objective values. Using such predictions as starting points of local optimization techniques allows us to converge to better local minima about twice as fast as optimizing without the network-based initialization.


  1. 1.
    Amos, B., Kolter, J.Z.: OptNet: differentiable optimization as a layer in neural networks. In: Proceedings of International Conference on Machine Learning (2017)Google Scholar
  2. 2.
    Andrychowicz, M., et al.: Learning to learn by gradient descent by gradient descent. In: Proceedings of International Conference on Neural Information Processing Systems (NIPS) (2016)Google Scholar
  3. 3.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: Proceedings of SIGGRAPH, pp. 187–194. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA (1999).
  4. 4.
    Chan, W.L., Deibel, J., Mittleman, D.M.: Imaging with terahertz radiation. Rep. Prog. Phys. 70(8), 1325 (2007)CrossRefGoogle Scholar
  5. 5.
    Chang, J.H., Li, C.L., Poczos, B., Kumar, B.V., Sankaranarayanan, A.: One network to solve them all – solving linear inverse problems using deep projection models. In: Proceedings of IEEE International Conference on Computer Vision (2017)Google Scholar
  6. 6.
    Coleman, T.F., Li, Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6(2), 418–445 (1996)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Cooper, K.B., Dengler, R.J., Llombart, N., Thomas, B., Chattopadhyay, G., Siegel, P.H.: THz imaging radar for standoff personnel screening. IEEE Trans. Terahertz Sci. Technol. 1(1), 169–182 (2011)CrossRefGoogle Scholar
  8. 8.
    Ding, J., Kahl, M., Loffeld, O., Haring Bolívar, P.: THz 3-D image formation using sar techniques: simulation, processing and experimental results. IEEE Trans. Terahertz Sci.Technol. 3(5), 606–616 (2013)CrossRefGoogle Scholar
  9. 9.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). Scholar
  10. 10.
    Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 315–323 (2011)Google Scholar
  11. 11.
    Heckel, R., Hand, P.: Deep decoder: Concise image representations from untrained non-convolutional networks. In: International Conference on Learning Representations (2019)Google Scholar
  12. 12.
    Hu, B.B., Nuss, M.C.: Imaging with terahertz waves. Opt. Lett. 20(16), 1716–1718 (1995)CrossRefGoogle Scholar
  13. 13.
    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of International Conference on Machine Learning (2015)Google Scholar
  14. 14.
    Jansen, C., Wietzke, S., Peters, O., Scheller, M., Vieweg, N., Salhi, M., Krumbholz, N., Jördens, C., Hochrein, T., Koch, M.: Terahertz imaging: applications and perspectives. Appl. Opt. 49(19), E48–E57 (2010)CrossRefGoogle Scholar
  15. 15.
    Kahl, M., et al.: Stand-off real-time synthetic imaging at mm-wave frequencies. In: Passive and Active Millimeter-Wave Imaging XV. vol. 8362, p. 836208 (2012)Google Scholar
  16. 16.
    Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)Google Scholar
  17. 17.
    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  18. 18.
    Kobler, E., Klatzer, T., Hammernik, K., Pock, T.: Variational networks: connecting variational methods and deep learning. In: Roth, V., Vetter, T. (eds.) GCPR 2017. LNCS, vol. 10496, pp. 281–293. Springer, Cham (2017). Scholar
  19. 19.
    Long, Z., Wang, T., You, C., Yang, Z., Wang, K., Liu, J.: Terahertz image super-resolution based on a deep convolutional neural network. Appl. Opt. 58(10), 2731–2735 (2019)CrossRefGoogle Scholar
  20. 20.
    McClatchey, K., Reiten, M., Cheville, R.: Time resolved synthetic aperture terahertz impulse imaging. Appl. Phys. Lett. 79(27), 4485–4487 (2001)CrossRefGoogle Scholar
  21. 21.
    Meinhardt, T., Moeller, M., Hazirbas, C., Cremers, D.: Learning proximal operators: using denoising networks for regularizing inverse imaging problems. In: Proceedings of IEEE International Conference on Computer Vision (2017)Google Scholar
  22. 22.
    Moeller, M., Möllenhoff, T., Cremers, D.: Controlling neural networks via energy dissipation (2019).
  23. 23.
    Munson, D.C., Visentin, R.L.: A signal processing view of strip-mapping synthetic aperture radar. IEEE Trans. Acoust. Speech Signal Process. 37(12), 2131–2147 (1989)CrossRefGoogle Scholar
  24. 24.
    Nah, S., Hyun Kim, T., Mu Lee, K.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3883–3891 (2017)Google Scholar
  25. 25.
    Plötz, T., Roth, S.: Benchmarking denoising algorithms with real photographs. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2017)Google Scholar
  26. 26.
    Schuler, C.J., Hirsch, M., Harmeling, S., Schölkopf, B.: Learning to deblur. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 38(7), 1439–1451 (2016)CrossRefGoogle Scholar
  27. 27.
    Siegel, P.H.: Terahertz technology. IEEE Trans. Microw. Theory Tech. 50(3), 910–928 (2002)CrossRefGoogle Scholar
  28. 28.
    Skolnik, M.I.: Radar Handbook. McGraw-Hill Book Co., New York (1970) Google Scholar
  29. 29.
    Standard, M.: Photographic lenses (1959).
  30. 30.
    Tewari, A., et al.: MoFA: model-based deep convolutional face autoencoder for unsupervised monocular reconstruction. In: Proceedings of IEEE International Conference on Computer Vision (2017)Google Scholar
  31. 31.
    Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  32. 32.
    Wong, T.M., Kahl, M., Haring Bolívar, P., Kolb, A.: Computational image enhancement for frequency modulated continuous wave (FMCW) THz image. J. Infrared Millimeter Terahertz Waves 40(7), 775–800 (2019)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Center for Sensor Systems (ZESS)University of SiegenSiegenGermany
  2. 2.Computer Graphics and Multimedia Systems GroupUniversity of SiegenSiegenGermany
  3. 3.Institute for High Frequency and Quantum Electronics (HQE)University of SiegenSiegenGermany
  4. 4.Computer Vision GroupUniversity of SiegenSiegenGermany

Personalised recommendations