Compressed Anomaly Detection with Multiple Mixed Observations

  • Natalie Durgin
  • Rachel Grotheer
  • Chenxi Huang
  • Shuang Li
  • Anna Ma
  • Deanna Needell
  • Jing Qin
Part of the Association for Women in Mathematics Series book series (AWMS, volume 17)


We consider a collection of independent random variables that are identically distributed, except for a small subset which follows a different, anomalous distribution. We study the problem of detecting which random variables in the collection are governed by the anomalous distribution. Recent work proposes to solve this problem by conducting hypothesis tests based on mixed observations (e.g., linear combinations) of the random variables. Recognizing the connection between taking mixed observations and compressed sensing, we view the problem as recovering the “support” (index set) of the anomalous random variables from multiple measurement vectors (MMVs). Many algorithms have been developed for recovering jointly sparse signals and their support from MMVs. We establish the theoretical and empirical effectiveness of these algorithms in detecting anomalies. We also extend the LASSO algorithm to an MMV version for our purpose. Further, we perform experiments on synthetic data, consisting of samples from the random variables, to explore the trade-off between the number of mixed observations per sample and the number of samples required to detect anomalies.



The initial research for this effort was conducted at the Research Collaboration Workshop for Women in Data Science and Mathematics, July 17–21 held at ICERM. Funding for the workshop was provided by ICERM, AWM, and DIMACS (NSF grant CCF-1144502). SL was supported by NSF CAREER grant CCF− 1149225. DN was partially supported by the Alfred P. Sloan Foundation, NSF CAREER #1348721, and NSF BIGDATA #1740325. JQ was supported by NSF DMS-1818374.


  1. 1.
    D. Angelosante, G.B. Giannakis, E. Grossi, Compressed sensing of time-varying signals, in 16th International Conference on Digital Signal Processing, Santorini-Hellas, Greece (2009), pp. 1–8Google Scholar
  2. 2.
    D. Baron, M.B. Wakin, M.F. Duarte, S. Sarvotham, R.G. Baraniuk, Distributed compressed sensing (2005). Preprint available at
  3. 3.
    D. Baron, S. Sarvotham, R.G. Baraniuk, Bayesian compressive sensing via belief propagation. IEEE Trans. Signal Process. 58(1), 269–280 (2010)MathSciNetCrossRefGoogle Scholar
  4. 4.
    M. Basseville, I.V. Nikiforov, Detection of Abrupt Changes: Theory and Application (Prentice-Hall, Upper Saddle River, 1993)zbMATHGoogle Scholar
  5. 5.
    T. Blumensath, M.E. Davies, Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009)MathSciNetCrossRefGoogle Scholar
  6. 6.
    E.J. Candès, Compressive sampling, in Proceedings of the International Congress of Mathematicians, vol. 3, Madrid, Spain (2006), pp. 1433–1452zbMATHGoogle Scholar
  7. 7.
    E.J. Candès, T. Tao, Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)MathSciNetCrossRefGoogle Scholar
  8. 8.
    E.J. Candès, T. Tao, Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)MathSciNetCrossRefGoogle Scholar
  9. 9.
    E.J. Candès, J. Romberg, T. Tao, Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)MathSciNetCrossRefGoogle Scholar
  10. 10.
    J. Chen, X. Huo, Theoretical results on sparse representations of multiple-measurement vectors. IEEE Trans. Signal Process. 54(12), 4634–4643 (2006)CrossRefGoogle Scholar
  11. 11.
    S.S. Chen, D.L. Donoho, M.A. Saunders, Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)MathSciNetCrossRefGoogle Scholar
  12. 12.
    M. Cho, W. Xu, L. Lai, Compressed hypothesis testing: to mix or not to mix? (2016). arXiv preprint arXiv:1609.07528Google Scholar
  13. 13.
    S.F. Cotter, B.D. Rao, K. Engan, K. Kreutz-Delgado, Sparse solutions to linear inverse problems with multiple measurement vectors. IEEE Trans. Signal Process. 53(7), 2477–2488 (2005)MathSciNetCrossRefGoogle Scholar
  14. 14.
    D.L. Donoho, Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)MathSciNetCrossRefGoogle Scholar
  15. 15.
    D.L. Donoho, X. Huo, Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47(7), 2845–2862 (2001)MathSciNetCrossRefGoogle Scholar
  16. 16.
    M.R. Duarte, M.B. Wakin, D. Baron, R.G. Baraniuk, Universal distributed sensing via random projections, in 5th International Conference on Information Processing in Sensor Networks, Nashville, TN (2006), pp. 177–185Google Scholar
  17. 17.
    M.F. Duarte, M.B. Wakin, D. Baron, S. Sarvotham, R.G. Baraniuk, Measurement bounds for sparse signal ensembles via graphical models. IEEE Trans. Inf. Theory 59(7), 4280–4289 (2013)MathSciNetCrossRefGoogle Scholar
  18. 18.
    B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, Least angle regression. Ann. Stat. 32(2), 407–499 (2004)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Y.C. Eldar, G. Kutyniok, Compressed Sensing: Theory and Applications (Cambridge University Press, Cambridge, 2012)CrossRefGoogle Scholar
  20. 20.
    J. Filos, E. Karseras, W. Dai, S. Yan, Tracking dynamic sparse signals with hierarchical Kalman filters: a case study, in 18th International Conference on Digital Signal Processing, Fira, Greece (2013), pp. 1–6Google Scholar
  21. 21.
    S. Foucart, H. Rauhut, A Mathematical Introduction to Compressive Sensing (Birkhäuser, Basel, 2013)Google Scholar
  22. 22.
    J. Haupt, R. Nowak, Signal reconstruction from noisy random projections. IEEE Trans. Inf. Theory 52(9), 4036–4048 (2006)MathSciNetCrossRefGoogle Scholar
  23. 23.
    S. Ji, Y. Xue, L. Carin, Bayesian compressive sensing. IEEE Trans. Signal Process. 56(6), 2346–2356 (2008)MathSciNetCrossRefGoogle Scholar
  24. 24.
    S.S. Keerthi, S. Shevade, A fast tracking algorithm for generalized LARS/LASSO. IEEE Trans. Neural Netw. 18(6), 1826–1830 (2007)CrossRefGoogle Scholar
  25. 25.
    L. Lai, Y. Fan, H.V. Poor, Quickest detection in cognitive radio: a sequential change detection framework, in IEEE Global Telecommunications Conference, New Orleans, LO (2008), pp. 1–5Google Scholar
  26. 26.
    S.G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41(12), 3397–3415 (1993)CrossRefGoogle Scholar
  27. 27.
    M. Malloy, R. Nowak, On the limits of sequential testing in high dimensions, in 45th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA (2011), pp. 1245–1249Google Scholar
  28. 28.
    M. Malloy, R. Nowak, Sequential analysis in high-dimensional multiple testing and sparse recovery, in IEEE International Symposium on Information Theory, St. Petersburg, Russia (2011), pp. 2661–2665Google Scholar
  29. 29.
    M.L. Malloy, G. Tang, R.D. Nowak, Quickest search for a rare distribution, in 46th Annual Conference on Information Sciences and Systems, Princeton, NJ (2012), pp. 1–6Google Scholar
  30. 30.
    M. Mishali, Y.C. Eldar, Reduce and boost: recovering arbitrary sets of jointly sparse vectors. IEEE Trans. Signal Process. 56(10), 4692–4702 (2008)MathSciNetCrossRefGoogle Scholar
  31. 31.
    D. Needell, R. Vershynin, Signal recovery from incomplete and inaccurate measurements via regularized orthogonal matching pursuit. IEEE J. Sel. Topics Signal Process. 4(2), 310–316 (2010)CrossRefGoogle Scholar
  32. 32.
    Y.C. Pati, R. Rezaiifar, P.S. Krishnaprasad, Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition, in 27th Asilomar Conference on Signals, Systems and Computers, vol. 1, Pacific Grove, CA (1993), pp. 40–44Google Scholar
  33. 33.
    S. Patterson, Y.C. Eldar, I. Keidar, Distributed compressed sensing for static and time-varying networks. IEEE Trans. Signal Process. 62(19), 4931–4946 (2014)MathSciNetCrossRefGoogle Scholar
  34. 34.
    H.V. Poor, O. Hadjiliadis, Quickest Detection (Cambridge University Press, Cambridge, 2008)CrossRefGoogle Scholar
  35. 35.
    J.A. Tropp, Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theory 52(3), 1030–1051 (2006)MathSciNetCrossRefGoogle Scholar
  36. 36.
    J.A. Tropp, A.C. Gilbert, Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53(12), 4655–4666 (2007)MathSciNetCrossRefGoogle Scholar
  37. 37.
    J.A. Tropp, A.C. Gilbert, M.J. Strauss, Simultaneous sparse approximation via greedy pursuit, in IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 5, Philadelphia, PA (2005), pp. 721–724Google Scholar
  38. 38.
    E. van den Berg, M.P. Friedlander, Theoretical and empirical results for recovery from multiple measurements. IEEE Trans. Inf. Theory 56(5), 2516–2527 (2010)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Y. Xia, D. Tse, Inference of link delay in communication networks. IEEE J. Sel. Areas Commun. 24(12), 2235–2248 (2006)CrossRefGoogle Scholar
  40. 40.
    G. Yu, G. Sapiro, Statistical compressed sensing of Gaussian mixture models. IEEE Trans. Signal Process. 59(12), 5842–5858 (2011)MathSciNetCrossRefGoogle Scholar

Copyright information

© The Author(s) and the Association for Women in Mathematics 2019

Authors and Affiliations

  • Natalie Durgin
    • 1
  • Rachel Grotheer
    • 2
  • Chenxi Huang
    • 3
  • Shuang Li
    • 4
  • Anna Ma
    • 5
  • Deanna Needell
    • 6
  • Jing Qin
    • 7
  1. 1.SpiceworksAustinUSA
  2. 2.Goucher CollegeBaltimoreUSA
  3. 3.Yale UniversityNew HavenUSA
  4. 4.Colorado School of MinesGoldenUSA
  5. 5.Claremont Graduate UniversityClaremontUSA
  6. 6.University of CaliforniaLos AngelesUSA
  7. 7.Montana State UniversityBozemanUSA

Personalised recommendations