Abstract
Low-dimensional embeddings have emerged as a key component in modern signal processing theory and practice. In particular, embeddings transform signals in a way that preserves their geometric relationship but makes processing more convenient. The literature has, for the most part, focused on lowering the dimensionality of the signal space while preserving distances between signals. However, there has also been work exploring the effects of quantization, as well as on transforming geometric quantities, such as distances and inner products, to metrics easier to compute on modern computers, such as the Hamming distance.Embeddings are particularly suited for modern signal processing applications, in which the fidelity of information represented by the signals is of interest, instead of the fidelity of the signal itself. Most typically, this information is encoded in the relationship of the signal to other signals and templates, as encapsulated in the geometry of the signal space. Thus, embeddings are very good tools to capture the geometry, while reducing the processing burden.In this chapter, we provide a concise overview of the area, including foundational results and recent developments. Our goal is to expose the field to a wider community, to provide, as much as possible, a unifying view of the literature, and to demonstrate the usefulness and applicability of the results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Technically, we could incorporate g(⋅ ) into \(d_{\mathscr{S}}(\cdot,\cdot )\) and remove it from this definition. However, we choose to make it explicit here and consider it a distortion to be explicitly analyzed. In an abuse of nomenclature, we generally refer to d(⋅ , ⋅ ) as distance, even if in some cases it is not strictly a distance metric but might be an inner product, or another geometric quantity of interest.
References
D. Achlioptas, Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66, 671–687 (2003)
D. Achlioptas, F. Mcsherry, B. Schölkopf, Sampling techniques for kernel methods, in Advances in Neural Information Processing Systems (2002), pp. 335–342
A. Ai, A. Lapanowski, Y. Plan, R. Vershynin, One-bit compressed sensing with non-gaussian measurements. Linear Algebra Appl. 441, 222–239 (2014)
N. Ailon, B. Chazelle, Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform, in Proceedings of the Thirty-Eighth Annual ACM Symposium on Theory of Computing (2006), pp. 557–563
A. Andoni, P. Indyk, Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Commun. ACM 51(1), 117–122 (2008). DOI:10.1145/1327452.1327494
A. Andoni, M. Deza, A. Gupta, P. Indyk, S. Raskhodnikova, Lower bounds for embedding edit distance into normed spaces, in Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms (2003), pp. 523–526
A.S. Bandeira, D.G. Mixon, B. Recht, Compressive classification and the rare eclipse problem (2014), arXiv preprint arXiv:1404.3203
Z. Bar-Yossef, T. Jayram, R. Krauthgamer, R. Kumar, Approximating edit distance efficiently, in Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science, 2004 (IEEE, Los Alamitos, 2004), pp. 550–559
R. Baraniuk, M. Wakin, Random projections of smooth manifolds. Found. Comput. Math. 9(1), 51–77 (2009)
R. Baraniuk, M. Davenport, R. DeVore, M. Wakin, A simple proof of the restricted isometry property for random matrices. Const. Approx. 28(3), 253–263 (2008)
R. Baraniuk, V. Cevher, M. Duarte, C. Hegde, Model-based compressive sensing. IEEE Trans. Inf. Theory 56(4), 1982–2001 (2010)
T. Blumensath, M. Davies, Sampling theorems for signals from the union of finite-dimensional linear subspaces. IEEE Trans. Inf. Theory 55(4), 1872–1882 (2009)
P.T. Boufounos, Universal rate-efficient scalar quantization. IEEE Trans. Inf. Theory 58(3), 1861–1872 (2012). DOI:10.1109/TIT.2011.2173899
P.T. Boufounos, Angle-preserving quantized phase embeddings, in Proceedings of SPIE Wavelets and Sparsity XV, San Diego, CA (2013)
P.T. Boufounos, On embedding the angles between signals, in Signal Processing with Adaptive Sparse Structured Representations, Lausanne, Switzerland (2013)
P.T. Boufounos, Sparse signal reconstruction from phase-only measurements, in Proceedings of International Conference on Sampling Theory and Applications, Bremen, Germany (2013)
P.T. Boufounos, H. Mansour, Universal embeddings for kernel machine classification, in Proceedings of Sampling Theory and Applications, Washington, DC (2015)
P.T. Boufounos, S. Rane, Secure binary embeddings for privacy preserving nearest neighbors, in Proceedings of the IEEE Workshop on Information Forensics and Security, Foz do Iguau, Brazil (2011). DOI:10.1109/WIFS.2011.6123149
P.T. Boufounos, S. Rane, Efficient coding of signal distances using universal quantized embeddings, in Proceedings of Data Compression Conference, Snowbird, UT (2013)
P.T. Boufounos, S. Rane, H. Mansour, Representation and coding of signal geometry (2015), arXiv preprint arXiv:1512.07636
J. Bourgain, S. Dirksen, J. Nelson, Toward a unified theory of sparse dimensionality reduction in euclidean space. Geom. Funct. Anal. 25(4), 1009–1088 (2015)
B. Brinkman, M. Charikar, On the impossibility of dimension reduction in l 1. J. ACM 52(5), 766–788 (2005)
E. Candès, The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci. I 346(9–10), 589–592 (2008)
E.J. Candes, J.K. Romberg, T. Tao, Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
S. Dasgupta, A. Gupta, An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algoritm. 22(1), 60–65 (2003)
M. Datar, N. Immorlica, P. Indyk, V.S. Mirrokni, Locality-sensitive hashing scheme based on p-stable distributions, in Proceedings of the Twentieth Annual Symposium on Computational Geometry (ACM, New York, 2004), pp. 253–262
M.A. Davenport, P.T. Boufounos, M.B. Wakin, R.G. Baraniuk, Signal processing with compressive measurements. IEEE J. Sel. Top. Sign. Proces. 4(2), 445–460 (2010). DOI:10.1109/JSTSP.2009.2039178. http://dx.doi.org/10.1109/JSTSP.2009.2039178
S. Dirksen, Dimensionality reduction with subgaussian matrices: a unified theory. Found. Comput. Math. 16(5), 1367–1396
Y. Eldar, M. Mishali, Robust recovery of signals from a structured union of subspaces. IEEE Trans. Inf. Theory 55(11), 5302–5316 (2009)
J. Haupt, R. Nowak, A generalized restricted isometry property. Tech. rep., University of Wisconsin-Madison (2007)
C. Hegde, A. Sankaranarayanan, W. Yin, R. Baraniuk, NuMax: a convex approach for learning near-isometric linear embeddings. IEEE Trans. Signal Process. 63(22), 6109–6121 (2015). DOI:10.1109/TSP.2015.2452228
P. Indyk, Stable distributions, pseudorandom generators, embeddings, and data stream computation. J. ACM 53(3), 307–323 (2006)
P. Indyk, R. Motwani, Approximate nearest neighbors: towards removing the curse of dimensionality, in ACM Symposium on Theory of computing (1998), pp. 604–613
L. Jacques, A quantized Johnson-Lindenstrauss lemma: the finding of buffon’s needle. IEEE Trans. Inf. Theory 61(9), 5012–5027 (2015). DOI:10.1109/TIT.2015.2453355
L. Jacques, D.K. Hammond, J.M. Fadili, Dequantizing compressed sensing: when oversampling and non-gaussian constraints combine. IEEE Trans. Inf. Theory 57(1), 559–571 (2011)
L. Jacques, D.K. Hammond, J.M. Fadili, Stabilizing nonuniformly quantized compressed sensing with scalar companders. IEEE Trans. Inf. Theory 59(12), 7969–7984 (2013)
L. Jacques, J.N. Laska, P.T. Boufounos, R.G. Baraniuk, Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Trans. Inf. Theory 59(4) (2013). DOI:10.1109/TIT.2012.2234823. http://dx.doi.org/10.1109/TIT.2012.2234823
T. Jayram, D.P. Woodruff, Optimal bounds for Johnson-Lindenstrauss transforms and streaming problems with subconstant error. ACM Trans. Algoritm. 9(3), 26 (2013)
A. Jimenez, B. Raj, J. Portelo, I. Trancoso, Secure modular hashing, in IEEE International Workshop on Information Forensics and Security (IEEE, Piscataway, 2015), pp. 1–6
W. Johnson, J. Lindenstrauss, Extensions of Lipschitz mappings into a Hilbert space. Contemp. Math. 26, 189–206 (1984)
F. Krahmer, R. Ward, New and improved Johnson-Lindenstrauss embeddings via the restricted isometry property. SIAM J. Math. Anal. 43(3), 1269–1281 (2011)
V.I. Levenshtein, Binary codes capable of correcting deletions, insertions, and reversals. Sov. Phys. Dokl. 10(8), 707–710 (1966)
M. Li, S. Rane, P.T. Boufounos, Quantized embeddings of scale-invariant image features for mobile augmented reality, in Processing of the IEEE International Workshop on Multimedia Signal Processing, Banff, Canada (2012)
R. Ostrovsky, Y. Rabani, Low distortion embeddings for edit distance. J. ACM 54(5), 23 (2007)
D. Otero, G.R. Arce, Generalized restricted isometry property for alpha-stable random projections, in IEEE International Conference on Acoustics, Speech and Signal Processing (2011), pp. 3676–3679
S. Oymak, B. Recht, Near-optimal bounds for binary embeddings of arbitrary sets (2015), arXiv preprint arXiv:1512.04433
Y. Plan, R. Vershynin, One-bit compressed sensing by linear programming. Commun. Pure Appl. Math. 66(8), 1275–1297 (2013)
Y. Plan, R. Vershynin, Robust 1-bit compressed sensing and sparse logistic regression: a convex programming approach. IEEE Trans. Inf. Theory 59(1), 482–494 (2013)
Y. Plan, R. Vershynin, Dimension reduction by random hyperplane tessellations. Discret. Comput. Geom. 51(2), 438–461 (2014)
G. Puy, M. Davies, R. Gribonval, Recipes for stable linear embeddings from Hilbert spaces to \(\mathbb{R}^{m}\) (2015), arXiv preprint arXiv:1509.06947
A. Rahimi, B. Recht, Random features for large-scale kernel machines, in Advances in Neural Information Processing Systems (2007), pp. 1177–1184
S. Rane, P.T. Boufounos, Privacy-preserving nearest neighbor methods: comparing signals without revealing them, in IEEE Signal Processing Magazine (2013). DOI:10.1109/MSP.2012.2230221, http://dx.doi.org/10.1109/MSP.2012.2230221
S. Rane, P.T. Boufounos, A. Vetro, Quantized embeddings: an efficient and universal nearest neighbor method for cloud-based image retrieval, in Proceedings of SPIE Applications of Digital Image Processing XXXVI, San Diego, CA (2013)
A. Sadeghian, B. Bah, V. Cevher, Energy-aware adaptive bi-Lipschitz embeddings, in Proceedings of International Conference on Sampling Theory and Applications, Bremen, Germany (2013)
C. Strecha, A. Bronstein, M. Bronstein, P. Fua, LDAHash: improved matching with smaller descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 66–78 (2012). DOI:10.1109/TPAMI.2011.103
Y. Weiss, A. Torralba, R. Fergus, Spectral hashing, in Advances in Neural Information Processing Systems 21 (MIT, London, 2009), pp. 1753–1760
X. Yi, C. Caramanis, E. Price, Binary embedding: fundamental limits and fast algorithm, in Proceedings of the 32nd International Conference on Machine Learning, Lille, France, vol. 37 (2015), pp. 2162–2170
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Boufounos, P.T., Rane, S., Mansour, H. (2017). Embedding-Based Representation of Signal Geometry. In: Balan, R., Benedetto, J., Czaja, W., Dellatorre, M., Okoudjou, K. (eds) Excursions in Harmonic Analysis, Volume 5. Applied and Numerical Harmonic Analysis. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-54711-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-54711-4_7
Published:
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-319-54710-7
Online ISBN: 978-3-319-54711-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)