Abstract
The properties of divergences in dually flat spaces are investigated beyond the local perspective. First, an approximative representation of divergence is shown on the basis of the expansion with respect to relative coordinates. A consideration concerning triangular relation is done from similar perspective and an equation which extends classical multidimensional scaling to incorporate Riemannian metric and affine connection coefficients is obtained. Furthermore, a consideration from a viewpoint of probability distribution is done. The representation of divergence can be regarded as Kullback–Leibler divergence between normal distributions with a common covariance matrix. Especially, it is shown that the asymmetry of divergence arises from the spatial change in differential entropy in those distributions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In the following, Einstein notation is also used for Roman letters as \(\langle \theta , \eta \rangle = \theta ^i \eta _i\) and derivatives are also written as \(\theta ^i = \partial \phi / \partial \eta _i = \partial ^i \phi \), \(\eta _i = \partial \psi / \partial \theta ^i = \partial _i \psi \). Furthermore, we identify specific points by Greek letters \(\iota \), \(\kappa \) etc. and use them as indices like \(\theta _\iota \), \(\eta ^\iota \) etc., representing a point \(\iota \) as a vector \(\theta _\iota = (\theta ^1_\iota \cdots \theta ^n_\iota )^\prime \) or a dual vector \(\eta ^\iota = (\eta ^\iota _1 \cdots \eta ^\iota _n)^\prime \). Einstein notation is not applied for these Greek letters.
H would be written as \(H = [ \log \det G^{-1} + n ( 1 + \log 2 \pi ) ]/2\) for n dimensional space without the abbreviation.
References
Ackermann, M., Blömer, J., Scholz, C.: Hardness and non-approximability of Bregman clustering problems. Technical report, Electronic colloquium on computational complexity (2011). http://eccc.hpi-web.de/report/2011/015/
Amari, S., Nagaoka, H.: Methods of Information Geometry. American Mathematical Society, Providence (2001)
Banerjee, A., Merugu, S., Dhillon, I.S., Ghosh, J.: Clustering with Bregman divergences. J. Mach. Learn. Res. 6, 1705–1749 (2005)
Bishop, C.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Borg, I., Groenen, P.: Modern Multidimensional Scaling, 2nd edn. Springer, New York (2005)
Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7, 200–217 (1967)
Brun, A., Knutsson, H.: Tensor Glyph Warping: Visualizing metric tensor fields using Riemannian exponential maps, In: Laidlaw, D. H., Weickert, J. (eds.) Visualization and Processing of Tensor Fields, pp. 139–160. Springer, Berlin Heidelberg (2009)
Chino, N., Shiraiwa, K.: Geometrical structures of some non-distance models for asymmetric MDS. Behaviormetrika 20, 35–47 (1993)
Chino, N.: A brief survey of asymmetric MDS and some open problems. Behaviormetrika 39, 127–165 (2012)
Cox, T.F., Cox, M.A.A.: Multidimensional Scaling, 2nd edn. Chapman and Hall/CRC, Boca Raton (2000)
Escoufier, Y., Grorud, A.: Analyse factorielle des matrices carées non-symétriques. In: Diday, E. (ed.) Data Analysis and Informatics, pp. 263–276. North-Holland, Amsterdam (1980)
Krumhansl, C.L.: Concerning the applicability of geometric models to similarity data: the interrelationship between similarity and spatial density. Psychol. Rev. 85, 445–463 (1987)
Kumagai, A.: Extension of classical MDS to treat dissimilarities not satisfying axioms of distance. Jpn. J. Ind. Appl. Math. 31, 111–124 (2014)
Kumagai, A.: Multidimensional scaling in dually flat spaces. Jpn. J. Indust. Appl. Math. 32, 51–63 (2015)
MacAdam, D.L.: Visual sensitivities to color differences in daylight. J. Opt. Soc. Am. 32, 247–273 (1942)
Nielsen, F., Nock, R.: Clustering multivariate normal distributions. Lect. Notes Comput. Sci. 5416, 164–174 (2009)
Okada, A., Imaizumi, T.: Nonmetric multidimensional scaling of asymmetric proximities. Behaviormetrika 21, 81–96 (1987)
Saito, T., Yadohisa, H.: Data Analysis of Asymmetric Structures. Marcel Dekker, New York (2005)
Torgerson, W.S.: Theory and Methods of Scaling. Wiley, New York (1958)
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Kumagai, A. Semilocal properties of canonical divergences in dually flat spaces. Japan J. Indust. Appl. Math. 33, 417–426 (2016). https://doi.org/10.1007/s13160-016-0219-7
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13160-016-0219-7