Two major current trends are the “geometrization” and “probabilization” of technical and scientific fields. Born in the mid-20th century, information geometry lies at their intersection, via the Fisher–Koszul–Souriau metric (see FOCUS). It introduces a distance between elements of a probability space endowed with a metric structure. This distance can be defined between two independent random variables or, equivalently, between two probability densities associated with them. The Japanese mathematician Shun-ichi Amari (born 1936), who developed the field’s tools with the Russian mathematician Nicolai N. Chentsov, received the Order of Culture from the Emperor in 2019.
A worldwide success --------------------
Information geometry is currently enjoying worldwide success. A Google search for “Information Geometry” returns 186,000 results! In France, the CNRS ISIS research group (Information, signal, image and vision) organized a summer school in Peyresq (Alpes-de-Haute-Provence) in 2019 that was so popular it had to turn applicants away. The Centre international de mathématiques et d’informatique (CIMI), a laboratory of excellence, and the Institut mathématique de Toulouse (Haute-Garonne) devoted an entire three-month period to the subject. The proceedings of the 2013, 2015, 2017 and 2019 Geometric Science of Information conferences fill four volumes of nearly 900 pages each, published by Springer, which has also launched a specialist journal devoted to the subject.
Information geometry is used in particular to understand “latent spaces” in deep learning and to build generative models. It has become a popular AI tool in industry: the GAFAMI companies (Google, Amazon, Facebook, Apple, Microsoft and IBM) use the natural gradient associated with the Fisher metric in their deep-learning tools.
Yann Ollivier, a recipient of the CNRS Bronze Medal, was recently hired by Facebook AI Paris, notably for his research on the natural gradient. Other mathematicians, such as Google DeepMind employees Guillaume Desjardins and James Martens and IBM Research’s Takayuki Osogami, have adapted information geometry’s natural gradient to deep learning. Gaetan Marceau Caron, a former doctoral student at Thales who was recently hired by the Montréal Institute for Learning Algorithms (MILA) in Canada, combined the natural gradient with stochastic-gradient Langevin dynamics to define a “natural Langevin dynamics” gradient, which can regularize the training of deep networks. In fact, machine learning is inherently highly geometric.