Euler-Poincaré Characteristic for the Oscillator Representation

Abstract

Suppose (G, G′) is a dual pair of subgroups of a metaplectic group. The dual pair correspondence is a bijection between (subsets of the) irreducible representations of G and G′, defined by the non-vanishing of Hom(ω, π ×π′), where ω is the oscillator representation. Alternatively one considers Hom_G(ω, π) as a G′-module. It is fruitful to replace Hom with Extⁱ, and general considerations suggest that the Euler-Poincare characteristic EP(ω, π), the alternating sum of Extⁱ(ω, π), will be a more elementary object. We restrict to the case of p-adic groups, and prove that EP(ω, π) is a well defined element of the Grothendieck group of finite length representations of G′, and show that it is indeed more elementary than Hom(ω, π). We expect that computation of EP, together with vanishing results for higher Ext groups, will be a useful tool in computing the dual pair correspondence, and will help to elucidate the structure of Hom(ω, π).

Dedicated to Roger Howe on his 70th birthday

The first author was supported in part by NSF grant DMS-1317523.

The second author held the Jean-Morlet Chaire at CIRM, Marseille during part of this work.

The third author was supported in part by NSF grant DMS-1359774.