Selberg and Ruelle zeta functions for non-unitary twists

Abstract

In this paper we study the dynamical zeta functions of Ruelle and Selberg associated with the geodesic flow of a compact hyperbolic odd-dimensional manifold. These are functions of a complex variable s in some right half-plane of $\mathbb {C}$. Using the Selberg trace formula for arbitrary finite dimensional representations of the fundamental group of the manifold, we establish the meromorphic continuation of the dynamical zeta functions to the whole complex plane. We explicitly describe the singularities of the Selberg zeta function in terms of the spectrum of certain twisted Laplace and Dirac operators.

Appendix

In Sects. 5 and 6, we define the twisted Bochner–Laplace and Dirac operator, respectively. These operators are associated with a non-unitary representation of the subgroup $\varGamma$ and are no longer self-adjoint. However, the twisted operators $\varDelta _{\tau ,\chi }^{\sharp }$ and $D^{\sharp }_{\chi }(\sigma )^{2}$ have the same principal symbol as the operators $\varDelta _{\tau }\otimes {{\mathrm{Id}}}_{V_{\chi }}$ and $D(\sigma )^{2}\otimes {{\mathrm{Id}}}_{V_{\chi }}$, respectively, i.e.,

$$\begin{aligned}&\sigma _{\varDelta _{\tau ,\chi }^\sharp }(x,\xi )=||\xi ||^ {2}{{\mathrm{Id}}}_{({V_{\tau }\otimes V_{\chi })_{x}}},\\&\sigma _{D^{\sharp }_{\chi }(\sigma )^{2}}(x,\xi )=||\xi ||^{2}{{\mathrm{Id}}}_{(V_{\tau _{s}(\sigma )}\otimes V_{\chi })_{x}}, \end{aligned}$$

where $x\in X$, $\xi \in {{\mathrm{T}}}_{x}^{*}X,\xi \ne 0$, We include here the the spectral theory of non-self-adjoint operators, which is needed to develop the trace formulas and further to provide the proofs of the meromorphic continuations of the twisted dynamical zeta functions.

Setting 8.1

Let $E\rightarrow X$ be a complex vector bundle over a smooth compact Riemannian manifold X of dimension d. Let $D:C^{\infty }(X,E)\rightarrow C^{\infty }(X,E)$ be an elliptic differential operator of order $m\ge 1$. Let $\sigma _{D}$ be its principal symbol.

Definition 8.2

A spectral cut is a ray

$$\begin{aligned} R_{\theta }:=\left\{ \rho e^ {i\theta }: \rho \in [0,\infty ]\right\} , \end{aligned}$$

where $\theta \in [0,2\pi )$.

Definition 8.3

The angle $\theta$ is a principal angle for an elliptic operator D if

$$\begin{aligned} {{\mathrm{spec}}}(\sigma _{D}(x,\xi ))\cap R_{\theta }=\emptyset ,\quad \forall x\in X,\forall \xi \in T_{x}^{*}X,\xi \ne 0. \end{aligned}$$

Definition 8.4

We define the solid angle $L_{I}$ associated with a closed interval I of $\mathbb {R}$ by

$$\begin{aligned} L_{I}:=\{\rho e^ {i\theta }: \rho \in (0,\infty ), \theta \in I \}. \end{aligned}$$

Definition 8.5

The angle $\theta$ is an Agmon angle for an elliptic operator D, if it is a principal angle for D and there exists $\varepsilon >0$ such that

$$\begin{aligned} {{\mathrm{spec}}}(D)\cap L_{[\theta -\varepsilon ,\theta +\varepsilon ]}=\emptyset . \end{aligned}$$

Lemma 8.6

Let $\varepsilon$ be an angle such that the principal symbol $\sigma _{D}(x,\xi )$ of D, for $\xi \in T_{x}^{*}X,\xi \ne 0$, does not take values in $L_{[-\varepsilon ,\varepsilon ]}$. Then, the spectrum ${{\mathrm{spec}}}(D)$ of the operator D is discrete and for every $\varepsilon \in (0,\frac{\pi }{2})$ there exists $R>0$ such that ${{\mathrm{spec}}}(D)$ is contained in the set $B(0,R)\cup L_{[-\varepsilon ,\varepsilon ]}\subset \mathbb {C}$.

Proof

The discreteness of the spectrum follows from [26, Theorem 8.4]. For the second statement see [26, Theorem 9.3].

Let $\lambda _{k}$ be an eigenvalue of D and $V_{\lambda _{k}}$ be the corresponding eigenspace. This is a finite dimensional subspace of $C^{\infty }(X,E)$ invariant under D. We have that for every $k\in \mathbb {N}$, there exist $N_{k}\in \mathbb {N}$ such that

$$\begin{aligned}&(D-\lambda _{k}{{\mathrm{Id}}})^{N_{k}}V_{\lambda _{k}}=0;\\&\lim _{k\rightarrow \infty }|\lambda _{k}|=\infty . \end{aligned}$$

By [16] the space $L^{2}(X,E)$ can be decomposed as

$$\begin{aligned} L^{2}(X,E)=\overline{\bigoplus _{k\ge 1}V_{\lambda _{k}}}. \end{aligned}$$

This is the generalization of the eigenspace decomposition of a self-adjoint operator .

Definition 8.7

We call algebraic multiplicity $m(\lambda _{k})$ of the eigenvalue $\lambda _{k}$ the dimension of the corresponding eigenspace $V_{\lambda _{k}}$.

Fig. 1

The discrete spectrum of the operator $D^{\sharp }_{\chi }(\sigma )^{2}$

By Lemma 8.6, there exists $\varepsilon >0$ such that

$$\begin{aligned} {{\mathrm{spec}}}(D^{\sharp }_{\chi }(\sigma )^{2})\cap L_{[\theta -\varepsilon ,\theta +\varepsilon ]}=\emptyset . \end{aligned}$$

Since $D^{\sharp }_{\chi }(\sigma )^{2}$ has discrete spectrum (Fig. 1), there exists also an $r_{0}>0$ such that

$$\begin{aligned} {{\mathrm{spec}}}(D^{\sharp }_{\chi }(\sigma )^{2})\cap \{z\in \mathbb {C}:|z+1|\le 2r_{0}\}=\emptyset . \end{aligned}$$

We define a contour $\varGamma _{\theta ,r_{0}}$ as follows.

$$\begin{aligned} \varGamma _{\theta ,r _{0}}=\varGamma _{1}\cup \ \varGamma _{2}\cup \varGamma _{3}, \end{aligned}$$

where $\varGamma _{1}=\{-1+re^ {i\theta }:\infty >r\ge r_{0}\}$, $\varGamma _{2}=\{-1+r_{0}e^ {ia}:\theta \le a\le \theta +2\pi \}$, $\varGamma _{3}=\{-1+re^{i(\theta +2\pi )}:r_{0}\le r< \infty \}$. On $\varGamma _{1}$, r runs from $\infty$ to $r_{0}$, $\varGamma _{2}$ is oriented counterclockwise, and on $\varGamma _{3}$, r runs from $r_{0}$ to $\infty$. We put

$$\begin{aligned} e^{-tD^{\sharp }_{\chi }(\sigma )^{2}}=\frac{i}{2\pi }\int _{\varGamma _{\theta ,r_{0}}}e^{-t\lambda }\big (D^{\sharp }_{\chi }(\sigma )^{2}- \lambda {{\mathrm{Id}}}\big )^{-1} \mathrm{d}\lambda . \end{aligned}$$

(8.1)

We have $|e^{-t\lambda }|\le e^{-t\text {Re}(\lambda )}$. Furthermore, by [26, Corollary 9.2], there exists a positive constant $c>0$ such that $||(D^{\sharp }_{\chi }(\sigma )^{2}-\lambda {{\mathrm{Id}}})^{-1}||\le c |\lambda |^{-1}$, for $\lambda \in \varGamma _{\theta ,r_{0}}$. Hence, the integral in (8.1) are well defined.

Given an Agmon angle $\theta$ for the operator $A_{\chi }^{\sharp }(\sigma )$ (see equations (5.24)–(5.26), p. 29) and $r_{0}>0$, we consider a contour $\varGamma _{\theta ,r_{0}}$ in the same way as for the operator $D^{\sharp }_{\chi }(\sigma )^{2}$. Then, we put

$$\begin{aligned} e^{-tA_{\chi }^{\sharp }(\sigma )}=\frac{i}{2\pi }\int _{\varGamma _{\theta ,r_{0}}}e^{-t\lambda }(A_{\chi }^{\sharp }(\sigma )-\lambda {{\mathrm{Id}}})^{-1} \mathrm{d}\lambda . \end{aligned}$$

(8.2)

By [26, Corollary 9.2] and the fact that $|e^{-t\lambda }|\le e^{-t\text {Re}(\lambda )}$, the integral in (8.2) is well defined.