GREEN FUNCTIONS ASSOCIATED TO COMPLEX REFLECTION GROUPS, II

TOSHIAKI SHOJI

Department of Mathematics Science University of Tokyo Noda, Chiba 278-8510, Japan

Abstract. Green functions associated to complex reflection groups G(e, 1, n) were discussed in the author’s previous paper. In this paper, we consider the case of complex reflection groups W = G(e, p, n). Schur functions and Hall-Littlewood functions associated to W are introduced, and Green functions are described as the transition matrix between those two symmetric functions. Furthermore, it is shown that these Green functions are determined by means of Green functions associated to various G(e′, 1, n′). Our result involves, as a special case, a combinatorial approach to the Green functions of type D_{n}.

0. Introduction

This paper is a continuation of [S]. In [S], Hall-Littlewood functions associated to the complex reflection group G(e, 1, n) were introduced. Green functions associated to G(e, 1, n) are defined as a solution of a certain matrix equation arising from the combinatorics of e-symbols. It was shown that such Green functions are obtained as coefficients of the expansion of Schur functions in terms of Hall-Littlewood functions. In the case where e = 2, G(e, 1, n) coincides with the Weyl group of type B_{n}, and the Green function in that case coincides with the Green function associated to finite classical groups Sp_{2n}(F_{q}) or SO_{2n+1}(F_{q}) introduced by Deligne-Lusztig, in a geometric way. So our result is regarded as a first step towards the combinatorial description of such Green functions, just as in the case of Green polynomials of GL_{n}(F_{q}).

In this paper, we take up the complex reflection group G(e, p, n), and show that a similar formalism as in the case of G(e, 1, n) works also for such groups. In the case of G(e, p, n), symmetric functions such as Schur functions, Hall-Littlewood functions, etc. appear as p-tuples of similar functions associated to the various complex reflection groups G(e′, 1, n′). In particular, Green functions associated to G(e, p, n) can be described in terms of Green functions associated to G(e′, 1, n′). In the case where e = p = 2, the group G(e, p, n) is equal to the Weyl group of type D_{n}. In this case our Green functions coincide with the Green functions associated to the finite groups SO_{2n}(F_{q}) of split type or non-split type. So our result implies, in this case, that the Green functions of type D_{n} can be described completely in terms of various “Green functions” of type B_{n′}. However, note that the Green function of type B_{n′} appearing in this context is not the Green function associated to SO_{2n′+1}. They are the functions introduced in [S], associated to different type of symbols.

In the case where n = 2, the group G(e, e, n) is equal to the dihedral group of degree 2e. In this case, our Green function coincides with the function obtained by Geck-Malle [GM] in connection with special pieces and unipotent characters of finite reductive groups (see [L2]).

The author is grateful to Gunter Malle for some explicit computation by using a computer. The tables in section 5 are due to him.

Table of contents

0. Introduction

1. Schur-Weyl reciprocity for G(e, p, n)

2. Frobenius formula for G(e, p, n)

3. Hall-Littlewood functions associated to G(e, p, n)

4. Green functions associated to G(e, p, n)

5. Examples

1. Schur-Weyl reciprocity for G(e, p, n)

1.1. Let W̃ ≃ 𝔖_{n}⋉(ℤ/eℤ)^{n} be the imprimitive complex reflection group G(e, 1, n) acting on the complex vector space ℂ^{n}. Let e_{1}, …, e_{n} be the standard basis of ℂ^{n}. Then W̃ is realized as a subgroup of GL(ℂ^{n}) consisting of w such that w(e_{i}) = ξ_{i}e_{x(i)} for i = 1, …, n, where x ∈ 𝔖_{n} and ξ_{i} is an e-th root of unity (depending on w). We fix a primitive e-th root of unity ζ. Then ξ_{i} = ζ^{a_{i}} with a_{i} ∈ ℤ/eℤ, and w ∈ W̃ can be written as w = x(a_{1}, …, a_{n}) uniquely. The complex reflection group W_{0} = G(e, e, n) is defined as the subgroup of W̃ consisting of w ∈ W̃ such that ∑_{i=1}^{n} a_{i} ≡ 0 (mod e). W_{0} is a normal subgroup of W̃ of index e. Let us define, for k = 0, …, e − 1, t_{k} ∈ W̃ by

t_{k}(e_{i}) = \begin{cases}ζe_{i} & if i = k, \\ e_{i} & otherwise. \end{cases}

Then w = x(a_{1}, …, a_{n}) can be written as w = x ∏_{i=1}^{n} t_{i}^{a_{i}}. Hence if we put σ = t_{1}, W̃ is a semidirect product of W_{0} with the cyclic group of order e generated by σ. For each factor p of e, we put W = W_{0} ⋊⟨σ^{p}⟩. Then W is the subgroup of W̃ consisting of w such that ∑_{i=1}^{n} a_{i} ≡ 0 (mod p), and is isomorphic to the complex reflection group G(e, p, n).

1.2. For a factor p of e, we put d = e/p. Let P_{n,e} be the set of e-tuples α = (α^{(0)}, …, α^{(e−1)}) of partitions such that ∑_{k=0}^{e−1} |α^{(k)}| = n. An element α ∈ P_{n,e} is called an e-partition of n. We put l(α) = ∑_{k=0}^{e−1} l(α^{(k)}), where l(α^{(k)}) denotes the number of parts of the partition α^{(k)}. Let us define an operator θ = θ_{p} on P_{n,e} by θ(α) = (α^{(k−d)}) for α = (α^{(k)}) ∈ P_{n,e}. We denote by c_{α,p} the number of elements in the θ-orbit of α. Then the irreducible representations of W are described as follows. It is known that the set of isomorphism classes of irreducible representations of W̃ is parametrized by P_{n,e}. Let Z_{α} be an irreducible W̃-module corresponding to α ∈ P_{n,e}. Then Z_{α} is decomposed, as a W -module, as

(1.2.1)  Z_{α} = Z_{α,1}^{p} ⊕ ⋯ ⊕ Z_{α,r}^{p},

where r = p/c_{α,p}, and σ permutes factors in a cyclic way. Each Z_{α,i}^{p} is an irreducible W -module, mutually non-isomorphic of the same dimension. Furthermore, if α and β are in the same orbit under θ, then Z_{α} ≃ Z_{β} as W -modules. We denote by χ^{α} the irreducible character of W̃ afforded by Z_{α}, and by χ_{p}^{α,i} the irreducible character of W afforded by Z_{α,i}^{p} for i = 1, …, p/c_{α,p}. We write α ∼_{p} β if α and β are in the same θ-orbit, and by P_{n,e}/∼_{p} the set of θ-orbits in P_{n,e}. Then the set W^{∧} of irreducible characters of W is parametrized as

(1.2.2)  W^{∧} = {χ_{p}^{α,i} | α ∈ P_{n,e}/∼_{p}, 1 ≤ i ≤ p/c_{α,p}}.

1.3. Here we recall the Schur-Weyl reciprocity between W̃ and a certain Levi subgroup of a general linear group over ℂ. Let V = ⨁_{i=0}^{e−1} V_{i} be a vector space over ℂ with dim V_{i} = m_{i}. We fix a basis E = {v_{j}^{(k)} | 1 ≤ j ≤ m_{k}} of V_{k} for 0 ≤ k ≤ e − 1. Then v_{1}^{(0)}, …, v_{m_{0}}^{(0)}, v_{1}^{(1)}, …, v_{m_{1}}^{(1)}, … gives a basis of V, which we write in this order as v_{1}, …, v_{M} with M = ∑ m_{i}. Let G = GL_{m_{0}} × ⋯ × GL_{m_{e−1}}. Here GL_{m_{i}} acts on V_{i} in a natural way. Hence we have an action of G on V, and so on the n-fold tensor space V^{⊗n}. On the other hand, 𝔖_{n} acts on V^{⊗n} by permuting the factors of the tensor product. This action commutes with the action of G. We extend the action of 𝔖_{n} to that of W̃ as follows: For a basis element v = v_{i_{1}} ⊗ ⋯ ⊗ v_{i_{n}} in V, we put

t_{k}(v) = ζ^{j}v

if v_{i_{k}} ∈ V_{j}. Then this action of t_{1}, …, t_{n} on V gives rise to an action of W̃, commuting with the action of G. It is known that the following Schur-Weyl reciprocity holds (e.g., see [SS], where the Hecke algebra version is discussed).

(1.3.1) Let σ_{1} : ℂW̃ → End V^{⊗n}, ρ_{1} : ℂG → End V^{⊗n} be the representations of W̃ and G, respectively, (ℂW̃, etc. denote the group algebra of W̃, etc. over ℂ). Then σ_{1}(ℂW̃) and ρ_{1}(ℂG) are the centralizer algebras of each other in End V^{⊗n}. More precisely, the following holds. Put m = (m_{0}, …, m_{e−1}), and let Λ_{m} be the set of α ∈ P_{n,e} such that l(α^{(i)}) ≤ m_{i}. Then W̃ × G-module V^{⊗n} is decomposed as

V^{⊗n} = ⨁_{α∈Λ_{m}} Z_{α} ⊗ V_{α},

where Z_{α} is the irreducible W̃-module as before, and V_{α} is an irreducible G-module with highest weight α. In particular, if m_{i} ≥ n for any i, all the irreducible W̃-modules are realized in V^{⊗n}.

1.4. We shall extend the Schur-Weyl reciprocity to the case of W. Here we pose the following assumption on m.

(1.4.1) m_{k} = m_{k+d} for k = 0, 1, …, p − 1 in m = (m_{0}, …, m_{e−1}).

Let us define a linear automorphism τ = τ_{p} on V by τ(v_{j}^{(k)}) = v_{j}^{(k+d)} for 1 ≤ j ≤ m_{k} and for integers k (here we regard k ∈ ℤ/eℤ). Then τ is an element of GL_{M} of order p, and normalizes the subgroup G. We denote by G̃ the subgroup of GL_{M} generated by G and τ, which is isomorphic to the semidirect product G ⋊⟨τ⟩, where τ acts on G = GL_{m_{0}} × ⋯ × GL_{m_{e−1}} by τ(GL_{m_{k}}) = GL_{m_{k+d}}. Now we have an action of G̃ on V^{⊗n}, where τ acts on V^{⊗n} by

τ(v_{i_{1}}^{(k_{1})} ⊗ ⋯ ⊗ v_{i_{n}}^{(k_{n})}) = v_{i_{1}}^{(k_{1}+d)} ⊗ ⋯ ⊗ v_{i_{n}}^{(k_{n}+d)}.

We denote this action also by ρ_{1}. Here ρ_{1}(τ) is an automorphism of order p on V^{⊗n}, commuting with the action of 𝔖_{n}. Moreover, we have τστ^{−1} = ζ^{−d}σ on V^{⊗n}, and so ρ_{1}(τ) normalizes the subalgebra ρ_{1}(ℂW̃) of End V^{⊗n}. It is easy to check, for w ∈ W̃, that σ_{1}(w) commutes with ρ_{1}(τ) if and only if w ∈ W. Thus we have an action of W ×G̃ on V^{⊗n}. We consider the decomposition of V^{⊗n} as in (1.3.1). Since τ(V_{α}) \underrightarrow{∼} V_{θ(α)}, τ maps Z_{α} ⊗ V_{α} onto Z_{θ(α)} ⊗ V_{θ(α)}. Here we may choose, as a model of Z_{α} in V^{⊗n}, the space of highest weight vectors with highest weight α in V^{⊗n}. Then we have τ(Z_{α}) = Z_{θ(α)}. Put Γ =< τ >≃ ℤ/pℤ, and let Γ_{α} be the stabilizer of Z_{α} in Γ. Then Γ_{α} = ⟨τ^{c}⟩ with c = c_{α,p}, and Z_{α} turns out to be a Γ_{α}-module. For each ϕ ∈ Γ_{α}^{∧}, we denote by Z_{α,ϕ} the ϕ-isotropic subspace of Z_{α}, which is stable by W. Now σ permutes factors Z_{α,ϕ} transitively, and |Γ_{α}| = p/c_{α,p}. It follows that each Z_{α,ϕ} becomes an irreducible W -module and

Z_{α} = ⨁_{ϕ∈Γ_{α}^{∧}} Z_{α,ϕ}

gives the decomposition of Z_{α} into irreducible W -modules given in (1.2.1). On the other hand, let V′_{α,ϕ} be a G-submodule of V^{⊗n} generated by a highest weight vector in Z_{α,ϕ}. Then V′_{α,ϕ} is isomorphic to V_{α}, and is stable under the action of τ^{c}. We denote by V_{α,ϕ} the G̃-submodule of V^{⊗n} generated by V′_{α,ϕ}. Then V_{α,ϕ} is isomorphic to the G̃-module induced from Γ_{α}⋉G-module V′_{α,ϕ}, and turns out to be an irreducible G̃-module. Moreover, V_{α,ϕ} are mutually non-isomorphic for distinct pairs (α, ϕ). Put

Λ_{m}^{p} = {(α, ϕ) | α ∈ Λ_{m}/∼_{p}, ϕ ∈ Γ_{α}^{∧}},

where Λ_{m}/∼_{p} denotes the set of θ-orbits in Λ_{m}. It follows from the above discussion, we have the following Schur-Weyl reciprocity between W and G̃.

Proposition 1.5. σ_{1}(ℂW) and ρ_{1}(ℂG̃) are mutually the full centralizer algebras of each other in End V^{⊗n}. Moreover, W × G̃-module V^{⊗n} is decomposed as

V^{⊗n} = ⨁_{(α,ϕ)∈Λ_{m}^{p}} Z_{α,ϕ} ⊗ V_{α,ϕ}.

2. Frobenius formula for G(e, p, n)

2.1. In the remainder of this paper, we assume that m = (m_{0}, …, m_{e−1}) satisfies the condition (1.4.1) and the condition that m_{i} ≥ n for i = 0, …, e − 1. Then any irreducible W -module is realized as Z_{α,ϕ} by Proposition 1.5. In this case, Λ_{m}^{p} coincides with the set

P̃_{W} = {(α, ϕ) | α ∈ P_{n,e}/∼_{p}, ϕ ∈ Γ_{α}^{∧}}.

We denote by χ^{α,ϕ} the irreducible character of W corresponding to the W -module Z_{α,ϕ}. Under this notation, the parameterization of W^{∧} in (1.2.2) can be modified as

(2.1.1)  W^{∧} = {χ^{α,ϕ} | (α, ϕ) ∈ P̃_{W} }.

For later use, we consider a more general situation. Assume that q is a factor of e and that ⟨σ^{q}⟩×⟨σ^{p}⟩ is a subgroup of ⟨σ⟩ ≃ ℤ/eℤ, i.e., e/q and e/p are prime to each other. The typical case is that W = W_{0} and q is any factor of e. Now one can form a semidirect product ⟨σ^{q}⟩⋉W as a subgroup of W̃. We are now interested in the set of σ^{q}-stable characters in W^{∧}. It follows from the discussion in 1.4 that the irreducible W -module Z_{α,ϕ} is σ^{q}-stable if and only if q is divisible by |Γ_{α}| = p/c_{α,p}. So we put

P̃_{W}^{q} = {(α, ϕ) ∈ P̃_{W} | qc_{α,p} ≡ 0 (mod p)}.

We also write it as P̃_{n,e,p}^{q} to make the dependence on n, e, p more explicit. Let W_{ex,q}^{∧} be the set of σ^{q}-stable irreducible characters of W. Thus W_{ex,q}^{∧} is given as

(2.1.2)  W_{ex,q}^{∧} = {χ^{α,ϕ} ∈ W^{∧} | (α, ϕ) ∈ P̃_{W}^{q} }.

2.2. As in [S], we identify P_{n,e} with the set Z_{n}^{0,0} = Z_{n}^{0,0}(m). Here Z_{n}^{0,0} is the set of e-partitions α written as α = (α^{(0)}, …, α^{(e−1)}) with a partition α^{(k)} : α_{1}^{(k)} ≥ ⋯ ≥ α_{m_{k}}^{(k)} ≥ 0. We prepare the indeterminates x_{i}^{(k)} for 0 ≤ k < e, 1 ≤ i ≤ m_{k}. We write x = {x_{i}^{(k)}}, and also write as x^{(k)} = {x_{1}^{(k)}, …, x_{m_{k}}^{(k)}} for a fixed k. Recall that a power sum symmetric function p_{α}(x) is defined for α ∈ Z_{n}^{0,0} as

(2.2.1)  p_{α}(x) = ∏_{k=0}^{e−1} ∏_{j=1}^{m_{k}} p_{α_{j}^{(k)}}^{(k)}(x),

where we put, for each integer r ≥ 1,

(2.2.2)  p_{r}^{(i)}(x) = ∑_{j=0}^{e−1} ζ^{ij}p_{r}(x^{(j)}),

with usual r-th power sum symmetric functions p_{r}(x^{(j)}) with variables x^{(j)}, and put p_{0}^{(i)}(x) = 1. Also, Schur functions s_{α}(x) and monomial symmetric functions m_{α}(x) are defined as

s_{α}(x) = ∏_{k=0}^{e−1} s_{α^{(k)}}(x^{(k)}), m_{α}(x) = ∏_{k=0}^{e−1} m_{α^{(k)}}(x^{(k)}),

by using usual Schur functions s_{α^{(k)}} and monomial symmetric functions m_{α^{(k)}} associated to partitions α^{(k)}.

It is known that the set of conjugacy classes in W̃ is in bijection with P_{n,e}. The explicit correspondence will be given later in 2.3. We denote by w_{β} a representative of the conjugacy class in W̃ corresponding to β ∈ P_{n,e}. Then the Frobenius formula for the irreducible characters of W̃ is given as follows.

(2.2.3) (Frobenius formula for W̃). Let β ∈ P_{n,e}. Then we have

p_{β} = ∑_{α∈P_{n,e}} χ^{α}(w_{β})s_{α}.

2.3. We want to generalize (2.2.3) to the case of irreducible characters on σ^{q}W. First of all we describe the conjugacy classes in W̃ and W more precisely. The correspondence between the conjugacy classes in W̃ and P_{n,e} is given explicitly as follows. Assume that w maps e_{i_{1}}, e_{i_{2}}, …, e_{i_{m}} in a cyclic way, up to scalar, and leaves other e_{j} unchanged. If w^{m}(e_{i_{1}}) = ζ^{k}e_{i_{1}}, we say that w is a k-cycle (i_{1}, …, i_{m}) of length m. Now w ∈ W̃ can be written as a product of various disjoint cycles. If we group up, for a fixed k, the k-cycles among them, it produces a partition α^{(k)}, and α = (α^{(0)}, …, α^{(e−1)}) turns out to be an e-partition of n. This gives the required bijection. We denote by C_{α} the conjugacy class of W̃ corresponding to α ∈ P_{n,e}.

For α = (α^{(0)}, …, α^{(e−1)}) ∈ P_{n,e}, we define an integer Δ(α) by

(2.3.1)  Δ(α) = ∑_{k=0}^{e−1} l(α^{(k)})k.

It is easily checked that the conjugacy class C_{α} belongs to W if and only if Δ(α) ≡ 0 (mod p). The class C_{α} is decomposed into several conjugacy classes in W.

More generally, we consider a coset σ^{q}W in W̃. Then σ^{q}W is invariant under the adjoint action of W̃, and the class C_{α} lies in σ^{q}W if and only if Δ(α) ≡ q (mod p). The conjugacy class C_{α} ⊂ σ^{q}W is also decomposed into several W -orbits. The complete description of such W -orbits will be given in Proposition 2.13.

2.4. Let D be an operator on V^{⊗n} defined by

D(v_{i_{1}} ⊗ v_{i_{2}} ⊗ ⋯ ⊗ v_{i_{n}}) = x_{i_{1}}x_{i_{2}} ⋯ x_{i_{n}}(v_{i_{1}} ⊗ v_{i_{2}} ⊗ ⋯ ⊗ v_{i_{n}}).

(Although we must replace V by the scalar extension K ⊗_{ℂ} V with K = ℂ(x_{j}^{(k)}), we use the same notation by abbreviation). Then D commutes with the action of W̃, and the Frobenius formula (2.2.3) is obtained by computing the trace Tr (Dw, V^{⊗n}) for w ∈ W̃ in two different ways. We follow this strategy to establish the Frobenius formula for the characters of W, in our case by computing the traces Tr (Dτ^{j}w, V^{⊗n}) for j = 0, …, p − 1.

For an integer j (0 ≤ j < p), let h_{j} be order of the image j̄ of j in ℤ/pℤ. Then p is written as p = h_{j}j_{1}, where j_{1} is the greatest common divisor of j and p. We introduce new indeterminates X_{j} = {X_{i}^{(k)}} by

(2.4.1) X_{i}^{(k)} = x_{i}^{(k)}x_{i}^{(k+jd)}x_{i}^{(k+2jd)} ⋯ x_{i}^{(k+(h−1)jd)} (0 ≤ k < j_{1}d, 1 ≤ i ≤ m_{k}),

with h = h_{j}. We write also as X_{j}^{(k)} = {X_{1}^{(k)}, …, X_{m_{k}}^{(k)}} for a fixed k. One can define a function p_{r}^{(i)}(X_{j}) with respect to the variables X_{j} = {X_{j}^{(0)}, …, X_{j}^{(j_{1}d−1)}} by modifying (2.2.2) as follows.

(2.4.2)  p_{r}^{(i)}(X_{j}) = ∑_{k=0}^{j_{1}d−1} ζ^{ik}p_{r}(X_{j}^{(k)}).

Then we have the following lemma.

Lemma 2.5. Let w ∈ W̃ be an f-cycle of length n. Assume that w is of the form w = t_{1}^{a}t_{n}^{b}z, where z is a cyclic permutation (1, 2, …, n) ∈ 𝔖_{n} and a + b ≡ f (mod p). Then, for each j such that 0 ≤ j < p,

Tr (Dτ^{j}w, V^{⊗n}) = \begin{cases}hζ^{−(f+b)jd}p_{n/h}^{(f)}(X_{j}) & if h | n and h | f, \\ 0 & otherwise, \end{cases}

where h = h_{j} as before.

Proof. We have

τ^{j}w(v_{i_{1}}^{(k_{1})} ⊗ v_{i_{2}}^{(k_{2})} ⊗ ⋯ ⊗ v_{i_{n}}^{(k_{n})}) = ζ^{ak_{n}+bk_{n−1}}v_{i_{n}}^{(k_{n}+jd)} ⊗ v_{i_{1}}^{(k_{1}+jd)} ⊗ ⋯ ⊗ v_{i_{n−1}}^{(k_{n−1}+jd)}.

Now assume that Tr (Dτ^{j}w, V^{⊗n}) ≠ 0. Then there exists v_{i_{1}}^{(k_{1})}, …, v_{i_{n}}^{(k_{n})} ∈ E satisfying the relation

v_{i_{s}}^{(k_{s})} = v_{i_{s−1}}^{(k_{s−1}+jd)} for s = 1, …, n.

(Here we regard v_{i_{0}} = v_{i_{n}}, and k_{0} = k_{n}). This implies that

(2.5.1)  v_{i_{1}}^{(k_{1})} = v_{i_{1}}^{(k_{1}+njd)},
v_{i_{s}}^{(k_{s})} = v_{i_{1}}^{(k_{1}+(s−1)jd)} for s = 2, …, n.

In particular, we have njd ≡ 0 (mod e), or equivalently nj ≡ 0 (mod p). Under the notation in (2.4.1), we have

(2.5.2)  Tr (Dτ^{j}w, V^{⊗n}) = ζ^{−(a+2b)jd} ∑_{k=0}^{j_{1}d−1} ∑_{s=0}^{h−1} ζ^{f(k+sjd)} ∑_{i=1}^{m_{k}}(X_{i}^{(k)})^{n/h}.

This implies that Tr (Dτ^{j}w, V^{⊗n}) = 0 unless fj ≡ 0 (mod p). Here note that the condition nj ≡ 0 (mod p) is equivalent to h | n, and similarly for fj ≡ 0 (mod p). The second assertion follows from this.

We now assume that nj ≡ 0 and fj ≡ 0 (mod p). Then by (2.4.2) and (2.5.2), we have

Tr (Dτ^{j}w, V^{⊗n}) = hζ^{−(a+2b)jd}p_{n/h}^{(f)}(X_{j}).

This shows the first assertion, and the lemma is proved. □

2.6. We consider the general case where w is in the class C_{α} in W̃. We fix j and let h = h_{j} and j_{1} be as before. For α = (α_{i}^{(k)}), we consider the following condition (for a fixed j).

(2.6.1) h | α_{i}^{(k)} for any α_{i}^{(k)}, and

h | k if |α^{(k)}| ≠ 0.

Assume that α ∈ P_{n,e} satisfies (2.6.1). We define a j_{1}d-partition β = (β^{(0)}, …, β^{(j_{1}d−1)}) of n/h by β_{i}^{(k/h)} = α_{i}^{(k)}/h for any α_{i}^{(k)} such that h | k. We write β as β = α[j]. Then we have hΔ(α[j]) = Δ(α). Note that in this situation (2.4.2) can be regarded as a formula analogous to (2.2.2) for variables X_{j}, by replacing ζ by ζ^{h}. Thus one can define p_{α[j]}(X_{j}) just as in (2.2.1) as a product of various p_{β_{i}^{(k)}}(X_{j}).

As in 2.3, w can be written as a product of various cycles associated to α. Under the conjugation in W, w is changed to an element w′ of the following type; the cycle corresponding to α_{i}^{(k)} is of the form t_{i_{1}}^{k}z, where z = (i_{1}, i_{2}, …, i_{m}) ∈ 𝔖_{m} with m = α_{i}^{(k)}, except one cycle. The excepted one is of the form t_{i_{1}}^{a}t_{i_{m}}^{b}z, where z = (i_{1}, i_{2}, …, i_{m}) with a + b ≡ k (mod e). We may further assume that each cyclic permutation z occurring in the decomposition of w′ is of the form z = (i_{1}, i_{1} + 1, …, i_{1} + (m − 1)). We write this element as w′ = w_{α}(b).

Note that Tr (Dτ^{j}w, V^{⊗n}) remain unchanged if w is replaced by its conjugate w′, since W commutes with D and τ. Then Tr (Dτ^{j}w′, V^{⊗n}) can be computed by making use of Lemma 2.5. So, as a corollary to Lemma 2.5, we have

Proposition 2.7. Assume that w ∈ C_{α} in W̃, and that w is conjugate under W to w_{α}(b). Then Tr (Dτ^{j}w, V^{⊗n}) = 0 unless α satisfies the condition (2.6.1). If α satisfies the condition (2.6.1), then

Tr (Dτ^{j}w, V^{⊗n}) = h^{l(α[j])}ζ^{−(Δ(α)+b)jd}p_{α[j]}(X_{j}).

2.8. Next, we consider the decomposition V^{⊗n} = ⨁ V_{α,ϕ} ⊗ Z_{α,ϕ} as given in Proposition 1.5. Assume that w ∈ σ^{q}W. If χ^{α,ϕ} ∈ W^{∧} is not σ^{q}-stable, then Dτ^{j}w maps V_{α,ϕ} ⊗ Z_{α,ϕ} onto a different factor. It follows that

(2.8.1)  Tr (Dτ^{j}w, V^{⊗n}) = ∑_{(α,ϕ)∈P̃_{W}^{q}} Tr (Dτ^{j}w, V_{α,ϕ} ⊗ Z_{α,ϕ}).

Let χ^{α,ϕ} be a σ^{q}-stable irreducible character of W. Then χ^{α,ϕ} can be extended to an irreducible character of ⟨σ^{q}⟩⋉W ≃ W_{e,p′,n}, where p′ = pq/e. Let Γ′ ≃ ℤ/p′ℤ be the subgroup of Γ ≃ ℤ/pℤ generated by τ^{e/q}, and Γ′_{α} the stabilizer of Z_{α} in Γ′. Since q is divisible by |Γ_{α}| by (2.1.2) and since (e/p, e/q) = 1, we see that Γ′_{α} coincides with Γ_{α}. Hence irreducible characters of W occurring in the decomposition of χ^{α} are also regarded as irreducible characters of W_{e,p′,n}. In this way, we can fix an extension χ̃^{α,ϕ} of χ^{α,ϕ} to W_{e,p′,n}. Accordingly, the irreducible W -module Z_{α,ϕ} is extended to the irreducible W_{e,p′,n}-module corresponding to χ̃^{α,ϕ}, which we denote by Z̃_{α,ϕ}.

Remember that V_{α,ϕ} ⊗ Z_{α,ϕ} ≃ ⨁_{β∈O(α)} V′_{β,ϕ} ⊗ Z_{β,ϕ} as G × W -module, and τ permutes the summands of the right hand side. In the case where c_{α,p} | j, τ^{j} leaves each summand V′_{β,ϕ} ⊗ Z_{β,ϕ} stable. The last space is extended to a W_{e,p′,n}-module V′_{β,ϕ} ⊗ Z̃_{β,ϕ}, and we have an action of w on V_{α,ϕ} ⊗ Z̃_{α,ϕ} ≃ ⨁_{β∈O(α)} V′_{β,ϕ} ⊗ Z̃_{β,ϕ}.

Assume that α ∈ P_{n,e} satisfies the condition that c_{α,p} | j. We put α{j} = (α^{(0)}, …, α^{(j_{1}d−1)}). Since j_{1}d is divisible by c_{α,p}, α{j} turns out to be a j_{1}d-partition of n/h. Also note that, τ^{j} ∈ Γ_{α}. Under this setting, we have the following lemma.

Lemma 2.9. Let α ∈ P_{n,e}, and q be as in 2.1. Assume that χ^{α,ϕ} is σ^{q}-stable. Then for w ∈ σ^{q}W, we have

Tr (Dτ^{j}w,V_{α,ϕ} ⊗ Z̃_{α,ϕ}) =
ϕ(τ^{j}) ∑_{0≤i<c} ζ^{qid}s_{θ^{i}(α){j}}(X_{j})χ̃^{α,ϕ}(w)  if c | j,
0  otherwise,

where c = c_{α,p}, and s_{α{j}}(X_{j}) etc. are Schur functions given in 2.2 with respect to the variables X_{j}.

Proof. By the previous remark, Dτ^{j}w permutes the factors V′_{β,ϕ} ⊗ Z_{β,ϕ} if j is not a multiple of c, and so Tr (Dτ^{j}w, V_{α,ϕ} ⊗ Z̃_{α,ϕ}) = 0. Now assume that c | j. Then we can write

Tr (Dτ^{j}w, V_{α,ϕ} ⊗ Z̃_{α,ϕ}) = ∑_{β∈O(α)} Tr (Dτ^{j}, V′_{β,ϕ})χ̃^{β,ϕ}(w).

Since V′_{β,ϕ} = ϕ⊗V′_{β,1} as⟨τ^{c}⟩⋉G-modules, we have Tr (Dτ^{j}, V′_{β,ϕ}) = ϕ(τ^{j}) Tr (Dτ^{j}, V′_{β,1}). We note that

(2.9.1)  Tr (Dτ^{j}, V′_{β,1}) = s_{β{j}}(X_{j}).

In fact, V′_{β,1} is an irreducible G-module isomorphic to V_{β}, and it is known from (2.2.3) that Tr (D, V_{β}) = s_{β}(x). V_{β} has a basis consisting of weight vectors, on which D acts diagonally. Here for a vector v = v_{i_{1}}^{(k_{1})} ⊗ ⋯ ⊗ v_{i_{n}}^{(k_{n})} ∈ V^{⊗n}, the weight of v is given by an element γ(v) = (γ^{(0)}, …, γ^{(e−1)}) ∈ Z_{n}^{0,0}, where γ_{i}^{(k)} is the number of v_{i}^{(k)} occurring in the expression of v. On the other hand, τ^{j} acts on V_{β}, by permuting the weight vectors; if v is a weight vector with weight γ, then τ^{j}(v) is also a weight vector with weight γ′ = (γ^{(−jd)}, γ^{(1−jd)}, …, γ^{(e−1−jd)}). Hence, in order to compute Tr (Dτ^{j}, V_{β}), we have only to consider the weight vectors in V_{β} whose weights are of the type γ = (γ_{i}^{(k)}) such that γ^{(k)} = γ^{(k+jd)} for each k. Note that monomials ∏(x_{i}^{(k)})^{γ_{i}^{(k)}} obtained as weights of D produce the Schur function s_{β}(x). Then the corresponding weight for Dτ^{j} is given by ∏(X_{i}^{(k)})^{γ_{i}^{(k)}} for γ as above (the product is taken for k such that 0 ≤ k < j_{1}d). Hence Tr (Dτ^{j}, V′_{β,1}) is obtained by picking up the monomials of this type from s_{β}(x) = ∏_{k=0}^{e−1} s_{β}_{(k)}(x^{(k)}), which coincides with s_{β{j}}(X_{j}). This proves (2.9.1).

To prove the lemma, it is enough to show that

(2.9.2)  χ̃^{θ^{i}(α),ϕ}(w) = ζ^{qid}χ̃^{α,ϕ}(w)

for i = 0, …, c−1. We show (2.9.2). Since χ̃^{α,ϕ} and χ̃^{θ^{i}(α),ϕ} are both extensions of χ^{α,ϕ}, there exists a linear character φ of the cyclic group ⟨σ^{q}⟩ such that χ̃^{θ^{i}(α),ϕ} = φ ⊗ χ̃^{α,ϕ}. On the other hand, since τστ^{−1} = ζ^{−d}σ on V^{⊗n}, we see that the action of σ^{q} on Z̃_{θ^{i}(α),ϕ} = τ^{i}(Z̃_{α,ϕ}) corresponds, under the isomorphism τ^{i} of W -modules, to the action of σ^{q} on Z̃_{α,ϕ} multiplied by ζ^{qid}. This implies (2.9.2), and the lemma follows. □

2.10. In order to formulate the Frobenius formula for the σ^{q}-stable characters of W, we shall define Schur functions and power sum symmetric functions associated to σ^{q}W. Let (α, ϕ) ∈ P̃_{W}^{q}, and put, for each 0 ≤ j < p,

(2.10.1)  s_{α,ϕ}^{j}(x) = \begin{cases}ϕ(τ^{j}) ∑_{0≤i<c} ζ^{qid}s_{θ^{i}(α)\{j\}}(X_{j}) & if c | j, \\ 0 & otherwise, \end{cases}

where c = c_{α,p}. We put s_{α,ϕ}(x) = (s_{α,ϕ}^{j}(x))_{0≤j<p}, and call it Schur function for σ^{q}W associated to (α, ϕ).

Next, we consider power sum symmetric functions. For a pair (α, b), where α ∈ P_{n,e} and 0 ≤ b < e, and for 0 ≤ j < p, we put

(2.10.2)  p_{α,b}^{j}(x) = \begin{cases}h^{l(α[j])}ζ^{−(Δ(α)+b)jd}p_{α[j]}(X_{j}) & if α satisfies (2.6.1), \\ 0 & otherwise. \end{cases}

(For the notation, see 2.6.) We put p_{α,b}(x) = (p_{α,b}^{j}(x))_{0≤j<p}, and call it a power sum symmetric function for σ^{q}W associated to the pair (α, b).

We are now ready to formulate a Frobenius formula for the characters on σ^{q}W as follows. The proof is immediate from Proposition 2.7, (2.8.1) and Lemma 2.9.

Proposition 2.11 (Frobenius formula for σ^{q}W). Let (β, b) be such that β ∈ P_{n,e} and 0 ≤ b < e. Put w = w_{β}(b), and assume that w ∈ σ^{q}W, (cf. 2.6). Then we have

p_{β,b}(x) = ∑_{(α,ϕ)∈P̃_{W}^{q}} χ̃^{α,ϕ}(w)s_{α,ϕ}(x).

2.12. As in [S, 3.5], we consider the ring of symmetric polynomials Ξ_{m} = ⨂_{k=0}^{e−1} ℤ[x_{1}^{(k)}, …, x_{m_{k}}^{(k)}]^{𝔖_{m_{k}}} with respect to 𝔖_{m} = 𝔖_{m_{0}} × ⋯ × 𝔖_{m_{e−1}}. Ξ_{m} has a structure of graded ring Ξ_{m} = ⨁_{i≥0} Ξ_{m}^{i}, where Ξ_{m}^{i} consists of homogeneous symmetric polynomials of degree i. We can define a space of symmetric functions Ξ = ⨁_{i≥0} Ξ^{i}, where Ξ^{i} is the inverse limit of Ξ_{m}^{i} (cf. [loc. cit.]). For α ∈ P_{n,e}, the functions s_{α}(x), p_{α}(x) given in 2.2 make sense for infinitely many variables x_{1}^{(k)}, x_{2}^{(k)}, …, and give rise to elements in Ξ^{n}.

Schur functions s_{α,ϕ}(x) and power sum symmetric functions p_{β,b}(x) defined in 2.10 are also extended to the functions with infinitely many variables, and they can be regarded as elements in the space ⨁_{p} Ξ_{ℂ}^{n}, the direct sum of p copies of Ξ_{ℂ}^{n} = ℂ ⊗ Ξ^{n}. It is easy to see that Schur functions {s_{α,ϕ} | (α, ϕ) ∈ P̃_{W}^{q} } are linearly independent in ⨁_{p} Ξ_{ℂ}^{n}. We denote by Ξ_{ℂ}^{n}(p, q) the subspace of ⨁_{p} Ξ_{ℂ}^{n} generated by those s_{α,ϕ}. Thus Schur functions {s_{α,ϕ} | (α, ϕ) ∈ P̃_{W}^{q} } form a basis of Ξ_{ℂ}^{n}(p, q).

The space Ξ_{ℂ}^{n}(p, q) is also described as follows. Let Ξ_{p,q}^{n} be the subspace of Ξ_{ℂ}^{n} generated by ∑_{i=0}^{c−1} ζ^{qid}s_{θ^{i}(α)} for various α ∈ P_{n,e} (here c = c_{α,p}). Then

(2.12.1)  Ξ_{ℂ}^{n}(p, q) = ⨁_{j=0}^{p−1} Ξ_{p,q}^{n/h_{j}}(X_{j}),

where Ξ_{p,q}^{n/h_{j}}(X_{j}) denotes the space Ξ_{p,q}^{n/h_{j}} with respect to the variables X_{j}. (2.12.1) is immediate from (2.10.1).

It follows from Proposition 2.11 that p_{β,b} ∈ Ξ_{ℂ}^{n}(p; q) if w_{β}(b) ∈ σ^{q}W. By making use of the Frobenius formula, we can give a complete description of W -orbits of σ^{q}W. First we prepare some notations. Take w = w_{β}(b) ∈ σ^{q}W. For 0 ≤ j < p, we define f_{j}(w) ∈ ℤ/pℤ by

(2.12.2)  f_{j}(w) = \begin{cases}bj (mod p) & if β satisfies (2.6.1), \\ 0 & otherwise. \end{cases}

By Proposition 2.7 and (2.10.2), we can associate to each W -orbit in σ^{q}W a welldefined function p_{β,b}(x), where w = w_{β}(b) is a representative of this orbit. We denote p_{β,b}(x) also by p_{w}(x). As a corollary to Proposition 2.11, we have the following characterization of W -orbits in σ^{q}W.

Proposition 2.13. (i) Assume that w, w′ ∈ σ^{q}W. Then w and w′ are conjugate

under W if and only if p_{w}(x) = p_{w′}(x). (ii) Let w = w_{β}(b) and w′ = w_{β}(b′) are elements in σ^{q}W, (i.e., Δ(β) ≡ q (mod p),

cf. 2.3). Then w and w′ are conjugate under W if and only if f_{j}(w) = f_{j}(w′) for j = 0, …, p − 1.

Proof. It is known that the “character table” (χ̃^{α,ϕ}(w)) of σ^{q}W is a non-singular matrix. (Here (α, ϕ) runs over all the elements in P̃_{W}^{q}, and w runs over all the representatives of W -orbits in σ^{q}W). Since Schur functions form a basis of Ξ_{ℂ}^{n}(p, q), we see that {p_{w} | w ∈ σ^{q}W/∼} is also a basis by Proposition 2.11. In particular, they are all distinct. This proves (i). The second statement is then immediate from (2.10.2). □

2.14. In view of Proposition 2.13, we can determine the parameter set for the set of W -orbits in σ^{q}W. Put

P_{q}^{W} = {(β, b) | β ∈ P_{n,e}, 0 ≤ b < e, Δ(β) ≡ q (mod p)}.

We define an equivalence relation on P_{q}^{W} by (β, b) ∼ (β′, b′) if and only if β = β′ and p_{β,b}(x) = p_{β′,b′}(x). We denote by P̃_{q}^{W} = P̃_{q}^{n,e,p} the set of equivalence classes in P_{q}^{W}. Then P̃_{q}^{W} parameterizes the set of W -orbits in σ^{q}W, and so the set {p_{β,b} | (β, b) ∈ P̃_{q}^{W} } gives rise to a basis of Ξ_{ℂ}^{n}(p, q).

2.15. In this and next subsection, we give some examples of Schur functions and power sum symmetric functions in typical cases, i.e., the case where W = G(2, 2, n) and the case where W = G(e, e, 2).

First we assume that W = G(2, 2, n). So W (resp. W̃) is the Weyl group of type D_{n} (resp. type B_{n} ). In this case, θ^{2} = 1, and we have Γ_{α} ≃ ℤ/2ℤ if θ(α) = α, and Γ_{α} = 1 otherwise. In later discussion, the case P̃_{W}^{0} corresponds to the split D_{n} case, and the case P̃_{W}^{1} corresponds to the non-split D_{n} case. For each (α, ϕ) ∈ P̃_{W}^{0} , Schur function s_{α,ϕ} = (s_{α,ϕ}^{0}, s_{α,ϕ}^{1}) is defined as follows.

(2.15.1)  s_{α,ϕ}^{0}(x) = ∑_{β∈O(α)} s_{β}(x),

s_{α,ϕ}^{1}(x) = \begin{cases}±s_{β}(X) & if θ(α) = α, \\ 0 & otherwise. \end{cases}

Here if θ(α) = α, then α is written as α = (β; β) with a partition β of n/2, and s_{β}(X) is the Schur function with respect to the variables X_{j} = x_{j}^{(0)}x_{j}^{(1)}. (Note that β = α{1} in the previous notation). The sign takes + (resp. −) if ϕ = 1 (resp. ϕ ≠ 1).

On the other hand, (α, ϕ) ∈ P̃_{W}^{1} if and only if θ(α) ≠ α, and so Γ_{α} = {1}. In this case, Schur function s_{α,ϕ}(x) is given as follows.

(2.15.2)  s_{α,ϕ}^{0}(x) = s_{α}(x) − s_{θ(α)}(x),  s_{α,ϕ}^{1}(x) = 0.

Next we consider the power sum symmetric function p_{β,b} = (p_{β,b}^{0}, p_{β,b}^{1}). A conjugacy class C_{β} of W̃ lying in W decomposes into two W -orbits if and only if β = (β; −) with a partition β = (β_{1}, β_{2}, ⋯) such that β_{i} is even for each i. Let β′ = (β_{1}/2, β_{2}/2, …) be a partition of n/2. Then we have β′ = β[1] in the previous notation. Such a class C_{β} is called a degenerate class, and other classes are said to be non-degenerate. The function p_{β,b}(x) (b = 0, 1) associated to (β, b) ∈ P̃_{0}^{W} is given as follows.

(2.15.3)  p_{β,b}^{0}(x) = p_{β}(x),
p_{β,b}^{1}(x) = \begin{cases}0 & if C_{β} is non-degenerate, \\ ±2^{l(β)}p_{β′}(X) & if C_{β} is degenerate. \end{cases}

Here p_{β}(x) is the function for W̃, and p_{β′}(X) is the power sum symmetric function for 𝔖_{n/2} with variables X_{j} = x_{j}^{(0)}x_{j}^{(1)} associated to the partition β′. The sign takes + (resp. −) if b = 0 (resp. b = 1).

On the other hand, (β, b) ∈ P̃_{1}^{W} if and only if Δ(β) ≡ 1 (mod 2) and b = 0. In this case, p_{β,b}(x) is given as

p_{β,b}^{0}(x) = p_{β}(x), p_{β,b}^{1}(x) = 0.

2.16. We now consider the case where W = G(e, e, 2). So W is the dihedral group of order 2e. Put

α_{0} = (2; −; ⋯; −),
α_{ij} = (−; ⋯; 1; ⋯; 1; ⋯; −) (0 ≤ i ≤ j < e),

where in the second case, 1 appears only in the i-th and j-th entries. If i = j, we understand that α_{ii} = (−; ⋯; 11; ⋯; −). Then the set of representatives of θ-orbits of P_{2,e} is given as {α_{0}, α_{0j} | 0 ≤ j ≤ e/2}. Here Γ_{α} = {1} unless e is even and j = e/2, in which case Γ_{α_{0j}} = ℤ/2ℤ. Consequently, Schur functions s_{α,ϕ} = (s_{α,ϕ}^{k})_{0≤k<e} associated to (α, ϕ) ∈ P̃_{W}^{0} are given as follows.

(2.16.1)  s_{α,ϕ}^{k}(x) = \begin{cases}∑_{β∈O(α)} s_{β}(x) & if k = 0, \\ ± ∑_{β′∈O(α′)} s_{β}′(X) & if α = α_{0,e/2} and k = e/2, \\ 0 & otherwise. \end{cases}

In the second case, α′ = (1; −; ⋯; −) is an e/2-partition of 1 and coincides with α{e/2}. s_{α}′(X) is the Schur function associated to α′ with respect to the variables X^{(0)}, …, X^{(e/2−1)} for X_{j}^{(k)} = x_{j}^{(k)}x_{j}^{(e/2+k)}. Hence s_{α}′(X) = ∑_{j} x_{j}^{(0)}x_{j}^{(e/2)} and we have

s_{α,ϕ}^{e/2}(x) = ± ∑_{i=0}^{e/2−1} ∑_{j} x_{j}^{(i)}x_{j}^{(i+e/2)}.

The sign takes + (resp. −) if ϕ = 1 (resp. ϕ ≠ 1).

Next we consider power sum symmetric functions. A conjugacy class C_{β} ⊂ W̃ belongs to W in the following case;

{α_{0}, α_{i,e−i} (0 ≤ i ≤ e/2)}.
Then p_{β,b} = (p_{β,b}^{k}(x))_{0≤k<e} associated to (β, b) ∈ P̃_{0}^{W} is given as

(2.16.2)  p_{β,b}^{k}(x) = \begin{cases}p_{β}(x) & if k = 0, \\ (−1)^{b}2p_{α}′(X) & if k = e/2 and β = α_{0}, \\ 0 & otherwise, \end{cases}

14 T. SHOJI

where α′ = (1; −; ⋯; −) is the e/2-partition as given before and also coincides with α_{0}[e/2], and p_{α}′(X) is the power sum symmetric function associated to α′ with respect to variables {X^{(0)}, …, X^{(e/2−1)}}. Hence we have

p_{α}′(X) = ∑_{i=0}^{e/2−1} ∑_{j} x_{j}^{(i)}x_{j}^{(i+e/2)}.

3. Hall-Littlewood functions associated to G(e, p, n)

3.1. Here we review some results from [S] concerning symmetric functions associated to W̃. In what follows we regard the variables x_{i}^{(k)} defined for k ∈ ℤ/eℤ ≃ {0, 1, …, e − 1}. For each 0 ≤ k < e and an integer r ≥ 0, we define a function q̃_{r,±}^{(k)}(x; t) (according to the sign + or −) by

(3.1.1)  q̃_{r,±}^{(k)}(x; t) = ∑_{i≥1}(x_{i}^{(k)})^{r+δ} \frac{∏_{j} x_{i}^{(k)} − tx_{j}^{(k±1)}}{∏_{j≠i} x_{i}^{(k)} − x_{j}^{(k)}}  (r ≥ 1),

where δ = m_{k} − 1 − m_{k±1}. In the product of the denominator, x_{j}^{(k)} runs over all the variables in x^{(k)} except x_{i}^{(k)}, while in the numerator, x_{j}^{(k±1)} runs over all the variables in x^{(k±1)}. Then q̃_{r,±}^{(k)}(x; t) is a polynomial in ℤ[x; t] if δ ≥ 0, and lies in ℤ[x, x^{−1}; t] in general. We define q_{r,±}^{(k)} as follows. If δ ≥ 0 i.e., m_{k} ≥ m_{k±1} + 1, put q_{r,±}^{(k)} = q̃_{r,±}^{(k)}. If δ < 0, we add m_{k±1} + 1 − m_{k} variables x′ = x_{m_{k}+1}^{(k)}, …, x_{m_{k±1}+1}^{(k)} to x^{(k)}, and consider the polynomial q̃_{r,±}^{(k)} for such variables with δ = 0, and put q_{r,±}^{(k)} = q̃_{r,±}^{(k)}|_{x′=0}. Hence q_{r,±}^{(k)} ∈ ℤ[x; t] in all cases, and we have q_{0,±}^{(k)} = 1. (Note that in [S, 2.2], q_{r,±}^{(k)} was defined by using the formula (3.1.1). However, this definition is not appropriate for the case where δ < 0. The property stated in [S, Lemma 2.3] then holds for q_{r,±}^{(k)} defined above).

For an e-partition α = (α^{(0)}, …, α^{(e−1)}) ∈ P_{n,e}, we define a function q_{α,±}(x) by

(3.1.2)  q_{α,±}(x; t) = ∏_{k=0}^{e−1} ∏_{j=1}^{m_{k}} q_{α_{j}^{(k)},±}^{(k)}(x; t).

For α = (α_{j}^{(k)}) ∈ P_{n.e}, we define a function z_{α}(t) by
(3.1.3)  z_{α}(t) = z_{α} ∏_{k=0}^{e−1} ∏_{j=1}^{m_{k}}(1 − ζ^{k}t^{α_{j}^{(k)}})^{−1},

where in the product, we neglect the factors such that α_{j}^{(k)} = 0. z_{α} is the order of the centralizer of w_{α} in W̃. Explicitly, z_{α} is given as follows. For a partition β = (1^{n_{1}}, 2^{n_{2}}, …), put z_{β} = ∏_{i≥1} i^{n_{i}}n_{i}!. Then z_{α} = e^{l(α)} ∏_{k=0}^{e−1} z_{α^{(k)}}.

For later use, we also give the order of the stabilizer Z_{W} (w) of w ∈ σ^{q}W in W. Assume that C_{α} ⊂ σ^{q}W and that C_{α} decomposes into r distinct W -orbits. Since these W -orbits have the same cardinality, we have for w ∈ C_{α},

(3.1.4)  |Z_{W} (w)| = \frac{r}{p}|ZW̃(w)|.

We now introduce infinitely many variables x_{i}^{(k)}, y_{i}^{(k)} for i = 1, 2, … and for 0 ≤ k ≤ e − 1. As discussed in [S], p_{α}(x), q_{α,±}(x; t), m_{α}(x) can be regarded as functions with infinitely many variables x_{1}^{(k)}, x_{2}^{(k)}, . . .. Following [S], we introduce Cauchy’s reproducing kernel associated to W̃ by

(3.1.5)  Ω(x, y; t) = ∏_{k=0}^{e−1} ∏_{i,j} \frac{1 − tx_{i}^{(k+1)}y_{j}^{(k)}}{1 − x_{i}^{(k)}y_{j}^{(k)}} .

The following formula was proved in [S, Proposition 2.5].

Proposition 3.2. We have

(3.2.1)  Ω(x, y; t) = ∑_{α}q_{α,+}(x; t)m_{α}(y) = ∑_{α}m_{α}(x)q_{α,−}(y; t),
(3.2.2)  Ω(x, y; t) = ∑_{α}z_{α}(t)^{−1}p_{α}(x)p̄_{α}(y),

where α runs over all the e-partitions of any size. In (3.2.2), p̄_{α}(y) denotes the complex conjugate of p_{α}(y).

3.3. We define an isomorphism θ of ℂ[x] (the polynomial ring with infinitely many variables) by θ(x_{j}^{(k)}) = x_{j}^{(k+d)}. Then it follows from (2.2.2) that θ(p_{r}^{(k)}) = ζ^{−kd}p_{r}^{(k)}, and so we have

(3.3.1)  θ(p_{α}) = ζ^{−Δ(α)d}p_{α}.

On the other hand, it can be checked easily from the definition that

(3.3.2)  θ(s_{α}) = s_{θ(α)},  θ(m_{α}) = m_{θ(α)},  θ(q_{α,±}) = q_{θ(α),±}.

We now introduce an operator Θ_{q} : ℂ[x] → ℂ[x] for a factor q of e by

(3.3.3)  Θ_{q} = ∑_{i=0}^{p−1} ζ^{qid}θ^{i}.

Then it is easy to see that

(3.3.4)  Θ_{q}(p_{β}) = \begin{cases}p · p_{β} & if Δ(β) ≡ q (mod p), \\ 0 & otherwise. \end{cases}

Take α ∈ P_{n,e} and put c = c_{α,p} as before. Then it follows from (3.3.2) that we have

(3.3.5)  Θ_{q}(s_{α}) = \begin{cases}\frac{p}{c} ∑_{i=0}^{c−1} ζ^{qid}s_{θ^{i}(α)} & if qc ≡ 0 (mod p), \\ 0 & otherwise. \end{cases}

Similar formulas as (3.3.5) hold also for Θ_{q}(m_{α}) and Θ_{q}(q_{α,±}) by replacing s_{θ^{i}(α)} in (3.3.5) by m_{θ^{i}(α)} and q_{θ^{i}(α),±}.

Let us define m_{α,ϕ}^{j}(x) and q_{α,ϕ,±}^{j}(x; t), for each 0 ≤ j < p, as in the case of s_{α,ϕ}(x) by replacing s_{θ^{i}(α){j}} in (2.10.1) by m_{θ^{i}(α){j}} and q_{θ^{i}(α){j},±}. We put

(3.3.6)  q_{α,ϕ,±} = (q_{α,ϕ,±}^{j})_{0≤j<p},  m_{α,ϕ} = (m_{α,ϕ}^{j})_{0≤j<p}.

Note that θ induces an action θ(X_{i}^{(k)}) = X_{i}^{(k+d)} on the variables X_{j}. Thus similar formulas as (3.3.4) and (3.3.5) hold also for functions p_{α[j]}(X_{j}) and s_{α{j}}(X_{j}), etc. (More precisely, the formula for s_{α{j}} is completely similar to (3.3.5). But for p_{β}(X_{j}), ζ must be replaced by ζ^{h}. In particular, we have θ(p_{r}^{(k)}(X_{j})) = ζ^{−khd}p_{r}^{(k)}(X_{j}). Hence the condition in the first formula in (3.3.4) should be replaced by “if hΔ(β) ≡ q (mod p)” for a j_{1}d-partition β). Now q_{α,±} can be written as a linear combination of s_{β}. Then by applying Θ_{q}, ∑_{i} ζ^{qid}q_{θ^{i}(α),±} can be written as a linear combination of various ∑_{i} ζ^{qid}s_{θ^{i}(β)}. It follows that q_{α,ϕ,±}^{j} ∈ Ξ_{p,q}^{n/h_{j}}(X_{j}). Hence by (2.12.1), q_{α,ϕ,±} is contained in Ξ_{ℂ(t)}^{n}(p, q) = ℂ(t) ⊗ Ξ_{ℂ}^{n}(p, q). Similarly, m_{α,ϕ} is contained in Ξ_{ℂ(t)}^{n}(p, q). Now it is easy to see that the sets {q_{α,ϕ,±} | (α, ϕ) ∈ P̃_{W}^{q} } and {m_{α,ϕ} | (α, ϕ) ∈ P̃_{W}^{q} } form bases of Ξ_{ℂ(t)}^{n}(p, q).

For each j (0 ≤ j < p), we also consider the variables Y_{j} = (Y_{i}^{(k)}) defined in a similar way as X_{j} but obtained from the variables y = (y_{i}^{(k)}) instead of x = (x_{i}^{(k)}). We now define Cauchy’s reproducing kernel associated to σ^{q}W by Ω_{q}(x, y; t) = (Ω_{q}^{j}(x, y; t))_{0≤j<p}, with

(3.3.7)  Ω_{q}^{j}(x, y; t) = Θ_{q,X_{j}}(Ω(X_{j}, Y_{j}; t^{h_{j}})),

where Ω(X_{j}, Y_{j}; t^{h_{j}}) is the function defined in a similar way as (3.1.4), with respect to the variables X_{j}, Y_{j} and t^{h_{j}}. Θ_{q,X_{j}} stands for the action of Θ_{q} on the variables X_{j}. For w = w_{α}(b) ∈ σ^{q}W, we define z_{α,b}(t) by

(3.3.8)  z_{α,b}(t) = z_{α,b} ∏_{k=0}^{e−1} ∏_{j=1}^{m_{k}}(1 − ζ^{k}t^{α_{j}^{(k)}})^{−1}.

Hence, z_{α,b}(t) is the one obtained from z_{α}(t) by replacing z_{α} = |ZW̃(w)| by z_{α,b} = |Z_{W} (w)|, and so it coincides with rp^{−1}Z_{α}(t) by (3.1.4).

The following proposition gives a counter part of Proposition 3.2 to the case of σ^{q}W.

Proposition 3.4. Ω_{q}(x, y; t) has the following expansions.

(3.4.1)  Ω_{q}(x, y; t) = ∑_{(α,ϕ)} q_{α,ϕ,+}(x; t)\overline{m}_{α,ϕ}(y) = ∑_{(α,ϕ)} m_{α,ϕ}(x)\overline{q}_{α,ϕ,−}(y; t),
(3.4.2)  Ω_{q}(x, y; t) = ∑_{(α,b)} z_{α,b}(t)^{−1}p_{α,b}(x)\overline{p}_{α,b}(y),

where (α, ϕ) runs over all the elements in ⋃_{n=1}^{∞} P̃_{n,e,p}^{q} in (3.4.1). In (3.4.2), (α, b) runs over all the elements in ⋃_{n=1}^{∞} P̃_{q}^{n,e,p}. \overline{m}_{α,ϕ} (resp. \overline{q}_{α,ϕ,−}, \overline{p}_{α,b}) denotes the complex conjugate of m_{α,ϕ} (resp. q_{α,ϕ,−}, p_{α,b}), respectively.

Proof. First we show (3.4.1). We fix j and put h = h_{j} as before. Let us consider the expansion of Ω(X_{j}, Y_{j}; t^{h}) by making use of the first equality of (3.2.1). By applying the operator Θ_{q,X_{j}} on both sides of this expansion, we have

(3.4.3)  Ω_{q}^{j}(x, y; t) = ∑_{β}Θ_{q}(q_{β,+}(X_{j}; t^{h}))m_{β}(Y_{j}),

where β runs over j_{1}d-partitions of any size. Take β ∈ P_{n/h,j_{1}d}. In view of (3.3.5), we may only consider β such that qc ≡ 0 (mod p) in the sum (3.4.3), where c = c_{β,p}. By making use of the formula for Θ_{q}(q_{β,+}) which is similar to (3.3.5), we have

(3.4.4)  ∑_{β′∈O(β)} Θ_{q}(q_{β′,+})m_{β}′ = ∑_{β′∈O(β)}\{\frac{p}{c} ∑_{i=0}^{c−1} ζ^{qid}q_{θ^{i}(β′),+}\}m_{β}′

= \frac{p}{c} ∑_{i=0}^{c−1} ζ^{qid}q_{θ^{i}(β),+} ∑_{k=0}^{c−1} ζ^{−qkd}m_{θ^{k}(β)}.

Now there exists α ∈ P_{n,e} such that β = α{j}. Then c coincides with c_{α,p}. Moreover the action of θ on β is compatible with that on α. Then the last formula of (3.4.4) can be written as

∑_{ϕ∈Γ_{α}^{∧}}{ϕ(τ^{j}) ∑_{i=0}^{c−1} ζ^{qid}q_{θ^{i}(α){j},+}(X_{j}; t^{h})}{ϕ(τ^{−j}) ∑_{k=0}^{c−1} ζ^{−kid}m_{θ^{i}(α){j}}(Y_{j})}

= ∑_{ϕ∈Γ_{α}^{∧}} q_{α,ϕ,+}^{j}(x; t))\overline{m_{α,ϕ}^{j}(y)}.

It follows that

Ω_{q}^{j}(x, y; t) = ∑_{(α,ϕ)} q_{α,ϕ,+}^{j}(x; t)\overline{m_{α,ϕ}^{j}(y)},

where (α, ϕ) runs over all the elements in ⋃_{n=1}^{∞} P̃_{n,e,p}^{q}. This shows the first equality of (3.4.1). The second equality is shown similarly.

Next we show (3.4.2). We consider the expansion of Ω(X_{j}, Y_{j}; t^{h}) by (3.2.2). By applying Θ_{q,X_{j}} on this equality, together with (3.3.4), we have

(3.4.5)  Ω_{q}^{j}(x, y; t) = p ∑_{β}z_{β}(t^{h})^{−1}p_{β}(X_{j})\overline{p_{β}(Y_{j})},

where β runs over j_{1}d-partitions of any size such that hΔ(β) ≡ q (mod p) (see the discussion in 3.3). z_{β}(t^{h}) is the function defined similar to (2.1.3) for a j_{1}d-partition β, by replacing ζ by ζ^{h}. We fix β ∈ P_{n/h,j_{1}d} as above. Then there exists α ∈ P_{n,e} such that β = α[j]. Hence α satisfies (2.6.1), and Δ(α) = hΔ(β). Let r be the number of all the pairs (α, b) ∈ P̃_{q}^{n,e,p} for a fixed α. Then by (2.10.2), we have

(3.4.6)  p_{β}(X_{j})\overline{p_{β}(Y_{j})} = h^{−2l(β)}r^{−1} ∑_{b} p_{α,b}^{j}(x)\overline{p_{α,b}^{j}(y)},

where the sum is taken over all the pairs (α, b) for a fixed α. Now using the explicit description of z_{α}(t) in (3.1.3) and subsequent parts, one can check that

(3.4.7)  z_{β}(t^{h}) = h^{−2l(β)}z_{α}(t).

Substituting (3.4.6) and (3.4.7) into (3.4.5), together with (3.3.8), we have

Ω_{q}^{j}(x, y; t) = ∑_{(α,b)} z_{α,b}(t)^{−1}p_{α,b}^{j}(x)\overline{p_{α,b}^{j}(y)},

where (α, b) runs over all the elements in ⋃_{n=1}^{∞} P̃_{q}^{n,e,p}. This implies (3.4.2), and the proposition is proved. □

3.5. As in [S], we denote by Z_{n}^{0,0} = Z_{n}^{0,0}(m) the set of e-partitions α such that |α| = n and that each α^{(k)} is regarded as an element in ℤ^{m_{k}}, written in the form α^{(k)} : α_{1}^{(k)} ≥ ⋯ ≥ α_{m_{k}}^{(k)} ≥ 0. We fix an integer r > 0, and consider an e-partition Λ^{0} = Λ^{0}(m) = (Λ_{0}, …, Λ_{e−1}) defined by

(3.5.1)  Λ_{i} : (m_{i} − 1)r > ⋯ > 2r > r > 0  for 0 ≤ i ≤ e − 1.

In [S], the set Z_{n}^{r,s} of symbols are introduced. In this paper, we are only concerned with the symbol of the form Z_{n}^{r,0} = Z_{n}^{r,0}(m), i.e., the set of e-partitions of the form Λ = α+Λ^{0}, where α ∈ Z_{n}^{0,0} and the sum is taken entry-wise. We denote by Λ = Λ(α) if Λ = α + Λ^{0}, and call it the e-symbol of type (r, 0) corresponding to α. We write |Λ| = n if Λ ∈ Z_{n}^{r,0}.

As in [S], we consider the shift operation Z_{n}^{r,0}(m) → Z_{n}^{r,0}(m′) for m′ = (m_{0} + 1, …, m_{e−1} + 1) by associating Λ′ = α + Λ^{0}(m′) to Λ = α + Λ^{0}(m), where α is regarded as an element of Z_{n}^{0,0}(m′) by adding 0 in the entries of α. We often consider the symbols as elements in the equivalence class in the set ∐_{m′} Z_{n}^{r,0}(m′) under the shift operation. We denote by Z̄_{n}^{r,0} the set of equivalence classes. Note that Λ^{0} is regarded as a symbol in Z_{n}^{r,0} with n = 0.

Two elements Λ and Λ′ in Z̄_{n}^{r,0} are said to be similar (and denoted as Λ ∼ Λ′) if there exist representatives in Z_{n}^{r,0} such that all the entries of them coincide each other with multiplicities. The equivalence class with respect to this relation is called a similarity class in Z̄_{n}^{r,0}.

We shall define a function a : Z̄_{n}^{r,0} → ℕ. For Λ ∈ Z_{n}^{r,0}, we put

(3.5.2)  a(Λ) = ∑_{λ,λ′∈Λ}min(λ, λ′) − ∑_{μ,μ′∈Λ^{0}} min(μ, μ′),

which induces a well-defined function a on Z̄_{n}^{r,0}. The a-function takes a constant value on each similarity class in Z_{n}^{r,0}. By using the bijection Z_{n}^{0,0} ≃ Z_{n}^{r,0}, a-functions and similarity classes are defined for Z_{n}^{0,0}, for which we use the same notation as for Z_{n}^{0,0}.

Under the natural bijection P_{n,e} ≃ Z_{n}^{0,0} ≃ Z_{n}^{r,0}, the operation θ is transfered to the action on Z_{n}^{r,0}, which we denote also by θ. It is clear that θ preserves each similarity class in Z_{n}^{r,0}.

3.6. As discussed in [S], {s_{α}(x)} and {m_{α}(x)} form bases of the ℤ[t]-module ℤ[t] ⊗ Ξ, and {q_{α,±}(x; t)} form a basis of Ξ_{ℚ(t)} = ℚ(t) ⊗_{ℤ} Ξ. Moreover, {p_{α}(x)} gives a basis of ℂ(t)-space Ξ_{ℂ(t)} = ℂ(t) ⊗_{ℤ} Ξ.

Following [S], we define a scalar product on Ξ_{ℚ}[t] by the condition that

(3.6.1)  ⟨q_{α,+}(x; t), m_{β}(x)⟩= δ_{α,β},

and extend it to a sesquilinear form on Ξ_{ℂ(t)}. Then by Proposition 3.2, we also have

(3.6.2)  ⟨m_{α}(x), q_{β,−}(x; t)⟩= δ_{α,β},
⟨p_{α}(x), p_{β}(x)⟩= z_{α}(t)δ_{α,β}.

The Hall-Littlewood functions P_{Λ}^{±}(x; t) and Q_{Λ}^{±}(x; t) were introduced in [S]. By Corollary 4.6 in [S], they satisfy the following formulas.

(3.6.3)  Ω(x, y; t) = ∑_{Λ,Λ′} b_{Λ,Λ′}(t)P_{Λ}^{+}(x; t)P_{Λ′}^{−}(y; t).
(3.6.4)  Ω(x, y; t) = ∑_{Λ}Q_{Λ}^{+}(x; t)P_{Λ}^{−}(y; t) = ∑_{Λ}P_{Λ}^{+}(x; t)Q_{Λ}^{−}(y; t),

where in (3.6.3), Λ, Λ′ run over all the elements in ⋃_{n=1}^{∞} Z_{n}^{r,0}, and b_{Λ,Λ′}(t) = 0 unless |Λ| = |Λ′| and Λ ∼ Λ′. In (3.6.4), Λ runs over all the elements in ⋃_{n=1}^{∞} Z_{n}^{r,0}.

We now define a total order ≺ on the set Z_{n}^{0,0} satisfying the condition that each similarity class forms an interval and that a(α) > a(β) implies that α ≺ β. The following characterization of Hall-Littlewood functions was given in Proposition 4.8 in [S].

Proposition 3.7. (i) P_{Λ}^{±} are characterized by the following two properties.

(a) P_{Λ}^{±}(x; t) can be expressed in terms of s_{β}(x) as

P_{Λ}^{±} = s_{α} + ∑_{β}u_{α,β}^{±}s_{β}

with u_{α,β}^{±} ∈ ℚ(t), where Λ = Λ(α), and u_{α,β}^{±} = 0 unless β ≺ α and β ≁ α.

(b) ⟨P_{Λ}^{+}, P_{Λ′}^{−}⟩= 0 unless Λ ∼ Λ′. (ii) The functions {Q_{Λ}^{±}} are characterized as the dual basis of {P_{Λ}^{±}}, i.e.,

⟨P_{Λ}^{+}, Q_{Λ′}^{−}⟩=⟨Q_{Λ}^{+}, P_{Λ′}^{−}⟩= δ_{Λ,Λ′}.

3.8. We now define a sesquilinear form on Ξ_{ℂ(t)}^{n}(p, q) by the condition that

(3.8.1)  ⟨q_{z,+}(x; t), m_{z′}(x)⟩= δ_{z,z′}

for z, z′ ∈ P̃_{n,e,p}^{q}. Then by Proposition 3.4, we also have

(3.8.2)  ⟨m_{z}(x), q_{z′}(x; t)⟩= δ_{z,z′}  (z, z′ ∈ P̃_{n,e,p}^{q}),
⟨p_{ξ}(x), p_{ξ′}(x)⟩= z_{ξ}(t)δ_{ξ,ξ′}  (ξ, ξ′ ∈ P̃_{q}^{n,e,p}).

Let us denote by Z̃_{n,q}^{0,0} (resp. Z̃_{n,q}^{r,0} ) the set of pairs (α, ϕ) (resp. (Λ(α), ϕ)) such that (α, ϕ) ∈ P̃_{n,e,p}^{q} and that α ∈ Z_{n}^{0,0}. We often identify Z̃_{n,q}^{0,0} and Z̃_{n,q}^{r,0} with P̃_{n,e,p}^{q}. A similarity class and an a-function on Z̃_{n,q}^{0,0} are defined as those inherited from Z_{n}^{0,0}. We define a total order ≺ on Z̃_{n,q}^{0,0} compatible with the order on Z_{n}^{0,0}, i.e., for z = (α, ϕ), z′ = (α′, ϕ′) ∈ Z̃_{n,q}^{0,0}, we have z ≺ z′ if a(α) > a(α′), and the set {(α, ϕ)}, where α falls in a fixed similarity class in Z_{n}^{0,0}, form an interval in this order.

The scalar product on Ξ_{ℂ(t)} given in 3.6 induces a sesquilinear form ⟨ , ⟩_{j} on the space ℂ(t) ⊗ Ξ_{p,q}^{n/h_{j}}(X_{j}), which satisfies the formula

(3.8.3)  ⟨q_{α,ϕ,+}^{j}(x), m_{α′,ϕ′}^{j}(x)⟩_{j} = \begin{cases}ϕ(τ^{j})\overline{ϕ′(τ^{j})}c_{α,p} & if α = α′ and c_{α,p} | j, \\ 0 & otherwise. \end{cases}

This implies that for u = (u^{j}), v = (v^{j}) ∈ Ξ_{ℂ(t)}^{n}(p, q) = ⨁_{j=0}^{p−1} ℂ(t) ⊗ Ξ_{p,q}^{n/h_{j}}(X_{j}), the following formula holds, since it certainly holds for u = q_{α,ϕ,+} and v = m_{α′,ϕ′} by (3.8.3).

(3.8.4)  ⟨u, v⟩= \frac{1}{p} ∑_{j=0}^{p−1}⟨u^{j}, v^{j}⟩_{j} .

3.9. In view of (3.3.2) and (3.6.1), the automorphism θ leaves the scalar product on Ξ_{ℚ(t)} invariant. It follows by Proposition 3.7 that we have

θ(P_{Λ}^{±}) = P_{θ(Λ)}^{±}, θ(Q_{Λ}^{±}) = Q_{θ(Λ)}^{±}.

Hence a similar formula as (3.3.5) holds also for P_{Λ}^{±} (resp. for Q_{Λ}^{±}), by replacing s_{θ^{i}(α)} by P_{θ^{i}(Λ)}^{±} (resp. by Q_{θ^{i}(Λ)}^{±}). Now for each z = (α, ϕ) ∈ Z̃_{n,q}^{0,0}, we define

(3.9.1)  P_{z}^{±} = (P_{z}^{±,j})_{0≤j<p},  Q_{z}^{±} = (Q_{z}^{±.j})_{0≤j<p},

where P_{z}^{±,j} and Q_{z}^{±,j} are defined as in the case of s_{α,ϕ}(x) by replacing s_{θ^{i}(α){j}} in (2.10.1) by P_{θ^{i}(Λ){j}}^{±} and Q_{θ^{i}(Λ){j}}^{±}. By using a similar argument as in 3.3, we see that the sets {P_{z}^{±}} and {Q_{z}^{±}} give rise to bases of Ξ_{ℂ(t)}^{n}(p, q).

It is known that P_{Λ}^{±}(x; 0) = s_{α}(x) for Λ(α) = Λ by [S]. Hence by (3.9.1), we have

(3.9.2)  P_{Λ(α),ϕ}^{±}(x; 0) = s_{α,ϕ}(x).

The functions P_{z}^{±}(x) and Q_{z}^{±}(x) are characterized by the following properties, in analogy to Proposition 3.7.

Proposition 3.10. (i) P_{z}^{±} (z ∈ Z̃_{n,q}^{r,0}) are characterized by the following two properties.

(a) P_{z}^{±}(x; t) can be expressed in terms of s_{z′}(x) as

P_{z}^{±} = s_{z} + ∑_{z′} u_{z,z′}^{±}s_{z′}

with u_{z,z′}^{±} ∈ ℂ(t), and u_{z,z′}^{±} = 0 unless z′ ≺ z and z′ ≁ z.

(b) ⟨P_{z}^{+}, P_{z′}^{−}⟩= 0 unless z ∼ z′. (ii) The functions {Q_{z}^{±}} are characterized as the dual basis of {P_{z}^{±}}, i.e.,

⟨P_{z}^{+}, Q_{z′}^{−}⟩=⟨Q_{z}^{+}, P_{z′}^{−}⟩= δ_{z,z′} (z, z′ ∈ Z̃_{n,q}^{r,0}).

Proof. By substituting t = 0 into (3.4.2), we have

Ω_{q}(x, y; 0) = ∑_{(α,b)} z_{α,b}^{−1}p_{α,b}(x)\overline{p}_{α,b}(y),

where z_{α,b} = |Z_{W} (w)| for w = w_{α}(b). On the other hand, starting from the expansion of Ω(x, y; 0) by means of Schur functions s_{α}(x) as given in Remark 4.9 in [S], by using a similar argument as in the proof of (3.4.1), one can show that

Ω_{q}(x, y; 0) = ∑_{(α,ϕ)} s_{α,ϕ}(x)s_{α,ϕ}(y).

It follows that one can define a hermitian form on Ξ_{ℂ}^{n}(p, q) satisfying the properties that ⟨p_{ξ}, p_{ξ′}⟩ = δ_{ξ,ξ′} (ξ, ξ′ ∈ P̃_{q}^{n,e,p}), and that ⟨s_{z}, s_{z′}⟩ = δ_{z,z′} (z, z′ ∈ P̃_{n,e,p}^{q}). Hence the sesquilinear form on Ξ_{ℂ(t)}^{n}(p, q) is reduced to the hermitian form on Ξ_{ℂ}^{n}(p, q) by substituting t = 0. Now the arguments in the proof of Proposition 4.8 and Remark 4.9 in [S] can be applied to our situation, and one can show that there exist unique functions satisfying the properties (i), (ii) in the proposition, respectively. So, we have only to show that P_{z}^{±}, Q_{z}^{±} satisfy the properties in the proposition.

First we note that ⟨P_{Λ,ϕ}^{+,j}, P_{Λ′,ϕ′}^{−,j}⟩ = 0 unless Λ ∼ Λ′ by Proposition 3.7. It follows from (3.8.4) that we have ⟨P_{z}^{+}, P_{z′}^{−}⟩= 0 unless z ∼ z′. This shows the property (b) of (i). Next we shall show the property (a). Clearly the statement (a) is equivalent to the statement that

(3.10.1)  P_{z}^{±} = s_{z} + ∑_{z′} d_{z,z′}^{±}P_{z′}^{±}

for any z, z′ ∈ Z̃_{n,q}^{r,0}, where d_{z,z′}^{±} = 0 unless z′ ≺ z and z ≁ z′.

We consider the equation (3.10.1) for a fixed z with respect to the + sign. Take z′ such that z′ ≺ z and z ≁ z′. Taking the scalar product with P_{z′}^{−} on both sides of (3.10.1), we have

(3.10.2)  ⟨s_{z}, P_{z′}^{−}⟩+ ∑_{z′′∼z′} d_{z,z′′}^{+}⟨P_{z′′}^{+}, P_{z′}^{−}⟩= 0.

When z′ runs over all the elements in a fixed similarity class, (3.10.2) can be regarded as a system of equations with unknown variables {d_{z,z′′}^{+}}. As in the arguments in the first part of the proof (cf. Remark 4.9 in [S]), the matrix (⟨P_{z′′}^{+}, P_{z′}^{−}⟩) is non-singular. Hence (3.10.2) has a unique solution {d_{z,z′′}^{+}}. Put

F_{z} = s_{z} + ∑_{z′′} d_{z,z′′}^{+}P_{z′′}^{+}

using thus determined d_{z,z′′}^{+} for z′′ ≺ z and z ≁ z′′. Then clearly we have

(3.10.3)  ⟨P_{z}^{+}, P_{z′}^{−}⟩=⟨F_{z}, P_{z′}^{−}⟩  for z′ ≺ z and z′ ≁ z.

On the other hand, it follows from Proposition 3.7 that we have

⟨P_{Λ(α)}^{+}, P_{Λ′}^{−}⟩=⟨s_{α}, P_{Λ′}^{−}⟩ for Λ(α) ≺ Λ′ or Λ(α) ∼ Λ′.

This implies, in view of (3.8.4), that

(3.10.4)  ⟨P_{z}^{+}, P_{z′}^{−}⟩=⟨s_{z}, P_{z′}^{−}⟩=⟨F_{z}, P_{z′}^{−}⟩  for z ≺ z′ or z ∼ z′.

Now (3.10.3) and (3.10.4) implies that P_{z}^{+} = F_{z}, and (3.10.1) is proved for P_{z}^{+}. The proof for P_{z}^{−} is similar.

Finally, we show (ii). By Proposition 3.7 (ii), we have

⟨P_{z}^{+,j}, Q_{z′}^{−,j}⟩_{j} = \begin{cases}ϕ(τ^{j})\overline{ϕ′(τ^{j})}c_{α,p} & if c_{α,p} | j and z = z′, \\ 0 & otherwise. \end{cases}

This implies, by (3.8.4), that ⟨P_{z}^{+}, Q_{z′}^{−}⟩ = δ_{z,z′}. The formula ⟨Q_{z}^{+}, P_{z′}^{−}⟩ = δ_{z,z′} is shown similarly. □

As a corollary to Proposition 3.10, we have

Corollary 3.11. Ω_{q}(x, y; t) has the following expansions.

(3.11.1)  Ω_{q}(x.y; t) = ∑_{z,z′} b_{z,z′}(t)P_{z}^{+}(x; t)P̄_{z′}^{−}(y; t)
(3.11.2)  Ω_{q}(x, y; t) = ∑_{z} Q_{z}^{+}(x; t)P̄_{z}^{−}(y; t) = ∑_{z} P_{z}^{+}(x; t)Q̄_{z}^{−}(y; t),

where z = (Λ, ϕ), z′ = (Λ′, ϕ′) run over all the elements in ⋃_{n=1}^{∞} Z̃_{n,q}^{r,0}, and b_{z,z′} = 0 unless |Λ| = |Λ′| and Λ ∼ Λ′ in (3.9.1). In (3.9.2), z = (Λ, ϕ) runs over all the elements in ⋃_{n=1}^{∞} Z̃_{n,q}^{r,0}.

Proof. (3.11.2) follows from (3.4.1) and Proposition 3.10 (ii) by a standard argument, (e.g. [M, I, 4.6]). We show (3.11.1). We write P_{z}^{−} = ∑_{z′} a_{z,z′}Q_{z′}^{−}. Since ⟨P_{z′}^{+}, P_{z}^{−}⟩ = a_{z,z′}, Proposition 3.10 (b) implies that the transition matrix between two bases {P_{z}^{−}} and {Q_{z}^{−}} is a block diagonal matrix with respect to the partition by similarity classes. Hence its inverse matrix is also block diagonal, i.e., Q_{z}^{−} is written as Q_{z}^{−} = ∑_{z′} b_{z,z′}P_{z′}^{−}, where b_{z,z′} = 0 unless z ∼ z′. Substituting this into the second formula in (3.11.2), we obtain (3.11.1). Thus the corollary is proved. □

4. Green functions associated to G(e, p, n)

4.1. For functions f, h on σ^{q}W, we define an inner product ⟨f, h⟩_{q} by

⟨f, h⟩_{q} = |W |^{−1} ∑_{w∈σ^{q}W} f(w)\overline{h(w)}.

It is known in general that the number of σ^{q}-stable irreducible characters is equal to the number of W -orbits in σ^{q}W. An σ^{q}-stable irreducible character χ on W can be extended to an irreducible character of ⟨σ^{q}⟩⋉W, in e/q distinct ways. We fix an extension χ̃ of χ. Then χ̃ gives a function on σ^{q}W by restriction, which is constant on each W -orbit. It is known that

(4.1.1)  ⟨χ̃, χ̃′⟩_{q} = \begin{cases}1 & if χ̃ = χ̃′, \\ 0 & if χ ≠ χ′. \end{cases}

Now the set of σ^{q}-stable irreducible characters is parametrized by P̃_{W}^{q}. Also the set of W -orbits in σ^{q}W is parametrized by P̃_{q}^{W}. We fix a total order on P̃_{W}^{q} ≃ Z̃_{n,q}^{0,0} as in 3.6. Let X = (χ̃^{z}(ξ)) (z ∈ P̃_{W}^{q} , ξ ∈ P̃_{q}^{W}) be the character table of σ^{q}W. We define H as the diagonal matrix indexed by P̃_{n,e,p}^{q} whose diagonal entry corresponding to (ξ, ξ) is z_{ξ}^{−1} = |Z_{W} (w_{α}(b))|^{−1} for ξ = (α, b). Then (4.1.1) can be written in a matrix form

(4.1.2)  ^{t}XHX̄ = I.

4.2. Let M = ℂ^{n} be the natural reflection representation of W̃. We consider M as a W -module by the restriction. Let S(M) be the symmetric algebra of M, and I_{+} the ideal of S(M) generated by W -invariant homogeneous vectors of strictly positive degree. We denote by R = S(M)/I_{+} the coinvariant algebra of W. The Poincaré polynomial P_{W} (t) associated to W is defined in terms of the graded algebra R = ⨁ R_{i}. In our case, P_{W} (t) is explicitly given as

P_{W} (t) = ∏_{i=1}^{n−1} \frac{t^{ei} − 1}{t − 1} · \frac{t^{dn} − 1}{t − 1}.

Since σ normalizes W, σ acts naturally on R, stabilizing each summand R_{i}. We denote by R̃_{i} the thus obtained W̃-module.

Let det_{M} be the linear character of W̃ defined by the determinant on M. For each W̃-stable function f on σ^{q}W, we define R_{q}(f) ∈ ℂ(t) by

(4.2.1)  R_{q}(f) = (ζ^{qd}t^{dn} − 1) ∏_{i=1}^{n−1}(t^{ei} − 1) · \frac{1}{|W |} ∑_{w∈σ^{q}W} \frac{det_{M}(w)f(w)}{det_{M}(t · id_{M} −w)}.

Then we have

R_{q}(f) = ∑_{i}⟨f, R̃_{i}⟩_{q} t^{i},

and, in particular, R_{q}(χ̃^{z}) ∈ ℤ[ζ][t] for z ∈ P̃_{W}^{q}.

4.3. Let us define a square matrix Ω_{q} = (ω_{z,z′}) indexed by Z̃_{n,q}^{0,0} by
ω_{z,z′} = t^{N^{∗}}R_{q}(χ̃^{z} ⊗ χ̃^{z′} ⊗ \overline{det}_{M}),

where N^{∗} is the number of reflections in W. Also we define a matrix Ω′_{q} = (ω′_{z,z′}) by

(4.3.1)  ω′_{z,z′} = t^{N^{∗}}R_{q}(χ̃^{z} ⊗ \overline{χ̃^{z′}} ⊗ \overline{det}_{M}).

In either of the formulas, \overline{det}_{M} etc. denote the complex conjugates of them. Let P = (p_{z,z′}) and Λ = (λ_{z,z′}) be the matrices indexed by Z̃_{n,q}^{0,0}. We consider the following system of equations with unknown variables p_{z,z′}, λ_{z,z′};

λ_{z,z′} = 0 unless z ∼ z′,
(4.3.2)  p_{z,z′} = 0 unless either z ≻ z′ and z ≁ z′, or z = z′,
p_{z,z′} = t^{a(z)},
P Λ^{t}P = Ω_{q}.

We now assume that m_{0} = m_{1} = ⋯ = m_{e−1}. Then as discussed in [S, 1.5], the system of equations (4.3.2) is equivalent to the system of equations

(4.3.3)  P′Λ′ ^{t}P′′ = Ω′_{q},

where either of P′ = (p′_{z,z′}) and P′′ = (p_{z,z′}) satisfies similar conditions as in the second and third one in (4.3.2), and Λ′ = (λ_{z,z′}) as in the first one. (Actually, it is shown that P = P′). In the remainder of this section, we shall show that the solution of (4.3.3) can be described in a combinatorial way.

Remark 4.4. In the case where e = p = 2, G(e, e, n) is the Weyl group of type D_{n}. In this case, the equation (4.3.2) coincides with (4.3.3), and the solution for it describes the Green functions of SO_{2n} defined over a finite field of odd characteristic of split type (resp. non-split type) if q = 0 (resp. q = 1).

4.5. For two bases X, Y of Ξ_{ℂ(t)}^{n}(p, q), we denote by M(X, Y) the transition matrix between X and Y. Let X_{±}(t) be the transition matrix M(p, P^{±}) between the power sum symmetric functions {p_{ξ}} and the Hall-Littlewood functions {P_{z}^{±}}, i.e.,

(4.5.1)  p_{ξ}(x) = ∑_{z} X_{ξ,±}^{z}(t)P_{z}^{±}(x; t).

(Here ξ ∈ P̃_{q}^{n,e,p} is the row-index and z = (Λ, ϕ) ∈ P̃_{n,e,p}^{q} is the column-index.) Then by (3.9.1) and by the properties of P_{Λ} shown in [S], we see that X_{ξ;±}^{z}(t) ∈ ℂ(t) and is equal to zero unless |z| = |ξ|. (For z = (α, ϕ) and ξ = (β, b), we put |z| = |α| and |ξ| = |β|, respectively.) Moreover since P_{z}^{±}(x; 0) = s_{z}(x) by (3.9.2), we have

(4.5.2)  X_{ξ;±}^{z}(0) = χ̃^{α,ϕ}(w_{β}(b))

by Proposition 2.11. It follows that X_{±}(0) = M(p, s) is the character table of W. In particular, X_{±}(0) is independent of the sign, which we denote simply as X(0). By combining (3.4.2) and (3.11.1), we have

∑_{|ξ|=n} z_{ξ}(t)^{−1}p_{ξ}(x)\overline{p}_{ξ}(y) = ∑_{z,z′} b_{z,z′}(t)P_{z}^{+}(x; t)P̄_{z′}^{−}(y; t),

where in the right hand side, z, z′ run over all elements in Z̃_{n,q}^{r,0}. We put D(t) = D_{−} = (b_{z,z′}(t)), the matrix indexed by P̃_{n,e,p}^{q} ≃ Z̃_{n,q}^{0,0}, and denote by Z(t) the diagonal matrix indexed by P̃_{q}^{n,e,p}, where the entry corresponding to (ξ, ξ) is given by z_{ξ}(t).

Substituting (4.5.1) into the above equation, we have

tX_{+}(t)Z(t)^{−1}X̄_{−}(t) = D(t).

If we put Λ(t) = D(t)^{−1}, the above formula is equivalent to

(4.5.3)  X̄_{−}(t)Λ(t)^{t}X_{+}(t) = Z(t).

4.6. Let K_{±}(t) = M(s, P^{±}) be the transition matrix between Schur functions and Hall-Littlewood functions, i.e., for z, z′ ∈ P̃_{n,e,p}^{q},

(4.6.1)  s_{z′}(x) = ∑_{z} K_{z′,z}^{±}(t)P_{z}^{±}(x; t).

Then by Proposition 3.10, K_{±}(t) is a block lower triangular matrix with identity diagonal blocks, with entries in ℂ(t). Since M(s, P^{±}) = M(p, s)^{−1}M(p, P^{±}), we have K_{±}(t) = X(0)^{−1}X_{±}(t). Substituting this into (4.5.3), we have

(4.6.2)  K_{−}(t)Λ(t)^{t}K_{+}(t) = X̄(0)^{−1}Z(t)^{t}X(0)^{−1}.
We now define Green functions Q_{ξ;±}^{z}(t) ∈ ℂ(t) by
Q_{ξ;±}^{z}(t) = t^{a(z)}X_{ξ;±}^{z}(t^{−1}).

If we put K̃_{z′,z}^{±}(t) = t^{a(z)}K_{z′,z}^{±}(t^{−1}), Q_{ξ;±}^{z}(t) can be written as

Q_{ξ;±}^{z}(t) = ∑_{z′′} χ̃^{z′′}(w_{β}(b))K̃_{z′′,z}^{±}(t)

for ξ = (β, b). Let K̃_{±}(t) = (K̃_{z′,z}^{±}(t)). Then K̃_{±}(t) = K_{±}(t^{−1})T, where T is a diagonal matrix with diagonal entries t^{a(z)}. Hence (4.6.2) can be rewritten as

(4.6.3)  K̃_{−}(t)Λ′(t)^{t}K̃_{+}(t) = X̄(0)^{−1}Z(t^{−1})^{t}X(0)^{−1},

where Λ′(t) = T^{−1}Λ(t^{−1})T. Here Λ′(t) is still a block diagonal matrix, and K̃_{±}(t) are block lower triangular matrices, where the diagonal blocks consist of scalar matrices t^{a(z)}I.

Put Λ̃(t) = t^{−n}𝔾(t)Λ′(t) with

𝔾(t) = t^{N^{∗}}(ζ^{qd}t^{dn} − 1) ∏_{i=1}^{n−1}(t^{ei} − 1).

The following result gives a combinatorial description of the solution of (4.3.3).

Theorem 4.7. We have

K̃_{−}(t)Λ̃(t)^{t}K̃_{+}(t) = Ω′_{q}.

Hence P′ = K̃_{−}(t), P′′ = K̃_{+}(t) and Λ′ = Λ̃(t) give a solution for the equation (4.3.3).

Proof. Let Y be the right hand side of (4.6.3). We shall compute Y. Let w_{α}(b) be the element in σ^{q}W corresponding to ξ = (α, b). Since X(0) is the character table of σ^{q}W, we have ^{t}X(0)HX̄(0) = I by (4.1.2), where H is the diagonal matrix with diagonal entries z_{ξ}^{−1}. It follows that

(4.7.1)  Y = ^{t}X(0)HZ(t^{−1})HX̄(0).

Now it is known that if α = (α_{j}^{(k)}) ∈ P_{n,e}, we have

det_{M}(t · id_{M} −w_{α}(b)) = ∏_{k=0}^{e−1} ∏_{j=1}^{l(α^{(k)})}(t^{α_{j}^{(k)}} − ζ^{k}).

Hence by (3.3.8), z_{α,b}(t^{−1}) = z_{α,b}t^{n}det_{V} (t · id_{V} −w_{α}(b))^{−1}. In particular, the (z, z′)- entry of Y is equal to

t^{n}|W |^{−1} ∑_{w∈σ^{q}W} \frac{χ̃^{z}(w)\overline{χ̃^{z′}}(w)}{det_{M}(t · id_{M} −w)}.

Therefore Ω′_{q} = t^{−n}𝔾(t)Y and the theorem follows. □

4.8. Recall that Kostka functions K_{α,β}^{±}(t) associated to symbols in Z_{n}^{r,0}(m) are defined in [S] by the formula

(4.8.1)  s_{α}(x) = ∑_{β∈Z_{n}^{0,0}} K_{α,β}^{±}(t)P_{Λ(β)}^{±}(x; t).

By making use of the preceding discussions, we can describe the Kostka functions K_{z,z′}^{±} associated to Z̃_{n,q}^{r,0} in terms of K_{α,β}^{±}. For α, α′ ∈ Z_{n}^{0,0}, put c = c_{α,p}, c′ = c_{α′,p}. We assume that cq ≡ 0, c′q ≡ 0 (mod p). For each j such that c′ | j, put

(4.8.2)  L_{α,α′}^{j,±} = \begin{cases}∑_{i=0}^{c′−1} ζ^{−qid}K_{α\{j\},θ^{i}(α′)\{j\}}^{±} & if c | j, \\ 0 & otherwise. \end{cases}

We denote by L_{α,α′}^{±} the diagonal matrix of size |Γ_{α}′| whose ii-entry is given by L_{α,α′}^{c′i,±}. Let Φ′ = (ϕ′(τ^{j}))_{ϕ′,j} be the character table of Γ_{α}′ (ϕ′ ∈ Γ_{α′}^{∧}, c′|j), and let Φ = (ϕ(τ^{j}))_{ϕ,j} be a part of the character table of Γ_{α}, i.e., ϕ ∈ Γ_{α}^{∧}, but j is taken under the condition that c | j and c′ | j. For fixed α, α′ as above, we consider the matrix (K_{ϕ,ϕ′}^{±}), indexed by ϕ ∈ Γ_{α}^{∧}, ϕ′ ∈ Γ_{α′}^{∧}, defined by K_{ϕ,ϕ′}^{±} = K_{z,,z′}^{±}. We have the following proposition.

Proposition 4.9. The matrix (K_{ϕ,ϕ′}^{±}) can be written as

(K_{ϕ,ϕ′}^{±}) = |Γ_{α}′||Γ_{α}|^{−1}ΦL_{α,α′}^{±}Φ′^{−1}.

Proof. By using the relation (4.8.1) for Schur functions with respect to the variables X_{j}, we can express s_{α{j}} in terms of a linear combination of P_{β}^{±} with β ∈ Z_{n/h_{j}}^{0,0}. Then by applying Θ_{q} on both sides, we have

Θ_{q}(s_{α{j}}) = ∑_{α′} K_{α{j},α′{j}}^{±}Θ_{q}(P_{Λ(α′){j}}^{±}),

where c′ = c_{α′,p}, and α′ in the right hand side runs over all α′ ∈ Z_{n}^{0,0} such that j | c′ and that c′ ≡ 0 (mod p). Hence if we choose a representative α′ from each θ-orbit, the above formula turns out to be

(4.9.1)  Θ_{q}(s_{α{j}}) = ∑_{α′∈Z_{n}^{0,0}/∼_{p}} ∑_{i=0}^{c′−1} ζ^{−qid}K_{α{j},θ^{i}(α′){j}}^{±}Θ_{q}(P_{Λ(α′){j}}^{±}).

On the other hand, by using (4.6.1), we can express s_{z} in terms of a linear combination of P_{z′}^{±}. In particular, we have

s_{z}^{j} = ∑_{z′} K_{z,z′}^{±}P_{z′}^{±,j}

for any j such that 0 ≤ j < p. Now by making use of (3.3.5) and a similar formula for P_{Λ}, we can rewrite the above formula as

|Γ_{α}|^{−1}ϕ(τ^{j})Θ_{q}(s_{α{j}}) = ∑_{α′∈Z_{n}^{0,0}/∼_{p}} |Γ_{α}′|^{−1} ∑_{ϕ′∈Γ_{α′}^{∧}} K_{z,z′}^{±}ϕ′(τ^{j})Θ_{q}(P_{Λ(α′){j}}^{±})

for any j such that c′ | j. Note that the left hand side is understood to be 0 in the case where j is not a multiple of c. Since Θ_{q}(P_{Λ(α′{j}}^{±}) are linearly independent in

Ξ_{p,q}^{n/h_{j}}(X_{j}), by comparing with (4.9.1), we have

(4.9.2)  |Γ_{α}|^{−1}ϕ(τ^{j}) ∑_{i=0}^{c′−1} ζ^{−qid}K_{α{j},θ^{i}(α′){j}}^{±} = |Γ_{α}′|^{−1} ∑_{ϕ′} K_{ϕ,ϕ′}^{±}ϕ′(τ^{j})

for each α′. Now, (4.9.2) holds for any ϕ ∈ Γ_{α}^{∧} and any j such that c′ | j. Hence it can be translated to a matrix equation

|Γ_{α}|^{−1}ΦL_{α,α′}^{±} = |Γ_{α}′|^{−1}(K_{ϕ,ϕ′}^{±})Φ′.

This proves the proposition. □

The following special case would be worth mentioning.

Corollary 4.10. Assume that Γ_{α} = Γ_{α}′ = {1}. Then we have

K_{(α,1),(α′,1)}^{±} = ∑_{i=0}^{p−1} ζ^{−qid}K_{α,θ^{i}(α′)}^{±}.

As in the case of Green functions associated to G(e, 1, n), one can expect that, for z, z′ ∈ Z_{n,q}^{r,0}, K_{z,z′}^{±}(t) is a polynomial in t with positive integral coefficients. In view of Corollary 4.10, the conjecture for G(e, 1, n), i.e., K_{α,α′}^{±}(t) ∈ ℤ[t], implies that K_{z,z′}^{±}(t) ∈ K[t] with K = ℚ(ζ).

Remarks 4.11. (i) Preposition 4.9 asserts that Green functions associated to G(e, p, n) are completely described in terms of Green functions associated to G(e′, 1, n′) with various e′ | e, n′ | n. In the classical situation, this means that Green functions of type D_{n} can be described in terms of Green functions associated to Weyl groups of type B_{n}, and 𝔖_{n/2} for even n. However, note that this does not mean such Green functions are described in terms of Green functions of Sp_{2n} or SO_{2n+1}. In fact, the Green functions of Sp_{2n} or SO_{2n+1} are defined in terms of symbols of the type Z_{n}^{r,s}(m) with m = (m+1, m), while our Green functions are related to the symbols of the type Z_{n}^{r,0}(m) with m = (m, m), and the structure of similarity classes is different.

(ii) The group G(e, e, 2) coincides with the dihedral group of degree 2e. In particular, it coincides with the Weyl group of type A_{2}, B_{2} or G_{2} for e = 3, 4 or 6, respectively. The case where e = 3, our Green function coincides with the Green function associated to GL_{3}(F_{q}). However, as Lemma 5.4 in the next section shows, the Green function associated to G(e, e, 2) for e = 4 or e = 6 (for any r) does not coincide with the Green function of type B_{2} or G_{2}.

5. Examples

5.1. In this section, we give some examples of the matrices P = K̃_{−}(t) associated to the group G(e, e, n). But before doing it, we shall give a general remark in connection with fake degrees. Assume that W = G(e, p, n), and we consider the case where q = 0. We write P = (p_{z,z′}) for z, z′ ∈ P̃_{W}^{0} . For j = 0, …, e − 1, put β_{j} = (β^{(0)}, …, β^{(e−1)}), where β^{(j)} = (1^{n}) and β^{i} = ∅ for i ≠ j. In this case, Γ_{β_{j}} = {1}, and we identify z = (β_{j}, 1) with β_{j}. Then among P̃_{W}^{0} , F = {Λ(β_{0}), …, Λ(β_{d−1})} gives rise to a similarity class in Z̃_{n,0}^{r,0}, which has the maximum a-value.

For each z ∈ P̃_{W}^{0} , R_{0}(χ^{z}) coincides with the fake degree ∑_{i≥0}⟨χ^{z}, R_{i}⟩t^{i} (cf. 4.2). The following formula shows that the fake degree can be read from the matrix P, which is an analogue of Lemma 7.2 in [S]. The proof is completely similar to it.

Lemma 5.2. Let W = G(e, p, n) and assume that q = 0. Put b = a(β_{0}). Then we have

∑_{j=0}^{d−1} t^{−b}R(χ^{β^{j}})p_{z,β_{j}} = R(χ^{z}),

where R(χ^{β^{j}}) = t^{n(n−1)/2+jn}.

In particular, in the case where W = G(e, e, n), we have F = {Λ(β_{0})}, and p_{z,β_{0}} = t^{n(n−1)/2−b}R(χ^{z}).

5.3. Assume that W = G(e, e, 2), the dihedral group of order 2e. We shall determine the matrix P in the case where q = 0. Let α_{0}, α_{ij} be e-partitions defined in 2.16. The set P̃_{W}^{0} is given as follows.

α_{0,0}, α_{0,j} (1 ≤ j < e/2), α_{0,e/2}, α′_{0,e/2}, α_{0},

where in the case of Γ_{α} = {1}, we write as α to denote the element (α, 1) ∈ P̃_{W}^{0} . Moreover, we have Γ_{α} = ℤ/2ℤ for α = α_{0,e/2} (the case where e is even), and in this case, we write as α_{0,e/2}, α′_{0,e/2} instead of (α_{0,e/2}, 1), (α_{0.e/2}, −1), respectively. The similarity classes in Z̃_{n,0}^{r,0} are given as follows.

F_{1} = {Λ(α_{0,0})}, F_{2} = {Λ(α_{0,j}) | 1 ≤ j ≤ e/2}, F_{3} = {Λ(α_{0})},

where in F_{2}, Λ(α_{0,j}) for j = e/2 is understood as Λ(α_{0,e/2}), Λ(α′_{0,e/2}). The a-values on similarity classes are given as follows.

a(F_{1}) = e, a(F_{2}) = 1, a(F_{3}) = 0.

Note that the similarity classes and a-values of them are independent from the choice of r.

Next we describe irreducible characters χ^{z} associated to z ∈ P̃_{W}^{0} . If z = α_{0,0}, α_{0}, χ^{z} coincide with the sign character ε and the identity character 1 of W, respectively. If e is even, α_{0,e/2} and α′_{0,e/2} correspond two linear characters (long sign character η and short sigh character η′) of W. z = α_{0,1} corresponds to the reflection character ρ_{1} of W (hence deg ρ_{1} = 2). Other characters corresponding to α_{0,j} have all degree 2. We remark that the above partition of W^{∧} into similarity classes together with a-values coincide with the partition of W^{∧} into families and their a-values introduced by Lusztig [L1] in connection with the classification of unipotent characters of finite reductive groups, where 1, ε, ρ_{1} are the special characters corresponding to families.

The fake degrees R(χ^{z}) of W are given as follows.

(5.3.1)  R(1) = 1, R(ρ_{j}) = t^{j} + t^{e−j} (1 ≤ j < e/2)
R(η) = t^{e/2}, R(η′) = t^{e/2},
R(ε) = t^{e}.

We now define an order on the set W^{∧} by

ε, ρ_{1}, ρ_{2}, …, ρ_{m}, 1  if e = 2m + 1,
ε, ρ_{1}, ρ_{2}, …, ρ_{m−1}, η, η′, 1 if e = 2m,

and define the order on P̃_{W}^{0} accordingly. We consider the matrix equation P Λ^{t}P = Ω indexed by P̃_{W}^{0} . Following the partition of P̃_{W}^{0} into similarity classes, we regard P, Λ, Ω as block matrices. By our assumption, P and Λ have the following shape.

P = (\begin{matrix}t^{e} & 0 & 0 \\ P_{21} & tI_{k} & 0 \\ p_{31} & P_{32} & 1 \end{matrix}) , Λ = (\begin{matrix}λ_{11} & 0 & 0 \\ 0 & Λ_{22} & 0 \\ 0 & 0 & λ_{33} \end{matrix})

where k is equal to e/2 − 1 (resp. e/2 + 1) if e is odd (resp. even), Λ_{22} is a matrix of degree k. P_{21}, P_{32} are matrices of the shape (k, 1), (1, k), respectively.

Since the matrix equation P Λ^{t}P = Ω is completely the same as the matrix equation arising from the partition into families studied by Geck and Malle [GM], where the case of dihedral groups is explicitly computed, the matrix P in our case is described as follows. Note that in view of Lemma 5.2, the first column of P coincides with the fake degrees of W.

Lemma 5.4 ([GM]). Let W = G(e, e, 2) be the dihedral group of degree 2e. Assume that q = 0. Then the matrix P associated to Z̃_{n,0}^{r,0} is given as follows.

tP_{21} = \begin{cases}(t + t^{e−1}, t^{2} + t^{e−2}, …, t^{m} + t^{m+1}) & if e = 2m + 1, \\ (t + t^{e−1}, t^{2} + t^{e−2}, …, t^{m−1} + t^{m+1}, t^{m}, t^{m}) & if e = 2m. \end{cases}

and p_{31} = 1, P_{32} = (1, 0, …, 0).

Moreover, a part of the matrix Λ is given as follows.

λ_{11} = 1, (Λ_{22})_{1} = t^{−1}(t^{e} − 1) ·^{t}P_{21}, λ_{33} = t^{e−2}(t^{2} − 1)(t^{e} − 1),

where (Λ_{22})_{1} denotes the first row of the matrix Λ_{22}.

5.5. In the remainder of this section, we shall give some more examples of Green functions associated to G(e, e, n). Throughout those examples, we always assume that r = 2 and q = 0.

First assume that W = G(3, 3, 3). Then P̃_{W}^{0} is given as follows.

α_{1} = (1^{3}; −; −), α_{2} = (1^{2}; 1; −), α_{3} = (1; 1^{2}; −),
α_{4} = (1; 1; 1), α′_{4} = (1; 1; 1)′, α′′_{4} = (1; 1; 1)′′,
α_{5} = (21; −; −), α_{6} = (2; 1; −), α_{7} = (1; 2; −), α_{8} = (3; −; −).

Note that |Γ_{α_{4}}| = 3, and we write α_{4}, α′_{4}, α′′_{4} instead of (α, ϕ) with ϕ ∈ Γ_{α_{4}}. Γ_{α} = {1} for other α. Then the symbols and similarity classes in Z̃_{n,0}^{2,0} are given as

F_{1} = {Λ_{1} = (531; 420; 420)},
F_{2} = {Λ_{2} = (31; 30; 20), Λ_{3} = (30; 31; 20)},
F_{3} = {Λ_{4} = (1; 1; 1), Λ′_{4} = (1; 1; 1)′, Λ′′_{4} = (1; 1; 1)′′},
F_{4} = {Λ_{5} = (41; 20; 20)}, F_{5} = {Λ_{6} = (2; 1; 0), Λ_{7} = (1; 2; 0)},
F_{6} = {Λ_{8} = (3; 0; 0)}.

32 T. SHOJI

The matrices Λ(F_{i}) are given as follows;

Λ(F_{1}) = (1),  Λ(F_{2}) = (t^{6} − 1) (\begin{matrix}2t^{3} + 1 & t(t^{3} + 2) \\ t(t^{3} + 2) & 3t^{2} \end{matrix})
Λ(F_{3}) = t^{3}(t^{3} − 1)(t^{6} − 1)I_{3},  Λ(F_{4}) = (t^{3}(t^{3} − 1)(t^{6} − 1)),
Λ(F_{5}) = t^{3}(t^{3} − 1)^{2}(t^{6} − 1) (\begin{matrix}2 & t \\ t & 0 \end{matrix}) ,
Λ(F_{6}) = (t^{6}(t^{3} − 1)^{2}(t^{6} − 1)),

where I_{3} denotes the identity matrix of degree 3. The matrix P of Green functions is given in Table 1.

Table 1. G(3, 3, 3) (r = 2) \begin{matrix}(1^{3}; −; −) & t^{9} \\ (1^{2}; 1; −) & 2t^{7} + t^{4} & t^{4} \\ (1; 1^{2}; −) & t^{8} + 2t^{5} & & t^{4} \\ (1; 1; 1) & t^{6} + t^{3} & t^{3} & & t^{3} \\ (1; 1; 1)′ & t^{6} + t^{3} & t^{3} & & & t^{3} \\ (1; 1; 1)′′ & t^{6} + t^{3} & t^{3} & & & & t^{3} \\ (21; −; −) & t^{6} + t^{3} & t^{3} & & & & & t^{3} \\ (2; 1; −) & 2t^{4} + t & t & t^{3} & t & t & t & t & t \\ (1; 2; −) & t^{5} + 2t^{2} & 2t^{2} & & t^{2} & t^{2} & t^{2} & t^{2} & & t \\ (3; −; −) & 1 & 1 & & 1 & 1 & 1 & 1 & 1 & & 1 \end{matrix}

5.6. Assume that W = G(4, 4, 3). Then P̃_{W}^{0} is given as

α_{1} = (1^{3}; −; −; −), α_{2} = (1^{2}; 1; −; −), α_{3} = (1^{2}; −; 1; −),
α_{4} = (1; 1^{2}; −; −), α_{5} = (21; −; −; −), α_{6} = (1; 1; 1; −),
α_{7} = (2; 1; −; −), α_{8} = (2; −; 1; −), α_{9} = (1; 2; −; −),
α_{10} = (3; −; −; −).

The symbols and similarity classes in Z̃_{n,0}^{2,0} are given as

F_{1} = {Λ_{1} = (531; 420; 420; 420)},
F_{2} = {Λ_{2} = (31; 30; 20; 20), Λ_{3} = (31; 20; 30; 20), Λ_{4} = (30; 31; 20; 20)},
F_{3} = {Λ_{5} = (41; 20; 20; 20)},  F_{4} = {Λ_{6} = (1; 1; 1; 0)},
F_{5} = {Λ_{7} = (2; 1; 0; 0), Λ_{8} = (2; 0; 1; 0), Λ_{9} = (1; 2; 0; 0)},
F_{6} = {Λ_{10} = (3; 0; 0; 0)}.

The matrices Λ(F_{i}) are given as follows.

Λ(F_{1}) = (1),
Λ(F_{2}) = (t^{8} − 1) (\begin{matrix}(t^{5} + t^{4} + 1) & t(t^{5} + t + 1) & t(t^{4} + t^{2} + 1) \\ t(t^{5} + t + 1) & t^{2}(t^{2} + t + 1) & t^{2}(t^{2} + t + 1) \\ t(t^{4} + t^{2} + 1) & t^{2}(t^{2} + t + 1) & t^{2}(t^{4} + t^{2} + 1) \end{matrix}) ,
Λ(F_{3}) = (t^{5}(t^{3} − 1)(t^{8} − 1)), Λ(F_{4}) = (t^{4}(t^{2} + t + 1)(t^{4} − 1)(t^{8} − 1)),
Λ(F_{5}) = t^{5}(t^{3} − 1)(t^{4} − 1)(t^{8} − 1) (\begin{matrix}t + 1 & t^{2} & t \\ t^{2} & 0 & 0 \\ t & 0 & t^{2} \end{matrix}) ,
Λ(F_{6}) = (t^{9}(t^{3} − 1)(t^{4} − 1)(t^{8} − 1)).

The matrix of P of Green functions is given in Table 2.

Table 2. G(4, 4, 3) (r = 2) \begin{matrix}(1^{3}; −; −; −) & t^{12} \\ (1^{2}; 1; −; −) & t^{10} + t^{9} + t^{5} & t^{5} \\ (1^{2}; 1; −; 1) & t^{11} + t^{7} + t^{6} & & t^{5} \\ (1; 1^{2}; −; −) & t^{10} + t^{8} + t^{6} & & & t^{5} \\ (21; −; −; −) & t^{8} + t^{4} & t^{4} & & & t^{4} \\ (1; 1; 1; −) & t^{9} + t^{8} + t^{7} + t^{5} + t^{4} + t^{3} & t^{4} + t^{3} & & t^{4} & & t^{3} \\ (2; 1; −; −) & t^{6} + t^{5} & t & t^{4} & & t & t & t \\ (2; −; 1; −) & t^{7} + t^{3} + t^{2} & t^{3} + t^{2} & & & t^{3} & t^{2} & & t \\ (1; 2; −; −) & t^{6} + t^{4} + t^{2} & t^{2} & & t^{3} & t^{2} & t^{2} & & & t \\ (3; −; −; −) & 1 & 1 & & & 1 & 1 & 1 & & & 1 \end{matrix}

5.7. In this occasion, we shall correct some errors in [S]. The formula (2.5.2) in [S] does not hold in this form. If one wants to get this formula, the definition of powersum functions in 2.1 has to be replaced by

p_{r}^{(i)}(x) = ∑_{j=0}^{e−1} ζ^{−ij}p_{r}(x^{(j)}).

Then this leads to a different parametrization of irreducible characters of W in terms of e-partitions, i.e., the one obtained by taking the complex conjugate of the original characters. For example, (−; ⋯, −; 1^{n}) corresponds to the character det_{V} instead of \overline{det}_{V}.

However, it seems more econvenient to use the usual parametrization of irreducible characters of W as in 1.2 in [S]. In this case, the corrected form of the right hand side of (2.5.2) is obtained by replacing x and y from the old one. This change of variables will force some modifications to subsequent formulas. Instead of doing this, we argue as follows.

We define a new Ω(x, y; t) by exchanging the variables x and y from the old one,

(5.7.1)  Ω(x, y; t) = ∏_{k=0}^{e−1} ∏_{i,j} \frac{1 − tx_{i}^{(k)}y_{j}^{(k+1)}}{1 − x_{i}^{(k)}y_{j}^{(k)}} ,

and modify the definition of q_{r,±}^{(k)} as follows.

References

[GM] M. Geck and G. Malle; On special pieces in the unipotent variety, Experimental Math. 8 (1999).

[L1] G. Lusztig; Characters of reductive groups over a finite field, Annals of Math. Studies 107, Princeton University Press, Princeton, N.J., 1984.

[L2] G. Lusztig; Notes on unipotent classes, Asian J. Math. 1 (1997), 194 – 207.

[M] I.G. Macdonald; Symmetric functions and Hall Polynomials, second edition. Clarendon Press. Oxford 1995.

[S] T. Shoji; Green functions associated to complex reflection groups, to appear in J. Algebra.

[SS] M. Sakamoto and T. Shoji; Schur-Weyl reciprocity for Ariki-Koike algebras, J. Algebra 221 (1999), 293–314.