GREEN FUNCTIONS ASSOCIATED TO COMPLEX REFLECTION GROUPS

TOSHIAKI SHOJI

Department of Mathematics Science University of Tokyo Noda, Chiba 278-8510, Japan

Abstract. Green functions of classical groups are determined by the data from Weyl groups and by certain combinatorial objects called symbols. Generalizing this, we define Green functions associated to complex reflection groups G(e, 1, n), and study their combinatorial properties. We construct Hall-Littlewood functions and Schur functions in our scheme, and show that such Green functions are obtained as a transition matrix between those two symmetric functions, as in the case of GL_{n}.

0. Introduction

Green polynomials Q_{λ}^{μ}(q) of GL_{n}(F_{q}), where λ, μ are partitions of n, were first introduced by J.A. Green [G] in 1955 in a combinatorial framework of symmetric functions. They are obtained as the transition matrix between power sum symmetric functions and Hall-Littlewood functions. Those Green functions play a crucial role in [G] in describing irreducible characters of GL_{n}(F_{q}). More generally, we consider a connected reductive group G defined over a finite field F_{q}, with Frobenius map F. In 1976, P. Deligne and G. Lusztig constructed in [DL] Green functions associated to finite reductive groups G(F_{q}) in general. They defined a function Q_{T}^{G}, for each F -stable maximal torus T of G, as the restriction on unipotent elements of Deligne-Lusztig’s virtual characters R_{T}^{G}(θ). The Green function Q_{χ}, for an irreducible character χ of the Weyl group W of G, is defined as a certain linear combination of Q_{T}^{G} for various T. In the case of GL_{n}(F_{q}), Q_{χ^{λ}}(u_{μ}) coincides with K̃_{λ,μ}(q), the modified Kostka polynomial which is defined as a linear combination of various Q_{μ}^{ν}(q), where χ^{λ} is the irreducible character of W = 𝔖_{n} corresponding to λ, and u_{μ} is the unipotent class corresponding to μ. The computation of R_{T}^{G}(θ) is reduced to the computation of Green functions associated to various reductive subgroups. In [L5], (see also [S2]) Lusztig showed that Green functions have a geometric interpretation in terms of intersection cohomology complexes of the closure of unipotent classes. Using this, he proved in [L3], generalizing the result in [S1], that there exists a simple algorithm of computing Green functions. Here Green functions are determined as a unique solution of a certain type of matrix equation

(*)  P Λ^{t}P = Ω

with unknown P, Λ. (The entries of the matrix P describe Green functions Q_{χ}(u)). Note that the matrix Ω is completely determined by the property of irreducible characters of W, while the shape of P and Λ are determined by the property of the Springer correspondence between unipotent classes of G and irreducible characters of W. It is known, by the geometric interpretation of Green functions, that the entries of the matrices P and Λ are in ℤ[q].

In [GM], M. Geck and G. Malle considered an analogy of the equation (*). In their case, the matrix Ω is the same as above, but the shape of P and Λ are different, which is determined by the property of unipotent characters of G(F_{q}). They computed the solution of this equation in the case of exceptional groups, and checked that the entries of P and Λ are again in ℤ[q], and that those solutions provide the polynomials associated to special pieces discussed in [L6]. They conjecture that this will hold in general, but it is still open for the case of classical groups (see also [S3]).

In the case of classical groups, unipotent characters are parametrized by “symbols” introduced by Lusztig [L1]. While the Springer correspondence is described by “usymbols” which are also introduced by Lusztig [L2] by modifying the notion of symbols. The remarkable thing is that the matrix equation (*) for either case is completely described by the combinatorial property of symbols or u-symbols. The notion of symbols is generalized by Malle [Ma1] so that it fits to the case of complex reflection groups G(e, 1, n) as a generalization of Weyl group of type C_{n}, where he described the unipotent degrees of G(e, 1, n) in terms of such (generalized) symbols. Now the ingredients used to construct the equation (*) makes sense even for this case, and one can define a matrix equation by using the combinatorics of symbols. So it would be natural to call the solution of this equation as the Green function associated to G(e, 1, n).

In this paper, we develop a combinatorial theory of symmetric functions which fit to the situation for G(e, 1, n). In this case, the role of partitions are replaced by symbols. In particular, we construct Hall-Littlewood functions associated to G(e, 1, n), which are parametrized by symbols. But note that for a fixed G(e, 1, n), there exist several different type of symbols, such as symbols or u-symbols in the case of Weyl group of type C_{n}. Hence one obtains several class of Hall-Littlewood functions associated to each type of symbols. We show that a large part of arguments used to construct Green polynomials of GL_{n}(F_{q}) can be generalized to our setting, and that the Green functions are obtained as the transition matrix between Schur functions and Hall-Littlewood functions. Thus obtained Green functions turn out to be rational functions in ℚ(t), where t is an indeterminate.

As some examples show, it is very likely that these Green functions are actually polynomials in t with positive integral coefficients. Concerning this conjecture, we can only show in section 6, that Green functions are actually polynomials for some special cases, under the condition that e = 2.

The author is grateful for G. Malle and F. Lübeck for some useful discussions, and for Malle for the computation of examples by using a computer.

Table of contents

0. Introduction

1. Preliminaries

2. Symmetric functions associated to partitions

3. symmetric functions associated to symbols

4. Hall-Littlewood functions

5. Green functions

6. Some special cases

7. Examples

1. Preliminaries

1.1. Let W ≃ 𝔖_{n} ⋉ (ℤ/eℤ)^{n} be the imprimitive complex reflection group G(e, 1, n), and V = ℂ^{n} the natural reflection representation of W. Let S(V) be the symmetric algebra of V, and I_{+} the ideal of S(V) generated by the homogeneous W -invariant vectors of strictly positive degrees. We denote by R = ⊕R_{i} the coinvariant algebra of W, which is a graded algebra defined as the quotient of S(V) by I_{+}. Let t be an indeterminate. The Poincaré polynomial P_{W} (t) is defined as P_{W} (t) = ∑_{i≥0}(dim_{ℂ} R_{i})t^{i}, which is explicitly given as

P_{W} (t) = ∏_{i=1}^{n} \frac{t^{ei} − 1}{t − 1}.

For any class function f on W, we define R(f) by

(1.1.1)  R(f) = (t − 1)^{n}P_{W} (t)\frac{1}{|W |} ∑_{w∈W} \frac{det_{V} (w)f(w)}{det_{V} (t · id_{V} −w)},

where det_{V} denotes the determinant on V. Note that R(f) coincides with ∑_{i}⟨f, R_{i}⟩_{W} t^{i}, where⟨, ⟩_{W} denotes the inner product of class functions on W. Hence R(f) ∈ ℤ[t] for a generalized character f of W.

Let N^{∗} be the number of complex reflections in W. Then N^{∗} is the maximal degree in R, and the W -module R_{N^{∗}} coincides with \overline{det}_{V}, the complex conjugate of det_{V}.

1.2. An e-tuple of partitions α = (α^{(0)}, …, α^{(e−1)}) is called an e-partition. We define the size |α| of α by |α| = ∑_{k=0}^{e−1} |α^{(k)}|, where |α^{(k)}| is the size of a partition α^{(k)}. Let P_{n,e} be the set of e-partitions of size n. The set W^{∧} of irreducible characters of W is in natural bijection with the set P_{n,e}. We denote by χ^{α} the irreducible character corresponding to α ∈ P_{n,e}. In particular, the unit character corresponds to ((n), −, …, −) and \overline{det}_{V} corresponds to α = (−, …, −, (1^{n})).

Let m_{0}, m_{1}, …, m_{e−1} be positive integers such that m_{k} ≥ n, and put m = (m_{0}, …, m_{e−1}). We denote by Z_{n}^{0,0} = Z_{n}^{0,0}(m) the set of e-partitions α such that |α| = n and that each α^{(k)} is regarded as an element in ℤ^{m_{k}}, written in the form α^{(k)} : α_{1}^{(k)} ≥ ⋯ ≥ α_{m_{k}}^{(k)} ≥ 0. We fix integers r ≥ s ≥ 0, and consider an e-partition Λ^{0} = Λ^{0}(m) = (Λ_{0}, …, Λ_{e−1}) defined by

Λ_{0} : (m_{0} − 1)r ≥ ⋯ ≥ 2r ≥ r ≥ 0,  (1.2.1)
Λ_{i} : s + (m_{i} − 1)r ≥ ⋯ ≥ s + 2r ≥ s + r ≥ s  for 1 ≤ i ≤ e − 1.

We denote by Z_{n}^{r,s} = Z_{n}^{r,s}(m) the set of e-partitions of the form Λ = α + Λ^{0}, where α ∈ Z_{n}^{0,0} and the sum is taken entry-wise. We denote by Λ = Λ(α) if Λ = α+Λ^{0}, and call it the e-symbol of type (r, s) corresponding to α. We write |Λ| = n if Λ ∈ Z_{n}^{r,s}.

Put m′ = (m_{0} + 1, …, m_{e−1} + 1), and we define a shift operation Z_{n}^{r,s}(m) → Z_{n}^{r,s}(m′) by associating Λ′ = (Λ′_{0}, …, Λ′_{e−1}) ∈ Z_{n}^{r,s}(m) to Λ = (Λ_{0}, …, Λ_{e−1}) ∈ Z_{n}^{r,s}(m), where Λ′_{0} = (Λ_{0} + r) ∪ {0} and Λ′_{k} = (Λ_{k} + r) ∪ {s} for k = 1, …, e − 1. In other words, for Λ = Λ(α), Λ′ is obtained as Λ′ = α + Λ^{0}(m′), where α is regarded as an element of Z_{n}^{0,0}(m′) by adding 0 in the entries of α. We denote by Z̄_{n}^{r,s} the classes in ∐_{m′} Z_{n}^{r,s}(m′) under the equivalence relation generated by shift operations. Note that P_{n,e} coincides with the set Z̄_{n}^{0,0}. Also note that Λ^{0} is regarded as a symbol in Z_{n}^{r,s} with n = 0.

Two elements Λ and Λ′ in Z̄_{n}^{r,s} is said to be similar if there exist representatives in Z_{n}^{r,s}(m) such that all the entries of them coincide each other with multiplicities. The set of symbols which are similar to a fixed symbol is called a similarity class in Z_{n}^{r,s}.

We shall define a function a : Z̄_{n}^{r,s} → ℕ. For Λ ∈ Z_{n}^{r,s}, we put

(1.2.2)  a(Λ) = ∑_{λ,λ′∈Λ}min(λ, λ′) − ∑_{μ,μ′∈Λ^{0}} min(μ, μ′),

where in the first sum, we assume that λ ≠ λ′ if λ and λ′ are contained in the same Λ_{i}, and similarly for the second sum (this occurs only when r = 0). The function a on Z_{n}^{r,s} is invariant under the shift operation, and it induces a function a on Z̄_{n}^{r,s}. Clearly, the a-function takes a constant value on each similarity class in Z_{n}^{r,s}.

Remark 1.3. (i) The general notion of e-symbols is obtained by Malle [Ma2]. The e-symbols described above is just a part of such symbols corresponding to the set of e-partitions, which is enough for our later discussions. In [Ma1], the e-symbols are specified to the case where m = (m + 1, m, …, m) for some m > 0. However our arguments in subsequent sections work well for arbitrary m.

(ii) The notion of symbols had appeared in several articles in various forms. If e = 1, Z_{n}^{r,s} is in bijection with the set of partitions of n. In the case where e = 2, the sets Z̄_{n}^{1,0} was used in [L1] to parameterize unipotent characters of Sp_{2n}(F_{q}) or SO_{2n+1}(F_{q}). In that case, the a-function defined in (1.2.2) coincides with the original a-function, i.e., the exact power of t dividing the generic degree D_{ρ}(t) ∈ ℚ[t] of the corresponding unipotent character ρ. In the case e = 2, the set Z̄_{n}^{2,1} (resp. Z̄_{n}^{2,0}) was used in [L2] to describe the generalized Springer correspondence for G = Sp_{2n}(k) (resp. G = SO_{2n+1}(k)) in the case where ch k ≠ 2. In those cases, the similarity classes are in 1 to 1 correspondence with the unipotent classes of G, and the value a(Λ) for the symbol Λ corresponding to the unipotent class u coincides with dim B_{u}, where B_{u} is the variety of Borel subgroups in G containing u (see, for example [L4, 4.4]). While, the set Z̄_{n}^{4,2} was used in [LS] to describe the generalized Springer correspondence for Sp_{2n}(k) with ch k = 2. On the other hand, for arbitrary e, the set Z_{n}^{1,0} was used in [Ma1] to parameterize unipotent degrees associated to the complex reflection groups G(e, 1, n).

1.4. Under the identification of Z_{n}^{r,s} with P_{n,e}, we may consider the a function on the set P_{n,e}. Also the similarity classes in P_{n,e} are defined by inheriting the classes in Z_{n}^{r,s}. We denote by α ∼ β if α, β ∈ P_{n,e} are in the same similarity class. We choose a total order ≻ on P_{n,e} such that a(α) ≤ a(β) whenever α ≻ β, and that each similarity class in P_{n,e} forms an interval. In the following, we consider the matrices of degree |P_{n,e}| with respect to this order. For each α, β ∈ P_{n,e}, we define a polynomial ω_{α,β} ∈ ℤ[t] by

(1.4.1)  ω_{α,β} = t^{N^{∗}}R(χ^{α} ⊗ χ^{β} ⊗ \overline{det}_{V}),

and put Ω = (ω_{α,β}), the matrix of degree |P_{n,e}|. Let P = (p_{α,β}) and Λ = (λ_{α,β}) be matrices of degree |P_{n,e}|. We consider the following system of equations with respect to the unknown variables λ_{α,β}, p_{α,β}.

λ_{α,β} = 0 unless α ∼ β,
p_{α,β} = 0 unless either α ≻ β and α ≁ β, or α = β,  (1.4.2)
p_{α,α} = t^{a(α)}
P Λ^{t}P = Ω

In the case where e = 2 and Z_{n}^{r,s} = Z_{n}^{2,1} (resp. Z_{n}^{r,s} = Z_{n}^{2,0}), it is known by [L3] that the solution of (1.4.2) produces, by the substitution t ↦ q, the Green functions of Sp_{2n}(F_{q}) (resp. SO_{2n+1}(F_{q})) with ch F_{q} ≠ 2, which are known to be polynomials. Therefore in these cases, (1.4.2) has a solution with p_{α,β}, λ_{α,β} ∈ ℤ[t]. In the case where e = 2 and Z_{n}^{r,s} = Z_{n}^{1,0} it is shown in [GM] that (1.4.2) has a unique solution with p_{α,β}, λ_{α,β} ∈ ℚ(t), rational functions in t. They expect that p_{α,β} and λ_{α,β} are actually in ℤ[t], which is a part of their conjecture for general Weyl groups. In our setting for complex reflection groups W = G(e, 1, n), we can also conjecture that p_{α,β}, λ_{α,β} are in ℤ[t].

In the case of GL_{n}, the system of equations such as (1.4.2) arises naturally in a combinatorial setting concerning Schur functions and Hall-Littlewood functions. In later discussions, we show that this holds also in the case of complex reflection groups W.

1.5. There exists a system of equations which is closely related to (1.4.2). Let Ω′ = (ω′_{α,β}) be a matrix defined by

(1.5.1)  ω′_{α,β} = t^{N^{∗}}R(χ^{α} ⊗ \overline{χ}^{β} ⊗ \overline{det}_{V}),

where \overline{χ}^{β} is the complex conjugate of χ^{β}. We consider a matrix equation of the form

(1.5.2)  P′Λ′ ^{t}P′′ = Ω′,

where either of P′ = (p′_{α,β}) and P′′ = (p′′_{α,β}) satisfies similar conditions as in the second and third one in (1.4.2), and Λ′ = (λ′_{α,β}) as in the first one. Let σ = (σ_{α,β}) be the permutation matrix realizing the complex conjugation of irreducible characters of W, i.e., σ_{α,β} = 1 if χ^{β} = \overline{χ}^{α} and σ_{α,β} = 0 otherwise. Then we have Ω′ = Ωσ, and the equation P Λ^{t}P = Ω implies that

P · Λσ · σ^{−1}^{t}P σ = Ω′.

Note that if χ^{β} = \overline{χ}^{α} for α = (α^{(0)}, …, α^{(e−1)}), then β = (β^{(0)}, …, β^{(e−1)}) is given by β^{(k)} = α^{(e−k)} for k = 1, …, e − 1 and β^{(0)} = α^{(0)}. It follows that the permutation of symbols in Z_{n}^{r,s} induced from the complex conjugation χ^{α} ↦ \overline{χ}^{α} preserves each similarity class. In particular, the matrices Λσ, σ^{−1}^{t}P σ have the same shape as Λ and tP, respectively. Now it is easily checked that the equation (1.4.2) has a solution if and only if the equation (1.5.2) has a solution, and in that case, we have P′ = P, Λ′ = Λσ, P′′ = σ^{−1}^{t}P σ. In the remainder of the paper, we consider the equation (1.5.2) instead of (1.4.2).

2. Symmetric functions associated to partitions

2.1. In this section we shall define several symmetric functions associated to e-partitions. We fix m = (m_{0}, …, m_{e−1}) as in 1.2, and introduce indeterminates x_{j}^{(k)} (0 ≤ k ≤ e − 1, 1 ≤ j ≤ m_{k}). We denote by x the whole variables (x_{j}^{(k)}), and also denote by x^{(k)} the variables x_{1}^{(k)}, …, x_{m_{k}}^{(k)}. Power sum symmetric functions and Schur functions are defined as in [M, Appendix B]. Let ζ be a primitive e-th root of unity in ℂ. For each integer r ≥ 1 and i such that 0 ≤ i ≤ e − 1, put

p_{r}^{(i)}(x) = ∑_{j=0}^{e−1} ζ^{ij}p_{r}(x^{(j)}),

where p_{r}(x^{(j)}) denotes the r-th power sum symmetric function with respect to the variables x^{(j)}. We put p_{r}^{(i)}(x) = 1 for r = 0. For an e-partition α = (α^{(0)}, …, α^{(e−1)}) with α^{(k)} : α_{1}^{(k)} ≥ ⋯ ≥ α_{m_{k}}^{(k)}, we define a function p_{α}(x) by

(2.1.1)  p_{α}(x) = ∏_{k=0}^{e−1} ∏_{j=1}^{m_{k}} p_{α_{j}^{(k)}}^{(k)}(x).

(But note that our definition of the power symmetric function given in (2.1.1) is not the same as the one in [M], which coincides with the complex conjugate of ours.)

Next, we define the Schur function s_{α}(x) and monomial symmetric functions m_{α}(x) associated to α by

(2.1.2)  s_{α}(x) = ∏_{k=0}^{e−1} s_{α^{(k)}}(x^{(k)}),  m_{α}(x) = ∏_{k=0}^{e−1} m_{α^{(k)}}(x^{(k)}),

where s_{α^{(k)}}(x^{(k)}) (resp. m_{α^{(k)}}(x^{(k)} ) denotes the usual Schur function (resp. monomial symmetric function) associated to the partition α^{(k)} with respect to the variables x^{(k)}.

Note that the set of conjugacy classes of W is parametrized by P_{n,e} ([M, Appendix B], see also [S2]). We denote by w_{β} a representative of the class in W corresponding to β ∈ P_{n,e}. It is known by [M] that the following Frobenius type formula holds for the irreducible characters of W.

(2.1.3)  p_{β} = ∑_{α∈P_{n,e}} χ^{α}(w_{β})s_{α},

2.2. In what follows we regard the variables x_{i}^{(k)} defined for k ∈ ℤ/eℤ ≃ {0, 1, …, e − 1}. For each 0 ≤ k ≤ e − 1 and an integer r ≥ 0, we define a function q_{r,±}^{(k)}(x; t) by

(2.2.1)  q_{r,±}^{(k)}(x; t) = ∑_{i≥1}(x_{i}^{(k)})^{r+δ} \frac{∏_{j} x_{i}^{(k)} − tx_{j}^{(k±1)}}{∏_{j≠i} x_{i}^{(k)} − x_{j}^{(k)}}  (r ≥ 1),

where δ = m_{k} − 1 − m_{k±1}, and by q_{r,±}^{(k)}(x; t) = 1 for r = 0. In the product of the denominator, x_{j}^{(k)} runs over all the variables in x^{(k)} except x_{i}^{(k)}, while in the numerator, x_{j}^{(k±1)} runs over all the variables in x^{(k±1)}. Note that if x_{i}^{(k)} = x_{i}^{(k±1)} = x_{i} and m_{k} = m_{k±1}, q_{r,±}^{(k)}(x; t) coincide with the function q_{r}(x; t) given in [M, III, 2.9]. The following lemma describes the generating function for q_{r,±}^{(k)}(x; t).

Lemma 2.3. (i) q_{r,±}^{(k)}(x; t) is a polynomial in ℤ[x; t], which is homogeneous of de-

gree r with respect to the variables x^{(k)}, x^{(k±1)}. (ii) q_{r,±}^{(k)}(x; t) has the stability property for both variables x^{(k)}, x^{(k±1)}, i.e., the substi-

tution of x_{m_{k}}^{(k)} = 0 produces the function q_{r,±}^{(k)}(x, t) with respect to the variables x_{1}^{(k)}, …, x_{m_{k}−1}^{(k)}, and similarly for x^{(k±1)}.

(iii) Let u be an indeterminate. Then we have

(2.3.1)  ∑_{r=0}^{∞} q_{r,±}^{(k)}(x; t)u^{r} = \frac{∏_{i} 1 − tux_{i}^{(k±1)}}{∏_{j} 1 − ux_{j}^{(k)}} .

Proof. First we consider the case where δ = −1, and put m = m_{k} = m_{k±1}. By Lagrange’s interpolation, we have

∏_{i=1}^{m} \frac{z − b_{i}}{z − a_{i}} = 1 + ∑_{i=1}^{m} \frac{a_{i} − b_{i}}{z − a_{i}} ∏_{j≠i} \frac{a_{i} − b_{j}}{a_{i} − a_{j}} .

Substituting a_{i} = zux_{i}^{(k)}, b_{i} = ztux_{i}^{(k±1)} into this formula, we have

∏_{i=1}^{m} \frac{1 − tux_{i}^{(k±1)}}{1 − ux_{i}^{(k)}} = 1 + ∑_{i=1}^{m} \frac{ux_{i}^{(k)} − tux_{i}^{(k±1)}}{1 − ux_{i}^{(k)}} ∏_{j≠i} \frac{x_{i}^{(k)} − tx_{j}^{(k±1)}}{x_{i}^{(k)} − x_{j}^{(k)}} .

The coefficient of u^{r} in the expansion of the right hand side coincides with q_{r,±}^{(k)}(x; t). This shows (2.3.1) in the case where δ = −1. In particular, in this case q_{r,±}^{(k)}(x; t) is a polynomial in x^{(k)}, x^{(k±1)} and t, with integral coefficients, which is homogeneous of degree r with respect to the variables x. Now one can check by using the definition (2.2.1), that if q_{r,±}^{(k)}(x; t) is a polynomial in x^{(k)}, x^{(k±1)}, then it has a stability property for both variables x^{(k)}, x^{(k±1)}. It follows from this that q_{r,±}^{(k)}(x; t) satisfies the assertions (i), (ii) in the lemma even in the general case δ ≠ −1. Since the right hand side of (2.3.1) also has the stability property, the formula (2.3.1) for the general case follows from the case δ = −1. The lemma is proved. □

2.4. For an e-partition α = (α^{(0)}, …, α^{(e−1)}) ∈ P_{n,e}, we define a function q_{α,±}(x) by

(2.4.1)  q_{α,±}(x; t) = ∏_{k=0}^{e−1} ∏_{j=1}^{m_{k}} q_{α_{j}^{(k)},±}^{(k)}(x; t).

For a partition α^{(k)} : α_{1}^{(k)} ≥ ⋯ ≥ α_{m_{k}}^{(k)} ≥ 0, we define a function z_{α^{(k)}}(t) by

z_{α^{(k)}}(t) = ∏_{j=1}^{m_{k}}(1 − ζ^{k}t^{α_{j}^{(k)}})^{−1},

which is determined only by parts such that α_{j}^{(k)} ≠ 0. Then we define z_{α}(t) by

(2.4.2)  z_{α}(t) = z_{α} ∏_{k=0}^{e−1} z_{α^{(k)}}(t),

where z_{α} is the order of the centralizer of w_{α} in W. Explicitly, z_{α} is given as follows. For α ∈ P_{n,e}, put l(α) = ∑_{k=0}^{e−1} l(α^{(k)}), where l(α^{(k)}) is the number of parts in the partition α^{(k)}. For a partition α = (1^{n_{1}}, 2^{n_{2}}, …), put z_{α} = ∏_{i≥1} i^{n_{i}}n_{i}!. Then z_{α} = e^{l(α)} ∏_{k=0}^{e−1} z_{α^{(k)}}.

We now introduce infinitely many variables x_{i}^{(k)}, y_{i}^{(k)} for i = 1, 2, … and for 0 ≤ k ≤ e − 1. Since p_{α}(x), q_{α,±}(x; t), m_{α}(x) have the stability property for the variables x_{i}^{(k)}, we may regard them as functions with infinitely many variables x_{1}^{(k)}, x_{2}^{(k)}, . . .. Under this setting, we have the following proposition, which is a variant of Lemma 7.8 in [S4].

Proposition 2.5. Let

Ω(x, y; t) = ∏_{k=0}^{e−1} ∏_{i,j} \frac{1 − tx_{i}^{(k+1)}y_{j}^{(k)}}{1 − x_{i}^{(k)}y_{j}^{(k)}} .

Then we have

Ω(x, y; t) = ∑_{α}q_{α,+}(x; t)m_{α}(y) = ∑_{α}m_{α}(x)q_{α,−}(y; t),  (2.5.1)
Ω(x, y; t) = ∑_{α}z_{α}(t)^{−1}p_{α}(x)p̄_{α}(y),  (2.5.2)

where α runs over all the e-partitions of any size. In (2.5.2), p̄_{α}(y) denotes the complex conjugate of p_{α}(y).

Proof. We show the first equality of (2.5.1). It follows from Lemma 2.3 that we have

Ω(x, y; t) = ∏_{k=0}^{e−1} ∏_{j} ∑_{r_{j}^{(k)}=0}^{∞} q_{r_{j}^{(k)},+}^{(k)}(x; t)(y_{j}^{(k)})^{r_{j}^{(k)}}.

By multiplying out, the right hand side turns out to be ∑_{α}q_{α,+}(x; t)m_{α}(y), which shows the first equality. If we note the equality

(2.5.3)  Ω(x, y; t) = ∏_{k=0}^{e−1} ∏_{i,j} \frac{1 − tx_{i}^{(k)}y_{j}^{(k−1)}}{1 − x_{i}^{(k)}y_{j}^{(k)}} ,

the second equality in (2.5.1) follows from the first one, by replacing the role of x_{i}^{(k)} and y_{j}^{(k)}.

Next we show the equality (2.5.2). The proof is similar to the case where e = 1, which is given in [M, III, 4.1]. See also [S4, Lemma 7.8]. We start from the formula (2.5.3). By a similar argument as in [M], we have

log Ω(x, y; t) = ∑_{k} ∑_{i,j} ∑_{m=1}^{∞}\{\frac{1}{m}(x_{i}^{(k)}y_{j}^{(k)})^{m} − \frac{t^{m}}{m} (x_{i}^{(k)}y_{j}^{(k−1)})^{m}\}.

But since

\frac{1}{e} ∑_{a=0}^{e−1}(ζ^{−k}ζ^{k′})^{a} = δ_{k,k′},

we have

log Ω(x, y; t)

= ∑_{a=0}^{e−1} ∑_{m=1}^{∞} ∑_{k,k′,i,j}\{\frac{1}{em}ζ^{−ak}ζ^{ak′}(x_{i}^{(k)}y_{j}^{(k′)})^{m} − \frac{t^{m}}{em}ζ^{a}ζ^{−ak}ζ^{a(k′−1)}(x_{i}^{(k)}y_{j}^{(k′−1)})^{m}\}

= ∑_{a=0}^{e−1} ∑_{m=1}^{∞} \frac{1 − ζ^{a}t^{m}}{em} p_{m}^{(a)}(x)\overline{p_{m}^{(a)}(y)}.

Hence

Ω(x, y; t) = ∏_{a=0}^{e−1} ∏_{m=1}^{∞} exp\{\frac{1 − ζ^{a}t^{m}}{em} p_{m}^{(a)}(x)\overline{p_{m}^{(a)}(y)}\}

= ∑_{α}z_{α}(t)^{−1}p_{α}(x)p̄_{α}(y),

and we get (2.5.2). Thus, the proposition is proved. □

3. Symmetric functions associated to symbols

3.1. In this section, we assume that e > 1 and r > 0. In order to proceed the arguments smoothly, we need to consider a more general situation than section 2. Let Ẑ^{0,0} = Ẑ_{n}^{0,0}(m) be the set of e-compositions α = (α^{(0)}, …, α^{(e−1)}), where α^{(k)} = (α_{1}^{(k)}, …, α_{m_{i}}^{(k)}) ∈ ℤ_{≥0}^{m_{k}} such that ∑_{k,j} α_{j}^{(k)} = n. We denote by Ẑ_{n}^{r,s} = Ẑ_{n}^{r,s}(m) the set of “symbols” Λ = α + Λ^{0}(m), where α ∈ Ẑ_{n}^{0,0} and Λ^{0}(m) ∈ Z_{n}^{r,s}(m) is as in 1.2. Hence Ẑ_{n}^{0,0} ⊃ Z_{n}^{0,0} and Ẑ_{n}^{r,s} ⊃ Z_{n}^{r,s}. We denote by Λ(α) the element in Ẑ_{n}^{r,s} corresponding to α ∈ Ẑ_{n}^{0,0} under the above bijection.

In this section, we shall define a symmetric function R_{Λ}^{±}(x; t) associated to Λ = Λ(α) ∈ Ẑ_{n}^{r,s}. For this, first we prepare some notation. Take α ∈ Ẑ_{n}^{0,0} and write it as α = (α_{j}^{(k)}). We fix a total order < on the set M = {(i, k) | 1 ≤ i ≤ m_{k}, 0 ≤ k ≤ e−1}. Let ν_{0} = (i, k) ∈ M be the element such that α_{i}^{(k)} ≠ 0 and that α_{j}^{(h)} = 0 for any (j, h) such that (j, h) > ν_{0}.

We define a function I_{i,±}^{(k)}(x; t) for 0 ≤ k ≤ e − 1, 1 ≤ i ≤ m_{k} by

(3.1.1)  I_{i,±}^{(k)}(x; t) = \begin{cases}∏_{\begin{matrix}1≤j≤m_{k±1} \\ (i,k)<(j,k±1) \end{matrix}}(x_{i}^{(k)} − tx_{j}^{(k±1)}) & if (i, k) ≤ ν_{0}, \\ ∏_{\begin{matrix}1≤j≤m_{k} \\ (i,k)<(j,k) \end{matrix}}(x_{i}^{(k)} − tx_{j}^{(k)}) & if (i, k) > ν_{0}. \end{cases}

Put

δ_{i,±}^{(k)} = ♯{j | (i, k) < (j, k)} − ♯{j | (i, k) < (j, k ± 1)}

if (i, k) ≤ ν_{0}, and put δ_{i,±}^{(k)} = 0 if (i, k) > ν_{0}. Now set δ_{±}^{(k)} = (δ_{1,±}^{(k)}, …, δ_{m_{k},±}^{(k)}). For each integer r ≥ 1, let

v_{r}(t) = ∏_{i=1}^{r} \frac{1 − t^{i}}{1 − t} .

We define a polynomial v_{α}(t) by

v_{α}(t) = ∏_{k=0}^{e−1} v_{μ_{k}}(t),

where μ_{k} = ♯{j | (j, k) > ν_{0}}. For a sequence β = (β_{1}, …, β_{m_{k}}), we write as

(x^{(k)})^{β} = (x_{1}^{(k)})^{β_{1}}(x_{2}^{(k)})^{β_{2}} ⋯ (x_{m_{k}}^{(k)})^{β_{m_{k}}} .

Finally, put 𝔖_{m} = 𝔖_{m_{0}} × ⋯ × 𝔖_{m_{e−1}}. We now define a function R_{α,<}^{±}(x; t) associated to α and the order < on M by (3.1.2)

R_{α,<}^{±}(x; t) = v_{α}(t)^{−1} ∑_{w∈𝔖_{m}} w{∏_{k}(x^{(k)})^{α^{(k)}+δ_{±}^{(k)}} ∏_{k,i} I_{i,±}^{(k)}(x; t)/ ∏_{k} ∏_{(i,k)<(j,k)}(x_{i}^{(k)} − x_{j}^{(k)})}.

3.2. We regard 𝔖_{μ_{k}} as a subgroup of 𝔖_{m_{k}} as a permutation group with respect to the letters {1 ≤ j ≤ m_{k} | (j, k) > ν_{0}}. Thus 𝔖_{α} = 𝔖_{μ_{0}} × ⋯ × 𝔖_{μ_{e−1}} is regarded as a subgroup of 𝔖_{m}. We note that (3.1.2) can also be written as (3.2.1)

R_{α,<}^{±}(x; t) = ∑_{w∈𝔖_{m}/𝔖_{α}} w\{∏_{k}(x^{(k)})^{α^{(k)}+δ_{±}^{(k)}} ∏_{\begin{matrix}k,i \\ (i,k)≤ν_{0} \end{matrix}} I_{i,±}^{(k)}(x; t)/∏_{k} ∏_{\begin{matrix}(i,k)<(j,k) \\ (i,k)≤ν_{0} \end{matrix}}(x_{i}^{(k)} − x_{j}^{(k)})\}.

We show (3.2.1). It is known ([M, III]) that

(3.2.2)  ∑_{w∈𝔖_{n}} w(∏_{i<j} \frac{x_{i} − tx_{j}}{x_{i} − x_{j}} ) = v_{n}(t),

for variables x_{1}, …, x_{n}. Since the factors I_{i,±}^{(k)}(x; t), ∏_{(i,k)<(j,k)}(x_{i}^{(k)}−x_{j}^{(k)}) for (i, k) ≤ ν_{0}

and (x^{(k)})^{α^{(k)}+δ_{±}^{(k)}} are invariant under the action of 𝔖_{α}, we have

∑_{w∈𝔖_{α}}w{∏_{k}(x^{(k)})^{α^{(k)}+δ_{±}^{(k)}} ∏_{k,i} I_{i,±}^{(k)}(x; t)/ ∏_{k} ∏_{(i,k)<(j,k)}(x_{i}^{(k)} − x_{j}^{(k)})}

= ∏_{k}(x^{(k)})^{α^{(k)}+δ_{±}^{(k)}} ∏_{\begin{matrix}k,i \\ (i,k)≤ν_{0} \end{matrix}} I_{i,±}^{(k)}(x; t)/ ∏_{k} ∏_{\begin{matrix}(i,k)<(j,k) \\ (i,k)≤ν_{0} \end{matrix}}(x_{i}^{(k)} − x_{j}^{(k)})

× ∑_{w∈𝔖_{α}} w\{∏_{k} ∏_{\begin{matrix}(i,k)<(j,k) \\ (i,k)>ν_{0} \end{matrix}}(x_{i}^{(k)} − tx_{j}^{(k)})/(x_{i}^{(k)} − x_{j}^{(k)})\}.

The last sum is equal to v_{α}(t) by (3.2.2). Hence (3.2.1) holds.

We consider the special case where ν_{0} = (i, k) is the smallest element in M. Then we have α_{i}^{(k)} = r for some r > 0, and α_{j}^{(h)} = 0 for any (j, h) ≠ (i, k). Under this setting, we have

(3.2.3)  R_{α,<}^{±}(x; t) = q_{r,±}^{(k)}(x; t).

In fact, by the formula (3.2.1), we have

R_{α,<}^{±}(x; t) = ∑_{w∈𝔖_{m_{k}} /𝔖_{m_{k}−1}} w{(x_{i}^{(k)})^{r+δ_{i,±}^{(k)}} ∏_{j≥1}(x_{i}^{(k)} − tx_{j}^{(k±1)})/∏_{j≠i}(x_{i}^{(k)} − x_{j}^{(k)})},

where, 𝔖_{m_{k}} is the symmetric group with respect to the letters {1, 2, …, m_{k}} and 𝔖_{m_{k}−1} is the subgroup of 𝔖_{m_{k}} fixing the letter i, and δ_{i,±}^{(k)} = (m_{k} − 1) − m_{k±1}. Thus (3.2.3) holds.

Let ν_{1} = (q, p) be the smallest element in M. Put m′ = (m_{0}, …, m_{p}−1, …, m_{e−1}), and let α′ ∈ Z_{n′}^{0,0}(m′) be the element obtained from α by removing α_{q}^{(p)}, where n′ = n − α_{q}^{(p)}. The set M′ = M \{ν_{1}} has a total order < inherited from M. We consider the functions R_{α′,<}^{±}(x′; t) with variables x′ = {x_{j}^{(k)} | (j, k) ≠ (q, p)}. We have the following lemma.

Lemma 3.3. Let R_{α′,<}^{±[i]}(x; t) be the function obtained from R_{α′,<}^{±}(x′, t) by replacing the variables x_{1}^{(p)}, …, x̂_{q}^{(p)}, …, x_{m_{p}}^{(p)} (x̂_{q}^{(p)} is removed) by x_{1}^{(p)}, …, x̂_{i}^{(p)}, …, x_{m_{p}}^{(p)}. Then we have

R_{α,<}^{±} = ∑_{i=1}^{m_{p}}(x_{i}^{(p)})^{α_{q}^{(p)}+δ_{q}^{(p)}}g_{i}R_{α′,<}^{±[i]},

where g_{i} (1 ≤ i ≤ m_{p}) is given by

g_{i}(x; t) = ∏_{j}(x_{i}^{(p)} − tx_{j}^{(p±1)})/∏_{j≠i}(x_{i}^{(p)} − x_{j}^{(p)}).

Proof. Since (q, p) ≤ ν_{0}, we have 𝔖_{α} = 𝔖_{α}′ ⊂ 𝔖_{m′} ⊂ 𝔖_{m} and 𝔖_{m}/𝔖_{m′} ≃ 𝔖_{m_{p}}/𝔖_{m_{p}−1}. (𝔖_{m_{p}−1} is identified with the subgroup of 𝔖_{m_{p}} fixing the letter q). Note that since ν_{1} is minimum, no factor of the form x_{j}^{(p∓1)} − tx_{q}^{(p)} appears in the expression of R_{α,<}^{±}. It follows, by (3.2.1), that we have

R_{α,<}^{±}(x; t) = ∑_{w∈𝔖_{m}/𝔖_{m′}} w{(x_{q}^{(p)})^{α_{q}^{(p)}+δ_{q}^{(p)}}g_{q}R_{α′,<}^{±}(x′; t)},

which implies the equality in the lemma. □

3.4. Let

Ψ_{±}^{(k)}(u) = ∑_{r=0}^{∞} q_{r,±}^{(k)}(x; t)u^{r},
F (u_{1}, u_{2}) = (1 − u_{1})/(1 − tu_{2}).

We regard F (u_{1}, u_{2}) as a formal power series (1 − u_{1})(1 + tu_{2} + (tu_{2})^{2} + ⋯). We introduce infinitely many variables u_{1}^{(k)}, u_{2}^{(k)}, … for 0 ≤ k < e. We fix a total order < on the set M^{∗} = {(i, k) | 0 ≤ k < e, i = 1, 2, … }, and define a function Φ_{±} with multi-variables u = {u_{i}^{(k)}} by

Φ_{±}(u) = ∏_{k=0}^{e−1} ∏_{i≥1} Ψ_{±}^{(k)}(u_{i}^{(k)}) ∏_{\begin{matrix}(i,k)<(j,k) \\ (i,k)<(l,k∓1) \end{matrix}} F ((u_{i}^{(k)})^{−1}u_{j}^{(k)}, (u_{i}^{(k)})^{−1}u_{l}^{(k∓1)}).

Take α ∈ Z_{n}^{0,0}, and let M be as in 3.1. Then M can be embedded in M^{∗}. We say that M is compatible with M^{∗} if the embedding preserves the order, and if the inequality ν < ν′ holds for ν ∈ M, ν′ ∈ M^{∗} − M. We show the following proposition.

Proposition 3.5. Assume that M is compatible with M^{∗}. Then R_{α,<}^{±} coincides with the coefficient of u^{α} = ∏_{k,i}(u_{i}^{(k)})^{α_{i}^{(k)}} in the function Φ_{±}(u).

Proof. We prove the proposition by induction on l = ♯{ν ∈ M | ν ≤ ν_{0}}. First assume that l = 1. Then ν_{0} = (i, k) is the smallest element in M, and so in M^{∗}. If α_{i}^{(k)} = r, we have R_{α,<}^{±} = q_{r,±}^{(k)} by (3.2.3), and the assertion follows. Assume that l > 1. Let ν_{1} = (q, p) be the minimum element in M, and let α′ be as in 3.2. We may assume, by induction, that the assertion holds for R_{α′,<}^{±}(x′; t). We denote by Φ_{±}^{[i]} the function obtained from Φ_{±} by setting x_{i}^{(p)} = 0. Then by (2.3.1) we have

(3.5.1)  Φ_{±}^{[i]}(u) = Φ_{±}(u) ∏_{j,l≥1} F (x_{i}^{(p)}u_{j}^{(p)}, x_{i}^{(p)}u_{l}^{(p∓1)}).

By applying the induction hypothesis to R_{α′,<}^{[i]±}, together with Lemma 3.3, we see that R_{α,<}^{±}(x; t) is the coefficient of u^{α} in

∑_{a≥0}(u_{q}^{(p)})^{a} ∑_{i=1}^{m_{p}}(x_{i}^{(p)})^{a+δ_{1}^{(p)}}g_{i}Φ_{±}^{[i]}(u′),

where u′ = u − {u_{q}^{(p)}}. Now by (3.5.1), the last expression is equal to

(3.5.2)  Ψ_{±}(u′) ∑_{a≥0}(u_{q}^{(p)})^{a} ∑_{i=1}^{m_{p}}(x_{i}^{(p)})^{a+δ_{q}^{(p)}}g_{i} ∏_{j≠q} F (x_{i}^{(p)}u_{j}^{(p)}, x_{i}^{(p)}u_{l}^{(p∓1)}).

We expand the product in (3.5.2) as a power series in x_{i}^{(p)};

∏ F (x_{i}^{(p)}u_{j}^{(p)}, x_{i}^{(p)}u_{l}^{(p∓1)}) = ∑_{m≥0} f_{m}(u′; t)(x_{i}^{(p)})^{m},

where f_{m} = f_{m}(u′; t) is a polynomial in u′, t. Write the expression (3.5.2) as Φ_{±}(u′)X. Then substituting the above expansion into X, we have

X = ∑_{a≥0}(u_{q}^{(p)})^{a} ∑_{m≥0} f_{m} ∑_{i=1}^{m_{p}}(x_{i}^{(p)})^{a+δ_{q}^{(p)}+m}g_{i}

= ∑_{a,m≥0}(u_{q}^{(p)})^{a}q_{a+m,±}^{(p)}f_{m}

= ∑_{b=0}^{∞}(u_{q}^{(p)})^{b}q_{b,±}^{(p)} ∑_{m=0}^{b} f_{m}(u_{q}^{(p)})^{−m}.

Hence the positive degree part in X with respect to u_{q}^{(p)} coincides with

Ψ_{±}^{(p)}(u_{q}^{(p)}) ∑_{m≥0} f_{m}(u_{q}^{(p)})^{−m}

= Ψ_{±}^{(p)}(u_{q}^{(p)}) ∏_{\begin{matrix}j≠q \\ (q,p)<(l,p∓1) \end{matrix}} F ((u_{q}^{(p)})^{−1}u_{j}^{(p)}, (u_{q}^{(p)})^{−1}u_{l}^{(p∓1)}).

Thus R_{α,<}^{±} is the coefficient of u^{α} in

Ψ_{±}^{(p)}(u_{q}^{(p)}) ∏_{\begin{matrix}j≠q \\ (q,p)<(l,p∓1) \end{matrix}} F ((u_{q}^{(p)})^{−1}u_{j}^{(p)}, (u_{q}^{(p)})^{−1}u_{l}^{(p∓1)}) · Φ_{±}(u′) = Φ_{±}(u).

The proposition is proved. □

3.6. Let M = ∑_{i} m_{i}. We define an operator R_{ij} on the set ℤ^{M} by R_{ij}(λ) = λ′, where if λ = (λ_{1}, …, λ_{M}) ∈ ℤ^{M}, then λ′ ∈ ℤ^{M} is given by

λ′_{i} = λ_{i} + 1, λ′_{j} = λ_{j} − 1

and λ′_{l} = λ_{l} for l ≠ i, j. A raising operator (resp. lowering operator) R on ℤ^{M} is defined as a product of various R_{ij} with i < j (resp. i > j). We identify M with the set {1, 2, …, M} via the total order on M, and regard α ∈ Ẑ_{n}^{0,0} as an element in ℤ^{M}. Under this identification, we express the operator R_{ij} as R_{ν,ν′} for ν, ν′ ∈ M.

The definition of q_{α,±} can be applied to α ∈ Ẑ_{n}^{0,0} as well. We extend the definition of q_{r,±}^{(k)} to the case r < 0 by setting q_{r,±}^{(k)} = 0. Accordingly, we define q_{α,±} by setting q_{α,±} = 0 if some α_{j}^{(k)} < 0. Then an action of of raising operators on the functions q_{α,±} is defined by R(q_{α,±}) = q_{α′,±} with α′ = Rα.

Let us define a function b : M → ℕ by b(ν) = k if ν = (i, k) ∈ M. As a corollary to Proposition 3.5, we have the following.

Corollary 3.7. Let α ∈ Ẑ_{n}^{0,0}. Then the function R_{α,<}^{±} is expressed in terms of raising operators as

(3.7.1)  R_{α,<}^{±} = ∏_{ν<ν′}{ ∏_{b(ν′)=b(ν)}(1 − R_{νν′})/ ∏_{b(ν′)=b(ν)∓1}(1 − tR_{νν′})}q_{α,±}.

In particular, R_{α,<}^{±} is a polynomial in x, t, and is expressed as

(3.7.2)  R_{α,<}^{±}(x; t) = ∑_{β∈Z_{n}^{0,0}} c_{α,β}(t)q_{β,±}(x; t)

with c_{α,β}(t) ∈ ℤ[t].

Proof. We notice that ∏_{k,i} Ψ_{±}^{(k)}(u_{i}^{(k)}) = ∑_{β} q_{β,±}u^{β}. Then it follows from Proposition 3.5, by using a similar argument as in [M, III, 2], that one can write R_{α,<}^{±} in a form as in (3.7.1). The formula (3.7.2) follows easily from this. □

3.8. We consider Λ = Λ(α) ∈ Ẑ_{n}^{r,s}, and write it as Λ = (λ_{j}^{(k)}). We choose a total order < on M satisfying the property that ν < ν′ if λ_{i}^{(k)} > λ_{j}^{(l)} for ν = (i, k), ν′ = (j, l) ∈ M. We now define a function R_{Λ}^{±}(x; t) by R_{Λ}^{±} = R_{α,<}^{±}. For this R_{Λ}^{±}, we can obtain a more precise result than Corollary 3.7. First we show some properties of a-functions.

Let λ = (λ_{1}, …, λ_{M}), μ = (μ_{1}, …, μ_{M}) ∈ ℤ^{M}. We define a usual partial order λ ≥ μ on the set ℤ^{M} by λ_{1} + ⋯ λ_{i} ≥ μ_{1} + ⋯ μ_{i} for i = 1, …, M. Then as is well-known that

(3.8.1) Let R be a raising operator on ℤ^{M}. Then R(λ) ≥ λ for λ ∈ ℤ^{M}.

For a partition λ = (λ_{1} ≥ ⋯ ≥ λ_{M} ≥ 0), we define n(λ) by

n(λ) = ∑_{i≥1}(i − 1)λ_{i}.

It is known (e.g., [M, III, (6.5)]) that

(3.8.2) Let λ, μ be partitions such that λ ≥ μ. Then we have n(λ) ≤ n(μ), and the equality n(λ) = n(μ) occurs only when λ = μ.

A symbol Λ ∈ Z_{n}^{r,s} can be identified with an element in ℤ^{M} via the order on M. Hence the raising operator acts on symbols Λ also. Note that the a-function on Z_{n}^{r,s} can be extended to the function on Ẑ_{n}^{r,s}, which is also denoted by a. We have the following lemma.

Lemma 3.9. Let Λ ∈ Z_{n}^{r,s} and assume that Λ′ ∈ Ẑ_{n}^{r,s} is obtained from Λ by applying a raising operator. Then we have a(Λ) ≥ a(Λ′), and the equality a(Λ) = a(Λ′) occurs only when Λ = Λ′.

Proof. Let ξ ∈ ℤ^{M} be a partition corresponding to Λ, and ξ′ ∈ ℤ^{M} the element obtained by applying a raising operator on ξ. Then by (3.8.1), we have ξ′ ≥ ξ. If ξ′′ is the partition obtained from ξ′ by arranging the parts of ξ′ in decreasing order, then we have ξ′′ ≥ ξ′. Hence we have n(ξ′′) ≤ n(ξ) by (3.8.2). But for a partition ξ = (ξ_{1} ≥ ξ_{2} ≥ ⋯ ≥ ξ_{M}), n(ξ) is also expressed as

n(ξ) = ∑_{i≠j} min{ξ_{i}, ξ_{j}}.

Hence n(ξ′′) ≤ n(ξ) is equivalent to a(Λ′) ≤ a(Λ). The equality case also follows from (3.8.2). □

3.10. Let α ∈ Ẑ_{n}^{0,0}. We define the a-function on Ẑ_{n}^{0,0} by a(α) = a(Λ) if Λ = Λ(α) ∈ Ẑ_{n}^{r,s}. Let α′ be the element in Z_{n}^{0,0} obtained from α = (α^{(0)}, …, α^{(e−1)}) by arranging the entries in α^{(k)} = (α_{1}^{(k)}, …, α_{m_{k}}^{(k)}) in decreasing order so that it gives a partition. Then clearly we have q_{α,±} = q_{α′,±}. However, a(α) and a(α′) are different in general. Concerning this, we have the following lemma.

Lemma 3.11. Let α ∈ Ẑ_{n}^{0,0}, and α′ ∈ Z_{n}^{0,0} be as above. Then we have a(α′) ≤ a(α). The equality occurs only when α = α′.

Proof. We consider a sequence α^{(k)} = (α_{1}, …, α_{m_{k}}) ∈ ℤ_{≥0}^{m_{k}} and put Λ^{(k)} = (λ_{1}, …, λ_{m_{k}}), where λ_{i} = α_{i} + (m − i)r + s (for example, in the case where k ≥ 1. The case k = 0 is similar). Assume that α_{i} < α_{i+1} for some i. Let α′ = (α′_{1}, …, α′_{m}) be a sequence obtained from α by putting α′_{i} = α_{i+1}, α′_{i+1} = α_{i} and α′_{j} = α_{j} for j ≠ i, i + 1. Put Λ′ = (λ′_{1}, …, λ′_{m}) with λ′_{j} = α′_{j} + (m − j)r + s for each j, and let Λ′ be a symbol obtained from Λ by replacing Λ^{(k)} by Λ′ and by leaving other Λ^{(l)} unchanged. In order to prove the lemma, it is enough to show that a(Λ′) < a(Λ). Now it is easy to see that λ′_{i} = λ_{i+1} + r, λ′_{i+1} = λ_{i} − r. If λ_{i} ≤ λ_{i+1}, then Λ′ coincides with R_{i+1,i}^{r}Λ, and we have a(Λ′) < a(Λ). On the other hand, if λ_{i} > λ_{i+1}, then we have λ′_{i} = λ_{i} + c, λ′_{i+1} = λ_{i+1} − c with c = α_{i+1} − α_{i} > 0. Hence Λ′ coincides with R_{i,i+1}^{c}Λ, and again we have a(Λ′) < a(Λ). So the lemma is proved. □

We have the following.

Proposition 3.12. Let α ∈ Z_{n}^{0,0} and put Λ = Λ(α) ∈ Z_{n}^{r,s}. Let R_{Λ}^{±} be as in 3.8. Then

(i) R_{Λ}^{±} can be expressed as

R_{Λ}^{±}(x; t) = q_{α,±}(x; t) + ∑_{β∈Z_{n}^{0,0}} c_{α,β}(t)q_{β,±}(x; t)

with c_{α,β}(t) ∈ ℤ[t], where c_{α,β} = 0 unless a(β) < a(α). (ii) The polynomial R_{Λ}^{±}(x; t) has the stability property in the following sense; let

m′ = (m′_{0}, …, m′_{e−1}) with m′_{k} = m_{k} + 1. Let Λ′ ∈ Z_{n}^{r,s}(m′) be the symbol obtained from Λ ∈ Z_{n}^{r,s}(m) by the shift operation. Then the polynomial R_{Λ′}^{±}(x; t) with x_{m_{k}+1}^{(k)} = 0 (0 ≤ k ≤ e − 1) coincides with R_{Λ}^{±}(x; t).

Proof. We show (i). By Corollary 3.7, R_{Λ}^{±} is expressed as a linear combination of q_{β,±} with coefficient c_{α,β} ∈ ℤ[t], where β ∈ Ẑ_{n}^{0,0}. Since the formula (3.7.1) involves raising operators only, we see that c_{α,β} = 0 unless Λ(β) is obtained from Λ by applying the raising operators. This implies, in view of Lemma 3.9, that c_{α,β} = 0 unless a(β) ≤ a(α), and that the case a(β) = a(α) occurs only when β = α. Now by virtue of Lemma 3.11, we may replace β ∈ Ẑ_{n}^{0,0} by Z_{n}^{0,0}. Hence the assertion follows.

Next we show (ii). Let M′ be a set associated to Z_{n}^{r,s}(m′) similar to M. Then M ⊂ M′ and we may choose a total order on M′ compatible with the order on M given in 3.8. Then in the expression of R_{Λ′}^{±} given in (3.7.1), the factors corresponding to R_{νν′} with ν′ ≤ ν_{0} have exactly the same form as the factors for R_{Λ}^{±}. The factors corresponding to R_{νν′} with ν′ > ν_{0} are negligible since R_{νν′}α contains a negative part and so R_{νν′}q_{α,±} = 0. Hence (ii) follows from the stability property for q_{β,±} (see Lemma 2.3). The proposition is proved. □

3.13. Let Z̃_{n}^{0,0} be the set of α = (α_{j}^{(k)}) with α_{j}^{(k)} ∈ ℤ. The definition of Schur functions s_{α} given in (2.1.2) can be extended to the case where α ∈ Z̃^{0,0}. In that case, s_{α} is a Laurent polynomial with respect to the variables x_{j}^{(k)}. If α_{j}^{(k)} + (m_{k} − j) are positive and all distinct for j = 1, …, m_{k} (for a fixed k), then s_{α} coincides with the usual Schur function up to sign. If α_{j}^{(k)} + (m_{k} − j) are not all distinct for a fixed k, then s_{α} = 0. The operator R = ∏ R_{νν′} acts on Schur functions by R(s_{α}) = s_{α}′ if R(α) = α′. We now consider α ∈ Z_{n}^{0,0} and let Λ = Λ(α). Then we have the following formula.

(3.13.1)  R_{Λ}^{±} = v_{α}(t)^{−1} ∏_{\begin{matrix}ν<ν′,ν≤ν_{0} \\ b(ν′)=b(ν±1) \end{matrix}}(1 − tR_{ν′ν}) ∏_{\begin{matrix}ν<ν′,ν>ν_{0} \\ b(ν′)=b(ν) \end{matrix}}(1 − tR_{ν′ν})s_{α}.

First we note that

(3.13.2)  R_{Λ}^{±}(x; 0) = s_{α}(x).

In fact, if we substitute t = 0 to R_{Λ}^{±} in the formula (3.1.2), we have

R_{Λ}^{±}(x; 0) = ∑_{w∈𝔖_{m}} w{∏_{k}(x^{(k)})^{α^{(k)}+ε^{(k)}}/∏_{k} ∏_{(i,k)<(j,k)}(x_{i}^{(k)} − x_{j}^{(k)})},

where ε^{(k)} = (ε_{1}^{(k)}, …, ε_{m_{k}}^{(k)}) is defined by ε_{i}^{(k)} = ♯{j | (i, k) < (j, k)}. But since Λ ∈ Z_{n}^{r,s}, (i, k) < (j, k) is equivalent to i < j by the definition of the order on M. Hence we obtain (3.13.2). Now (3.13.1) is immediate from the definition of R_{α,<}^{±} in (3.1.2) together with (3.13.2).

Note that Schur functions s_{α} (in a general sense) are linearly independent if β = (β^{(0)}, …, β^{(e−1)}) are all distinct, where β^{(k)} is obtained from α^{(k)} by arranging the entries (α_{j}^{(k)} + (m_{k} − j)) in decreasing order. By Corollary 3.7, we know that R_{Λ}^{±} is a polynomial in x, t. Hence it is written as a linear combination of Schur functions s_{β}(x) for β ∈ Z_{n}^{0,0}. It follows that in the expansion of (3.13.1) as a sum of Schur functions, we may only pick up s_{β} with β ∈ Z_{n}^{0,0}. We have the following proposition.

Proposition 3.14. Assume that Λ ∈ Z_{n}^{r,s}. Then R_{Λ}^{±} can be written as

R_{Λ}^{±}(x; t) = ∑_{β∈Z_{n}^{0,0}} u_{α,β}(t)s_{β}(x)

with u_{α,β}(t) ∈ ℤ[t], where u_{α,β} ∈ tℤ[t] for β ≠ α, and u_{α,α} ∉ tℤ[t].

Proof. Since both of R_{Λ}^{±}(x; t) and s_{β}(x) have the stability property, we may assume that ν_{0} = (m_{k}, k) for some k. It follows that v_{α}(t) = 1 and we see that (3.13.1) can be written, in this case, as

(3.14.1)  R_{Λ}^{±} = ∏_{\begin{matrix}ν<ν′,ν≤ν_{0} \\ b(ν′)=b(ν±1) \end{matrix}}(1 − tR_{ν′ν})s_{α}.

Therefore R_{Λ}^{±} can be written as a linear combination of s_{β} with coefficient u_{α,β} ∈ ℤ[t]. Now the last assertion follows from (3.14.1) together with the fact that R_{Λ}^{±}(x; 0) = s_{α}(x). □

3.15. Let m be as in 2.1. We denote by Ξ_{m} = ⨂_{k=0}^{e−1} ℤ[x_{1}^{(k)}, …, x_{m_{k}}^{(k)}]^{𝔖_{m_{k}}} the ring of symmetric polynomials (with respect to 𝔖_{m}) with variables x = (x_{j}^{(k)}). Ξ_{m} has a structure of a graded ring Ξ_{m} = ⨁_{i≥0} Ξ_{m}^{i}, where Ξ_{m}^{i} consists of homogeneous symmetric polynomials of degree i, together with the zero polynomial. We consider the inverse limit

Ξ^{i} = lim_{\overleftarrow{m}} Ξ_{m}^{i}

with respect to homomorphisms ρ_{m′,m} : Ξ_{m′}^{i} → Ξ_{m}^{i}, where m′ = (m′_{0}, …, m′_{e−1}) with m′_{k} = m_{k} + l for some integer l ≥ 0, and ρ_{m′,m} is induced from the homomorphism ⨁_{k} ℤ[x_{1}^{(k)}, …, x_{m_{k}}^{(k)}] → ⨁_{k} ℤ[x_{1}^{(k)}, …, x_{m′_{k}}^{(k)}] given by sending x_{i}^{(k)} to 0 for i ≥ m_{k}, and

leaving other x_{j}^{(k)} invariant. Ξ = ⨁_{i≥0} Ξ^{i} is called the space of symmetric functions. Schur function s_{α}(x) with infinitely many variables x_{1}^{(k)}, … is regarded as an element in Ξ^{n} with n = |α|, and the set {s_{α}(x)} with α ∈ Z_{n}^{0,0} forms a ℤ-basis of Ξ^{n}. Put Ξ[t] = ℤ[t] ⊗_{ℤ} Ξ. For Λ ∈ Z_{n}^{r,s}, the function R_{Λ}^{±} is regarded as an element in Ξ[t], and by Proposition 3.14, the set {R_{Λ}^{±} | Λ ∈ Z_{n}^{r,s}} forms a basis of the ℚ(t)-space Ξ_{ℚ}^{n}[t] = ℚ(t) ⊗_{ℤ} Ξ^{n}. This implies, by Proposition 3.12 (ii), that {q_{α,±} | α ∈ Z_{n}^{0,0}} also gives rise to a basis of Ξ_{ℚ}^{n}[t]. A similar property holds if one replace Ξ^{n} by Ξ_{m}^{n}.

4. Hall-Littlewood functions

4.1. As described in Proposition 3.14, R_{Λ}^{±} can be expressed in terms of Schur functions. However, the transition matrix between R_{Λ}^{±} and s_{α} is not necessarily a triangular shape as a block matrix. In order to recover this property, we shall define a new function Q_{Λ}^{±} by modifying R_{Λ}^{±}. In this section we are only concerned with the set Z_{n}^{0,0} and Z_{n}^{r,s}. Let Λ = Λ(α) ∈ Z^{r,s} and M = M_{Λ} be as in 3.8. First we note the following lemma.

Lemma 4.2. For each 0 ≤ i < e, let α_{i} ∈ Z_{n}^{0,0} be an element such that α_{1}^{(i)} = n and that α_{j}^{(k)} = 0 for (j, k) ≠ (1, i). Assume that α ∈ Z_{n}^{0,0} is such that a(α) is minimum among Z_{n}^{0,0}. Then α coincides with some α_{i}.

Proof. Let Λ′ ∈ Ẑ_{n}^{r,s} and take λ, λ′ in the entry of Λ′ such that 0 < λ′ < λ. Let Λ′′ ∈ Ẑ_{n}^{r,s} be the element obtained from Λ′ by replacing λ by λ + 1, and λ′ by λ′ − 1. Then clearly we have a(Λ′′) < a(Λ′). Hence, repeating this operation, we reach to the minimum element with respect to a, which must have the form Λ(α_{i}). □

4.3. Let A be the subring of ℚ(t) consisting of functions which have no pole at t = 0. Then A is a local ring with the unique maximal ideal tA. Hence A^{∗} = A − tA is the set of units in A. In what follows, we identify the set P_{n,e} with Z_{n}^{0,0}, and fix a total order ≺ on Z_{n}^{0,0} as in 1.4. We consider a square matrix indexed by the set Z_{n}^{0,0}, and regard it as a block matrix, where each block corresponds to a similarity class.

Let X = {X_{α} | α ∈ Z_{n}^{0,0}} and Y = {Y_{α} | α ∈ Z_{n}^{0,0}} be two bases of ℚ(t) ⊗ Ξ_{m}^{n}, indexed by Z_{n}^{0,0}. We denote by M(X, Y) the transition matrix between two bases, i.e., M(X, Y) = (m_{α,β}) if X_{α} = ∑_{β} m_{α,β}Y_{β}. We also use Z_{n}^{r,s} as the index set under the identification Z_{n}^{r,s} ≃ Z_{n}^{0,0}. We shall construct two types of functions P_{Λ}^{±}(x; t) and Q_{Λ}^{±}(x; t) as linear combinations of R_{Λ}^{±}(x; t).

Theorem 4.4. (i) For each Λ ∈ Z_{n}^{r,s}, there exists a unique function P_{Λ}^{±}(x; t) satisfying the following properties.

(a) P_{Λ}^{±}(x; t) can be expressed as

P_{Λ}^{±}(x; t) = ∑_{β∈Z_{n}^{0,0}} c_{α,β}(t)q_{β,±}(x; t),

where c_{α,β}(t) ∈ ℚ(t) and c_{α,β}(t) = 0 unless β ≻ α or β ∼ α.

(b) P_{Λ}^{±}(x; t) can be expressed as

P_{Λ}^{±}(x; t) = s_{α}(x) + ∑_{β∈Z_{n}^{0,0}} u_{α,β}(t)s_{β}(x),

where u_{α,β}(t) ∈ tA, and u_{α,β}(t) = 0 unless β ≺ α and β ≁ α. (ii) For each Λ ∈ Z_{n}^{r,s}, there exists a unique function Q_{Λ}^{±}(x; t) satisfying the fol-

lowing properties.

(a) Q_{Λ}^{±}(x; t) can be expressed as

Q_{Λ}^{±}(x; t) = q_{α,±}(x; t) + ∑_{β∈Z_{n}^{0,0}} d_{α,β}(t)q_{β,±}(x; t),

where d_{α,β}(t) ∈ ℚ(t) and d_{α,β}(t) = 0 unless β ≻ α and β ≁ α.

(b) Q_{Λ}^{±}(x; t) can be expressed as

Q_{Λ}^{±}(x; t) = ∑_{β∈Z_{n}^{0,0}} w_{α,β}(t)s_{β}(x),

where w_{α,β}(t) ∈ A, and w_{α,β}(t) = 0 unless β ≺ α or β ∼ α. Moreover,

w_{α,β} ∈ tA if β ≠ α, and w_{α,α} ∈ A^{∗}.

(In the above formula, the coefficients c_{α,β}(t), etc. depend on the sign ±.)

Proof. We construct the functions P_{Λ}^{±} and Q_{Λ}^{±} by backward induction on the total order ≺ on Z_{n}^{r,s}. We assume that P_{Λ′}^{±}, Q_{Λ′}^{±} have been constructed for any Λ′ such that Λ′ ≻ Λ and that Λ′ ≁ Λ. Let Z be the similarity class containing Λ. We shall construct P_{Λ′′}^{±}, Q_{Λ′′}^{±} simultaneously for Λ′′ ∈ Z.

By Proposition 3.14, the function R_{Λ}^{±} can be written as R_{Λ}^{±} = ∑_{β} u′_{α,β}s_{β} with u′_{α,β}(t) ∈ tℤ[t] if α ≠ β. Let Z_{1} be the set of β ∈ Z^{0,0} such that β ≻ α and that β ≁ α. Then, by using the property (i) (b) in the theorem, one can find a function Q_{Λ}^{±} of the form Q_{Λ}^{±} = R_{Λ}^{±}−∑_{β∈Z_{1}} u′′_{α,β}P_{Λ(β)}^{±} with u′′_{α,β}(t) ∈ tA satisfying the following property;

(4.4.1)  Q_{Λ}^{±} = ∑_{β∼α}w_{α,β}s_{β} + X_{α},

where X_{α} is a sum of s_{γ} such that γ ≺ α and that γ ≁ α with coefficients in tA. Hence we see that w_{α,β} ∈ tA if β ≠ α, and that w_{α,α} ∈ A^{∗}, i.e., Q_{Λ}^{±} satisfies the property (ii), (b). On the other hand, it follows from Proposition 3.12, together with the property (i),(a) in the theorem, that Q_{Λ}^{±} satisfies the property (ii), (a). Thus we have constructed Q_{Λ}^{±}.

Next, we shall construct the function P_{Λ}^{±}. We now regard (4.4.1) as a system of equations for all Λ = Λ(α) in the similarity class Z. Let C = (w_{α,β}) be the matrix with entries in A, indexed by Z, which is the coefficient matrix of this system of equations. Since det C ∈ A^{∗}, the inverse matrix C^{−1} exists, with entries in A. Then by applying C^{−1} to the system of equations (4.4.1), one can find functions P_{Λ}^{±}(x; t) for each Λ = Λ(α) ∈ Z, which can be expressed in terms of Schur functions as

P_{Λ}^{±} = s_{α} + ∑_{β}u_{α,β}s_{β},

where β ∈ Z_{n}^{0,0} runs over all the elements such that β ≺ α and that β ≁ α, and u_{α,β} ∈ tA. Moreover, since P_{Λ}^{±} is a linear combination of various Q_{Λ′}^{±} with Λ′ ∼ Λ, it is expressed as a linear combination of q_{β,±} with β ≻ α or β ∼ α. Hence P_{Λ}^{±} satisfies the required properties, (a), (b) of (i).

It remains to construct P_{Λ}^{±} in the case where a(Λ) is minimal. If a(Λ) is minimal, then Λ = Λ(α_{i}) for some i by Lemma 4.2. Then R_{Λ}^{±} coincides with q_{α_{i},±} by (3.2.3). We consider the similarity class Z containing Λ which consists of certain Λ_{i} = Λ(α_{i}). Then the functions R_{Λ_{i}}^{±} already satisfy the properties required for Q_{Λ_{i}}^{±}. So we may put Q_{Λ_{i}}^{±} = R_{Λ_{i}}^{±}. Then by using a similar argument as above, one can find P_{Λ_{i}}^{±} satisfying (a), (b) of (i) for each Λ_{i} ∈ Z. Thus P_{Λ}^{±} can be constructed.

We show the uniqueness of {P_{Λ}^{±}}. The uniqueness of {Q_{Λ}^{±}} is proved similarly. Suppose that the functions {P′_{Λ}} satisfy the conditions (a) and (b) of (i). By the property (b), the sets {P_{Λ}} and {P′_{Λ}} form bases of the space ℚ(t) ⊗ Ξ_{m}^{n}. Let A = M(P, s) be the transition matrix between the functions {P_{Λ}^{±} | Λ ∈ Z_{n}^{r,s}} and {s_{α} | α ∈ Z_{n}^{0,0}}, and similarly define the transition matrix A′ = M(P′, s) between {P′_{Λ}} and {s_{α}}. Then by our assumption, both of A and A′ are block lower triangular matrices with identity diagonal blocks. Hence A(A′)^{−1} = M(P, P′) is also a lower triangular block matrix with identity diagonal blocks. On the other hand, let B = M(P, q) be the transition matrix between {P_{Λ}^{±}} and {q_{α,±}}, and similarly define B′ = M(P′, q). Then by our assumption, both of B and B′ are block upper triangular matrices, and so is B(B′)^{−1}. But since B(B′)^{−1} = M(P, P′) = A(A′)^{−1}, we see that A(A′)^{−1} is simultaneously lower triangular and upper triangular with identity diagonal blocks. Hence A = A′ and we conclude that P_{Λ}^{±} = P′_{Λ} as asserted. Thus the theorem is proved. □

Remarks 4.5. (i) We call P_{Λ}^{±}(x; t) the Hall-Littlewood function associated to the symbol Λ. It is likely that P_{Λ}^{±}(x; t) ∈ ℤ[x; t]. In some cases, including the classical case, where e = 2, r = 2, these statements are verified to be true. (See Remarks 5.5 (i)).

(ii) The previous construction of P_{Λ}^{±} depend on the choice of the total order ≻ compatible with the similarity classes. However, the construction still works if one replaces the relation α ∼ β by the equivalence relation defined by the condition that a(β) = a(α). In the above known cases, these two types of Hall-Littlewood functions coincide with each other. It would be interesting to know whether this is true in general since it will imply the indenpendence of P_{λ}^{±} from the chocie of the total order.

We can regard P_{Λ}^{±} as an element in ℚ(t) ⊗ Ξ^{n} associated to the symbol class Λ. As a corollary to Theorem 4.4, we have the following.

Corollary 4.6. Let Ω(x, y; t) be as in Proposition 2.5. Then we have

Ω(x, y; t) = ∑_{Λ,Λ′} b_{Λ,Λ′}(t)P_{Λ}^{+}(x; t)P_{Λ′}^{−}(y; t).  (4.6.1)
Ω(x, y; t) = ∑_{Λ}Q_{Λ}^{+}(x; t)P_{Λ}^{−}(y; t) = ∑_{Λ}P_{Λ}^{+}(x; t)Q_{Λ}^{−}(y; t).  (4.6.2)

where in (4.6.1), Λ, Λ′ run over all the elements in ⋃_{n=1}^{∞} Z_{n}^{r,s}, and b_{Λ,Λ′}(t) = 0 unless |Λ| = |Λ′| and Λ ∼ Λ′. In (4.6.2), Λ runs over all the elements in ⋃_{n=1}^{∞} Z_{n}^{r,s}.

Proof. Let

A_{±} = M(q_{±}, P^{±}),  B_{±} = M(m, P^{±}),  C_{±} = M(q_{±}, m)

be the transition matrices between {q_{α,±}} and {P_{Λ}^{±}}, {m_{α}} and {P_{Λ}^{±}}, and {q_{α,±}} and {m_{α}}, respectively. Put D_{±} = ^{t}B_{∓}A_{±}. Then by Theorem 4.4 (i),(a), (A_{±})^{−1} is a block upper triangular matrix, hence so is A_{±}. Now B_{±}^{−1} can be written as

B_{±}^{−1} = M(P^{±}, s)M(s, m).

By Theorem 4.4 (i),(b), M(P^{±}, s) is block lower triangular. On the other hand, M(s, m) is also lower triangular. In fact, for each partition λ, s_{λ} is written as a linear combination of m_{μ} with μ ≤ λ. It follows that s_{α} is a linear combination of m_{β}, where β satisfies the following property. If we write Λ(α) = (Λ_{0}, …, Λ_{e−1}) and Λ(β) = (Λ′_{0}, …, Λ′_{e−1}), then the partitions Λ_{i} and Λ′_{i} have the same size, and Λ′_{i} ≤ Λ_{i} (as partitions). This implies that a(Λ′) ≥ a(Λ) and the equality holds only when Λ = Λ′. Hence M(s, m) is lower triangular with identity diagonal blocks. As a conclusion, we see that B_{±}^{−1} is block lower triangular, and so D_{±} is a block upper triangular matrix.

On the other hand, by (2.5.1), the matrices C_{±} satisfy the relation ^{t}C_{+} = C_{−}. Since D_{±} = ^{t}B_{∓}C_{±}B_{±}, we see that ^{t}D_{+} = D_{−}. This implies that D_{±} is a block diagonal matrix. If we put A_{+} = (A_{α,Λ}^{+}), B_{−} = (B_{α,Λ′}^{−}), we have

∑_{α}q_{α,+}(x; t)m_{α}(y) = ∑_{Λ,Λ′,α}A_{α,Λ}^{+}B_{α,Λ′}^{−}P_{Λ}^{+}(x; t)P_{Λ′}^{−}(y; t)

= ∑_{Λ,Λ′} b_{Λ′,Λ}^{+}(t)P_{Λ}^{+}(x; t)P_{Λ′}^{−}(y; t),

where the matrix (b_{Λ′,Λ}^{+}(t)) coincides with D_{+}. Thanks to (2.5.1), by putting b_{Λ,Λ′}(t) = b_{Λ,Λ′}^{−}(t) = b_{Λ′,Λ}^{+} (here D_{−} = (b_{Λ,Λ′}^{−})), we obtain the formula (4.6.1).

We now look at the previous argument more precisely. In the argument below, we use a notation X_{D} to denote the diagonal part of the block matrix X. Let us consider the matrix D_{±} = ^{t}B_{∓}A_{±}. Since we know that D_{±} is block-wisely diagonal, and^{t}B_{∓}, A_{±} are block upper triangular, we see that D_{±} = (^{t}B_{∓})_{D}(A_{±})_{D}. But M(P^{±}, s)_{D} is the identity matrix by Theorem 4.4, (i), (b). Also it is easy to check that M(s, m)_{D} is the identity matrix, and so we have D_{±} = (A_{±})_{D}. Let M(Q^{±}, P^{±}) be the transition matrix between Q_{Λ}^{±} and P_{Λ}^{±}. Then we have

M(Q^{±}, P^{±}) = M(Q^{±}, P^{±})_{D} = M(Q^{±}, q_{±})_{D}M(q_{±}, P^{±})_{D}.

Since M(Q^{±}, q_{±})_{D} = I by Theorem 4.4 (ii), (a), and M(q_{±}, P^{±}) = A_{±} by definition, we have M(Q^{±}, P^{±}) = D_{±}. Now the formula (4.6.2) follows easily from (4.6.1), if we notice that b_{Λ,Λ′} = b_{Λ,Λ′}^{−} = b_{Λ′,Λ}^{+} with D_{±} = (b_{Λ,Λ′}^{±}). □

4.7. As discussed in 3.15, {s_{α}(x)} gives rise to a basis of the ℤ[t]-module Ξ[t], hence so is {m_{α}(x)}. Also, we see that {q_{α,±}(x; t)}, {Q_{Λ}^{±}(x; t)}, {P_{Λ}^{±}(x; t)} turn out to be bases of Ξ_{ℚ}[t]. Moreover, {p_{α}(x)} gives a basis of ℂ(t)-space Ξ_{ℂ}[t] = ℂ(t) ⊗_{ℤ} Ξ. We now define a scalar product on Ξ_{ℚ}[t] by the condition that

(4.7.1)  ⟨q_{α,+}(x; t), m_{β}(x)⟩= δ_{α,β},

and extend it to a sesquilinear form on Ξ_{ℂ}[t]. By using a similar argument as in [M, Chap. I, 4], it follows from Corollary 4.6 and Proposition 2.5 that we have

⟨m_{α}(x), q_{β,−}(x; t)⟩= δ_{α,β},
⟨P_{Λ}^{+}(x; t), Q_{Λ′}^{−}(x; t)⟩=⟨Q_{Λ}^{+}(x; t), P_{Λ′}^{−}(x; t)⟩= δ_{Λ,Λ′},  (4.7.2)
⟨p_{α}(x), p_{β}(x)⟩= z_{α}(t)δ_{α,β}.

In the special case where e = 2, the function q_{α,+} (resp. P_{Λ}^{+}, Q_{Λ}^{+}) coincides with q_{α,−} (resp. P_{Λ}^{−}, Q_{Λ}^{−}). We also have p_{α} = p̄_{α} and z_{α}(t) ∈ ℚ(t). Hence in this case, the scalar product ⟨ , ⟩ turns out to be symmetric.

Proposition 4.8. The functions P_{Λ}^{±} are characterized by the following two properties. (Then the functions Q_{Λ}^{±} are characterized as dual bases of P_{Λ}^{∓} in the sense of (4.7.2)).

(i) P_{Λ}^{±}(x; t) can be expressed in terms of s_{β}(x) as

P_{Λ}^{±} = s_{α} + ∑_{β}u_{α,β}^{±}s_{β}

with u_{α,β}^{±} ∈ ℚ(t), where Λ = Λ(α), and u_{α,β}^{±} = 0 unless β ≺ α and β ≁ α. (ii) ⟨P_{Λ}^{+}, P_{Λ′}^{−}⟩= 0 unless Λ ∼ Λ′.

Proof. The fact that P_{Λ}^{±} satisfy the properties (i) and (ii) follows from the previous discussion. We show, by induction on the order ≺, that (i) ∼ (iii) determine P_{Λ}^{±} uniquely. First we note that {P_{Λ}^{±} | Λ ∈ Z_{n}^{r,s}} give bases of Ξ_{ℚ}^{n}[t] by (i). Then the matrix (⟨P_{Λ}^{+}, P_{Λ′}^{−}⟩)_{Λ,Λ′} is non-singular. It follows, by (ii), that the diagonal submatrix corresponding to each similarity class is also non-singular. Now take Λ ∈ Z_{n}^{r,s}, and assume that P_{Λ′}^{±} are determined for Λ′ such that Λ′ ≺ Λ and that Λ′ ≁ Λ. Note that the expression in (i) is equivalent to the expression

(4.8.1)  P_{Λ}^{±} = s_{α} + ∑_{Λ′} d_{Λ,Λ′}^{±}P_{Λ′}^{±},

where d_{Λ,Λ′}^{±} = 0 unless Λ′ ≺ Λ and Λ′ ≁ Λ. Take such Λ′. Then we have

(4.8.2)  ⟨s_{α}, P_{Λ′}^{−}⟩+ ∑_{Λ′′∼Λ′} d_{Λ,Λ′′}^{+}⟨P_{Λ′′}^{+}, P_{Λ′}^{−}⟩= 0.

When Λ′ runs over symbols contained in a fixed similarity class, (4.8.2) may be regarded as a system of linear equations, with unknown variables {d_{Λ,Λ′′}^{+}}. Since the matrix (⟨P_{Λ′}^{+}, P_{Λ′′}^{−}⟩) (Λ′, Λ′′ are in a fixed similarity class) is non-singular by the above remark, we see that (4.8.2) determines the coefficients d_{Λ,Λ′′}^{+} uniquely. Hence P_{Λ}^{+} is determined uniquely. A similar argument, by using the equality ⟨P_{Λ′}^{+}, P_{Λ}^{−}⟩= 0 instead of (4.8.2), implies that P_{Λ}^{−} is also unique. Thus the proposition is proved. □

Remark 4.9. By making use of the arguments in Proposition 4.8, one can give an alternate construction of P_{Λ}^{±}. In fact, by substituting t = 0 into (2.5.2), we have

Ω(x, y; 0) = ∑_{α}z_{α}^{−1}p_{α}(x)p̄_{α}(y).

On the other hand, a similar argument as in [M, I, 4] implies that

Ω(x, y; 0) = ∑_{α}s_{α}(x)s_{α}(y).

It follows that one can define a hermitian form on ℂ ⊗_{ℤ} Ξ satisfying the properties that ⟨p_{α}, p_{β}⟩= z_{α}δ_{α,β} and that ⟨s_{α}, s_{β}⟩= δ_{α,β}. In particular, the sesquilinear form on ℂ[t] ⊗_{ℤ} Ξ defined in 4.7 is transfered to the hermitian form on ℂ ⊗_{ℤ} Ξ by substituting t = 0.

We now construct P_{Λ}^{±} satisfying two properties in Propostion 4.8 as follows. Take Λ ∈ Z_{n}^{r,s}. We assume that P_{Λ′}^{±} are already constructed for Λ′ such that Λ ≺ Λ and that Λ′ ≁ Λ. We assume further that P_{Λ′}^{±}(x; t) ∈ A_{ℂ}⊗_{ℤ}Ξ and that P_{Λ′}^{±}(x; 0) = s_{α}′(x) for Λ′ = Λ(α′). (Here A_{ℂ} = ℂ⊗A is the subring of ℂ(t) consisting of functions which have no pole at t = 0). We put P_{Λ}^{±} as in (4.8.1), and determine the coefficients d_{Λ,Λ′}^{±} so that it satisfies the condition (ii) in the proposition. We consider the system of equations (4.8.2). Note that the coefficient matrix (⟨P_{Λ′′}^{+}, P_{Λ′}^{−}⟩) is non-degenerate since it gives rise to a matrix (⟨s_{α}′′, s_{α}′⟩) by substituting t = 0, which is the identity matrix by the above remark. In particular, the determinant of this matrix lies in A_{ℂ}. Thus the equation (4.8.2) has a unique solution d_{Λ,Λ′} ∈ A_{ℂ} and we obtain P_{Λ}^{+} ∈ A_{ℂ} ⊗_{ℤ} Ξ. Since ⟨s_{α}, s_{α}′⟩ = 0, we have d_{Λ,Λ′}^{+} ∈ tA_{ℂ}. This implies that P_{Λ}^{+}(x; 0) = s_{α}(x). The case for P_{Λ}^{−} is done similarly, and P_{Λ}^{±} is constructed inductively.

5. Green functions

5.1. Let X_{±}(t) be the transition matrix M(p, P^{±}) between the power sum symmetric functions {p_{α}} and the Hall-Littlewood functions {P_{Λ}^{±}}, i.e.,

(5.1.1)  p_{α}(x) = ∑_{Λ}X_{α,±}^{Λ}(t)P_{Λ}^{±}(x; t).

(Here α is the row-index and Λ is the column-index.) Then by Theorem 4.4, X_{α,±}^{Λ}(t) ∈ ℂ(t) and is equal to zero unless |α| = |Λ|. Moreover since P_{Λ}^{±}(x; 0) = s_{α}(x) with Λ = Λ(α) by Theorem 4.4 (i) (b), we have

(5.1.2)  X_{β,±}^{Λ}(0) = χ^{α}(w_{β})

by (2.1.3). It follows that X_{±}(0) = M(p, s) is the character table of W. In particular, X_{±}(0) is independent of the sign, and we denote it simply as X(0). By combining Corollary 4.6 with (2.5.2), we have

∑_{|β|=n} z_{β}(t)^{−1}p_{β}(x)p̄_{β}(y) = ∑_{|Λ|=|Λ′|=n} b_{Λ,Λ′}(t)P_{Λ}^{+}(x; t)P_{Λ′}^{−}(y; t).

We put D(t) = D_{−} = (b_{Λ,Λ′}(t)), and denote by Z(t) the diagonal matrix with entries z_{β}(t). (Both are indexed by the set Z_{n}^{r,s} ≃ Z_{n}^{0,0}.) Substituting (5.1.1) into the above equation, we have

(5.1.3)  ^{t}X_{+}(t)Z(t)^{−1}X̄_{−}(t) = D(t),

where X̄_{−}(t) is the complex conjugate of the matrix X_{−}(t). (Note that P_{Λ}^{±}(x; t) ∈ ℚ(t)[x].) We put Λ(t) = D(t)^{−1}. Then the formula (5.1.3) is equivalent to

(5.1.4)  X̄_{−}(t)Λ(t)^{t}X_{+}(t) = Z(t).

5.2. Let K_{±}(t) = M(s, P^{±}) be the transition matrix between Schur functions and Hall-Littlewood functions, i.e.,

s_{β}(x) = ∑_{Λ(α)} K_{β,α}^{±}(t)P_{Λ(α)}^{±}(x; t).

Then by Theorem 4.4 (i) (b), , K_{±}(t) is a block lower triangular matrix with identity diagonal blocks, with entries K_{β,α}^{±}(t) in ℚ(t). Moreover, K_{±}(0) is the identity matrix. However, on the contrary to the classical case, K_{±}(1) is not necessarily equal to the Kostka matrix M(s, m). Since M(s, P^{±}) = M(p, s)^{−1}M(p, P^{±}), we have K_{±}(t) = X(0)^{−1}X_{±}(t). Substituting this into (5.1.4), we have

(5.2.1)  K_{−}(t)Λ(t)^{t}K_{+}(t) = X̄(0)^{−1}Z(t)^{t}X(0)^{−1}.
5.3. We now define Green functions Q_{β,±}^{Λ}(t) ∈ ℂ(t), with Λ = Λ(α), by
Q_{β,±}^{Λ}(t) = t^{a(Λ)}X_{β,±}^{Λ}(t^{−1}).

If we put K̃_{β,α}^{±}(t) = t^{a(α)}K_{β,α}^{±}(t^{−1}), Q_{β,±}^{Λ}(t) can be written as

Q_{β,±}^{Λ}(t) = ∑_{γ}χ^{γ}(w_{β})K̃_{γ,α}^{±}(t).

Let K̃_{±}(t) = (K̃_{β,α}^{±}(t)). Then K̃_{±}(t) = K_{±}(t^{−1})T for a diagonal matrix T with diagonal entries t^{a(α)}. Hence (5.2.1) can be rewritten as

(5.3.1)  K̃_{−}(t)Λ′(t)^{t}K̃_{+}(t) = X̄(0)^{−1}Z(t^{−1})^{t}X(0)^{−1},

where Λ′(t) = T^{−1}Λ(t^{−1})T. Note that Λ′(t) is still a block diagonal matrix, and K̃_{±}(t) are block lower triangular matrices, where the diagonal blocks consist of scalar matrices t^{a(α)}I.

Put 𝔾(t) = (t − 1)^{n}t^{N^{∗}}P_{W} (t) ∈ ℤ[t]. If W is a Weyl group of type B_{n}, 𝔾(q) is the order of SO_{2n+1}(F_{q}) or Sp_{2n}(F_{q}). Even in the case where W = G(e, 1, n), 𝔾(t) is regarded as an order of a formal group (see [BMM]). We put Λ̃(t) = t^{−n}𝔾(t)Λ′(t). Recall that Ω′ = (ω′_{α,β}) is the matrix defined in (1.5.1). Then the following result gives a combinatorial description of the solution for the equation (1.5.2).

Theorem 5.4. We have

K̃_{−}(t)Λ̃(t)^{t}K̃_{+}(t) = Ω′.

Hence P′ = K̃_{−}(t), P′′ = K̃_{+}(t) and Λ′ = Λ̃(t) gives a solution for the equation (1.5.2).

Proof. Let M be the right hand side of (5.3.1). We shall compute M. Let z_{α} be as in 2.4, and H the diagonal matrix with diagonal entries z_{α}^{−1}. Since X(0) is the character table of W, we have ^{t}X(0)HX̄(0) = I. It follows that

(5.4.1)  M = ^{t}X(0)HZ(t^{−1})HX̄(0).

Now it is easy to see that if α = (α_{j}^{(k)}) ∈ P_{n,e} and w_{α} is the corresponding element in W, then we have

det_{V} (t · id_{V} −w_{α}) = ∏_{k=0}^{e−1} ∏_{j=1}^{l(α^{(k)})}(t^{α_{j}^{(k)}} − ζ^{k}).

Hence by (2.4.2), z_{α}(t^{−1}) = z_{α}t^{n}det_{V} (t · id_{V} −w_{α})^{−1}. In particular, the αβ-entry of M is equal to

t^{n}|W |^{−1} ∑_{w∈W} \frac{χ^{α}(w)χ̄^{β}(w)}{det_{V} (t · id_{V} −w)}.

Therefore Ω′ = t^{N^{∗}−n}(t − 1)^{n}P_{W} (t)M and the theorem follows. □

Concerning the Kostka functions, we can expect that

Conjecture 5.5 The Kostka function K_{α,β}^{±}(t) is a polynomial in t with positive integral coefficients, with deg K_{α,β}^{±}(t) ≤ a(β) − a(α).

Remarks 5.6. (i) The conjecture implies that K̃_{α,β}^{±}(t) is also a polynomial with positive integral coefficients. In particular, we see that p_{α,β}(t), λ_{α,β}(t) ∈ ℤ[t] (cf. (1.4.2)). The conjecture also implies that P_{Λ}^{±} ∈ ℤ[x; t], and Q_{Λ}^{±} ∈ ℤ[x; t] also. In fact, from the discussion in 4.7, we see that {q_{α,±}(x; t)} and {m_{α}(x)}, and {Q_{Λ}^{±}(x; t)} and {P^{∓}(x; t)} are dual bases of each other. Hence we have

(5.6.1)  M(Q^{∓}, q_{∓}) = M(P^{±}, m)^{∗} = (K_{±}(t)^{−1}K)^{∗} = ^{t}K_{±}(t)K^{∗},

where K = M(s, m) is the Kostka matrix, and X^{∗} denotes the transposed inverse of the matrix X. Since K^{∗} is a matrix with entries in ℤ, we see that K_{±}(t) is a matrix with entries in ℤ[t] if and only if M(Q^{∓}, q_{∓}) is a matrix with entries in ℤ[t]. It follows that P_{Λ}^{±}(x; t) lie in ℤ[x; t] if an only if the coefficients d_{α,β}(t) in Theorem 4.4, (ii), (a) lie in ℤ[t] for any α, β ∈ Z_{n}^{0,0}.

(ii) The condition that Hall-Littlewood functions P_{Λ}^{±}(x; t) lie in ℤ[t; x] is equivalent to that K̃_{α,β}^{±}(t) ∈ ℤ[t, t^{−1}] for any α, β ∈ Z_{n}^{0,0}. In fact, if K̃_{α,β}^{±}(t) ∈ ℤ[t, t^{−1}], then K_{α,β}^{±}(t) ∈ ℤ[t, t^{−1}] also. But by Theorem 4.4, the coefficients of M(P^{±}, s) lie in A with det M(P^{±}, s) = 1. Hence the entries of the matrix M(s, P^{±}), i.e., the Kostka functions K_{α,β}^{±}(t) ∈ A. It follows that K_{α,β}^{±}(t) ∈ ℤ[t]. Therefore the entries of M(P^{±}, s) lie in ℤ[t], and we have P_{Λ}^{±}(x; t) ∈ ℤ[x; t]. The other implication is obvious.

It is known that Green functions of classical groups are polynomials in ℤ[t]. It follows that if e = 2, r = 2, then Kostka functions K̃_{α,β}^{−}(t) = K̃_{α,β}^{+}(t) lie in ℤ[t]. Hence in this case, we see that P_{Λ}^{±}(x; t) ∈ ℤ[x; t]. Also, it is verified by Geck and Malle that for small n, e with r = 1 or 2, the equation (1.5.2) has a solution with coefficients in ℤ[t]. Thus in these cases, also we have P_{Λ}^{±}(x; t) ∈ ℤ[x; t].

6. Some special cases

6.1. In this section, we shall derive some explicit results for Q_{Λ}^{±} in the case where e = 2 and the symbols Λ satisfy a certain special condition. So, from now on, except 6.7, we assume that e = 2, r ≥ 1, and that m_{0} = m + 1, m_{1} = m for some m > 0. Let Λ ∈ Z_{n}^{r,s} be a symbol. We arrange the entries in Λ in decreasing order ν_{1} ≥ ν_{2} ≥ ⋯ ≥ ν_{M}, where M = 2m + 1. We say that Λ is very special if ν_{i} ≥ ν_{i+1} + r for i = 1, 2, …. We show the following.

Proposition 6.2. Assume that e = 2 and that Λ = Λ(α) ∈ Z_{n}^{r,s} is very special. Then Q_{Λ}^{±} coincides with R_{Λ}^{±}. In particular, Q_{Λ}^{±} can be expressed as

Q_{Λ}^{±}(x; t) = q_{α}(x; t) + ∑_{β∈Z_{n}^{0,0}} d_{α,β}(t)q_{β}(x; t),

where d_{α,β}(t) ∈ ℤ[t] and d_{α,β}(t) = 0 unless β ≻ α and β ≁ α.

6.3. The proposition will be proved in 6.6 after some preliminaries. In view of Theorem 4.4 (ii), together with Propositions 3.12, 3.14, it is enough to show that R_{Λ}^{±} can be expressed as a linear combination of s_{β}(x) with β ∈ Z_{n}^{0,0} such that β ≺ α or β ∼ α. Now we may assume that ν_{0} = (m_{k}, k) for some k. Then R_{Λ}^{±} can be expressed as in (3.14.1) by using lowering operators, and so it is written as a linear combination of s_{α}′(x) with α′ ∈ Ẑ_{n}^{0,0}. Since s_{α}′ with α′ ∈ Ẑ_{n}^{0,0} is rewritten as ±s_{α}′′ with α′′ ∈ Z_{n}^{0,0}, R_{Λ}^{±} can be written as a linear combination of s_{α}′′ (α′′ ∈ Z_{n}^{0,0}) with coefficients in ℤ[t].

In order to see the values a(α′′) for α′′ ∈ Z_{n}^{0,0} appearing in the expansion of R_{Λ}^{±} in terms of Schur functions, one needs to consider the symbol Λ′′ = Λ(α′′). The process for obtaining Λ′′ from Λ = Λ(α) is described as follows; for a partition α : α_{1} ≥ ⋯ ≥ α_{m′} ≥ 0, let λ : λ_{1} > ⋯ > λ_{m′} ≥ 0 (resp. μ : μ_{1} > ⋯ > μ_{m′} ≥ 0) be a sequence defined by λ_{i} = α_{i} + (m′ − i)r + s′ (resp. μ_{i} = α_{i} + (m′ − i)), respectively. Let μ′ = (μ′_{1}, …, μ′_{m′}) be a sequence obtained by applying R to μ. We assume that μ′_{i} ≥ 0. Let μ′′ be a partition μ′′_{1} ≥ ⋯ ≥ μ′′_{m′} ≥ 0 obtained from μ′ by rearranging the entries in decreasing order. We define a partition λ′′ : λ′′_{1} ≥ ⋯ ≥ λ′′_{m′} ≥ 0 by λ′′_{i} = μ′′_{i} + (m′ − i)(r − 1) + s′, and put λ′′ = R̃λ. Now Λ′′ = (Λ′′_{0}, Λ′′_{1}) is obtained from Λ = (Λ_{0}, Λ_{1}) by Λ′′_{k} = R̃Λ_{k}, where s′ = s, m′ = m + 1 (resp. s′ = 0, m′ = m) if k = 0 (resp. k = 1), respectively.

The procedure for obtaining Λ′′ from Λ has an alternate description. Let Λ = (λ_{1}, …, λ_{m_{1}}) be a sequence of integers and assume that λ_{i} < λ_{i+1} + (r − 1) for some i. We define Λ′ = R_{i}^{∨}Λ = (λ′_{1}, …, λ′_{m_{1}}) by

λ′_{i} = λ_{i+1} + (r − 1),
λ′_{i+1} = λ_{i} − (r − 1),  (6.3.1)
λ′_{j} = λ_{j}  (j ≠ i, i + 1).

Then we have λ′_{i} ≥ λ′_{i+1} + r. Let Λ′ = (Λ′_{0}, Λ′_{1}) be the pair of compositions obtained from Λ by putting Λ′_{k} = RΛ_{k}. We rearrange the entries in Λ′_{k} by the repeated use of the operators R_{i}^{∨}. (Note that in each step of the operation of R_{i}^{∨}, the resulting sequence again belongs in ℤ_{≥0}^{m′}.) Finally we obtain the sequence whose entries are arranged in decreasing order, which coincides with Λ′′_{k}.

6.4. Let Λ = (Λ_{0}, Λ_{1}) be a very special symbol in Z_{n}^{r,s}. We arrange the entries of Λ as ν_{1} > ν_{2} > ⋯ > ν_{M}. Hence Λ_{0} = (ν_{1} > ν_{3} > ⋯ > ν_{2m+1}) and Λ_{1} = (ν_{2} > ν_{4} > ⋯ > ν_{2m}). Let R = ∏ R_{ν′,ν} be a lowering operator occurring in the expansion of (3.14.1), and put α′ = R(α). We consider α′′ and Λ′′ as in 6.3. For each u ≥ 1, let ν′′_{1} ≥ ν′′_{2} ≥ ⋯ ≥ ν′′_{u} be the largest u elements in the entries of Λ′′. Let ν′_{2a_{1}−1}, ν′_{2a_{2}−1}, …, ν′_{2a_{p}−1}, ν′_{2b_{1}}, ν′_{2b_{2}}, …, ν′_{2b_{q}} be the entries in Λ′ which produce ν′′_{1}, …, ν′′_{u}, where a_{1} < a_{2} < ⋯ < a_{p}, b_{1} < b_{2} < ⋯ < b_{q} with p + q = u. (Hence ν_{2a_{1}−1}, ν_{2a_{2}−1}, … are entries in Λ_{0}, and ν_{2b_{1}}, ν_{2b_{2}}, … are in Λ_{1}). First we show the following lemma.

Lemma 6.5. Let the notations be as above. Assume that p ≥ q. Then we have

∑_{i=1}^{u} ν′′_{i} ≤ ∑_{i=1}^{p} ν_{2a_{i}−1} + ∑_{j=1}^{q} ν_{2b_{j}} + r{∑_{i=1}^{p}(a_{i} − (q + 1)) + ∑_{j=1}^{q} b_{j}}.

Proof. Put X = ∑_{i=1}^{p}(a_{i} − (q + 1)) + ∑_{j=1}^{q} b_{j}. First we show the following formula.

(6.5.1)  ∑_{i=1}^{u} ν′′_{i} ≤ ∑_{i=1}^{p} ν′_{2a_{i}−1} + ∑_{j=1}^{q} ν′_{2b_{j}} + (r − 1)X.

We write Λ′ = (Λ′_{0}, Λ′_{1}) (resp. Λ′′ = (Λ′′_{0}, Λ′′_{1})) as Λ′_{0} = (λ′_{i}), Λ′_{1} = (μ′_{j}) (resp. Λ′′_{0} = (λ′′_{i}), Λ′′_{1} = (μ′′_{j})). Let λ′_{i_{1}} be the element which produces λ′′_{1} by the operations R_{i}^{∨} given in (6.3.1). Then we have λ′′_{1} ≤ λ′_{i_{1}} + (r − 1)(i_{1} − 1) since it is obtained by at most (i_{1} − 1) iteration of R_{i}^{∨}. If λ′_{i_{2}} is the element which produces λ′′_{2}, then we have λ′′_{2} ≤ λ′_{i_{2}} + (r − 1)(i_{2} − 2). In fact, this is clear if i_{1} < i_{2}. If i_{1} > i_{2}, then λ′_{i_{2}} is modified to λ_{i_{2}}^{∗} = λ′_{i_{2}} − (r − 1) under the process of obtaining λ′′_{1} from λ′_{i_{1}} (cf. (6.3.1)). After that λ′′_{2} is obtained from λ_{i_{2}}^{∗} by at most (i_{2} − 1) iteration of R_{i}^{∨}, and we get the required formula. A similar argument implies that, in general, if λ′_{i_{a}} produces λ′′_{a}, then λ′′_{a} ≤ λ′_{i_{a}} + (r − 1)(i_{a} − a). The same is true for Λ′_{1}.

Applying this to our situation, we see that

∑_{i=1}^{u} ν′′_{i} ≤ ∑_{i=1}^{p} ν′_{2a_{i}−1} + ∑_{j=1}^{q} ν′_{2b_{j}} + (r − 1)Y,

where

Y = ∑_{i=1}^{p} a_{i} + ∑_{j=1}^{q} b_{j} − \frac{1}{2}p(p + 1) − \frac{1}{2}q(q + 1).

But since p ≥ q, we have p(p + 1)/2 + q(q + 1)/2 ≥ p(q + 1). Hence Y ≤ X, and (6.5.1) holds.

Next we show that

(6.5.2)  ∑_{i=1}^{p} ν′_{2a_{i}−1} + ∑_{j=1}^{q} ν′_{2b_{j}} ≤ ∑_{i=1}^{p} ν_{2a_{i}−1} + ∑_{j=1}^{q} ν_{2b_{j}} + X.

For i = 1, …, q + 1, let us define a subset I_{i} of {1, 2, …, p} by

I_{i} = {j | b_{i−1} < a_{j} ≤ b_{i}},

where by convention, b_{0} = 0, b_{q+1} = ∞. Now take ν_{2a_{i}−1} for i ∈ I_{1}. Then by the effect of the lowering operator, we have ν′_{2a_{i}−1} ≤ ν_{2a_{i}−1} + (a_{i} − 1). (Note that ν_{2a_{i}−1} > ν_{2a_{i}} by our assumption). A similar argument may work also for ν_{2b_{1}}. But in this case, if the pair (ν_{2a_{i}−1}, ν_{2b_{1}}) with a_{i} ∈ I_{1} occurs in the expression of lowering operator R = ∏ R_{ν′,ν}, ν_{2a_{i}−1} must be replaced by ν_{2a_{i}−1} − 1. Hence, we have

∑_{i∈I_{1}} ν′_{2a_{i}−1} + ν′_{2b_{1}} ≤ ∑_{i∈I_{1}} ν_{2a_{i}−1} + ν_{2b_{1}} + ∑_{i∈I_{1}}(a_{i} − 1) + (b_{1} − |I_{1}|).

Extending this argument to the general situation, we see that

∑_{i=1}^{p} ν′_{2a_{i}−1} + ∑_{j=1}^{q} ν′_{2b_{j}} ≤ ∑_{i=1}^{p} ν_{2a_{i}−1} + ∑_{j=1}^{q} ν_{2b_{j}} + Z,

where

Z = ∑_{i∈I_{1}}(a_{i} − 1) + ∑_{i∈I_{2}}(a_{i} − 2) + ⋯ + ∑_{i∈I_{q+1}}(a_{i} − (q + 1))
+ (b_{1} − |I_{1}|) + (b_{2} − (|I_{1}| + |I_{2}|)) + ⋯ + (b_{q} − (|I_{1}| + ⋯ + |I_{q}|)).

But then

Z = ∑_{i=1}^{p} a_{i} + ∑_{j=1}^{q} b_{j} − ∑_{i=1}^{q+1} i|I_{i}| − ∑_{j=1}^{q}(q − j + 1)|I_{j}|

= ∑_{i=1}^{p} a_{i} + ∑_{j=1}^{q} b_{j} − (q + 1) ∑_{i=1}^{q+1} |I_{i}|

= ∑_{i=1}^{p} a_{i} + ∑_{j=1}^{q} b_{j} − (q + 1)p.

Hence we have Z = X, and (6.5.2) holds. Now the lemma follows immediately from (6.5.1) and (6.5.2). □

6.6. We now prove the proposition. In order to prove it, it is enough to show the following. For u = 1, …, M, we have

(6.6.1)  ν′′_{1} + ν′′_{2} + ⋯ + ν′′_{u} ≤ ν_{1} + ν_{2} + ⋯ + ν_{u}.

In fact, (6.6.1) implies that a(Λ′′) ≥ a(Λ), and that Λ′′ ∼ Λ if a(Λ′′) = a(Λ). It follows that α′′ ≺ α or α′′ ∼ α.

Let p and q are as in 6.4. We prove (6.6.1) under the assumption that p ≥ q. The case where p < q is done in a similar way, by using an appropriate modification of Lemma 6.5. Let l be the largest integer such that a_{l} ≤ q. Then since Λ is very special, we see that

(6.6.2)  ν_{2a_{i}−1} + r(a_{i} − (q + 1)) ≤ ν_{a_{i}+q}

for i > l. Moreover in this case, a_{i} + q ≥ 2q + 1. Next, let k be the largest integer such that a_{k} ≤ q and that 2a_{k} − k ≤ q. (Since a_{1} < a_{2} < ⋯, the sequence {2a_{i} − i} increases along i). Then we have

A = ∑_{j=1}^{q} ν_{2b_{j}} + r ∑_{j=1}^{q} b_{j}

≤ ∑_{j_{1}=1}^{2a_{1}−2} ν_{b_{j_{1}}} + ∑_{j_{2}=2a_{1}−1}^{2a_{2}−3} ν_{b_{j_{2}}+1} + ⋯ + ∑_{j_{k+1}=2a_{k}−k}^{q} ν_{b_{j_{k+1}}+k},

since b_{j_{2}} ≥ j_{2} ≥ 1, …, b_{j_{k+1}} ≥ j_{k+1} = 2a_{k} − k ≥ k. Hence

A ≤ ∑_{j_{1}=1}^{2a_{1}−2} ν_{j_{1}} + ∑_{j_{2}=2a_{1}−1}^{2a_{2}−3} ν_{j_{2}+1} + ⋯ + ∑_{j_{k+1}=2a_{k}−k}^{q} ν_{j_{k+1}+k}.

It follows that

(6.6.3)  A + ∑_{i=1}^{k} ν_{2a_{i}−1} ≤ ∑_{j=1}^{q+k} ν_{j}.

Suppose that k < i ≤ l. Then 2a_{i} − i > q, and so 2a_{i} − 1 ≥ q + i. It follows that ν_{2a_{i}−1} − r((q + 1) − a_{i}) ≤ ν_{2a_{i}−1} ≤ ν_{q+i}. By substituting this into (6.6.3), we have

A + ∑_{i=1}^{l} ν_{2a_{i}−1} − r ∑_{i=1}^{l}((q + 1) − a_{i}) ≤ ∑_{j=1}^{q+l} ν_{j}.

Now combining the last formula with (6.6.2), we have

∑_{i=1}^{p} ν_{2a_{i}−1} + ∑_{j=1}^{q} ν_{2b_{j}} + rX ≤ ∑_{j=1}^{q+l} ν_{j} + ∑_{i=l+1}^{p} ν_{a_{i}+q} ≤ ∑_{i=1}^{u} ν_{i}.

This formula implies (6.6.1) in view of Lemma 6.5. Hence Proposition 6.2 is proved.

6.7. In this subsection, we assume that e is arbitrary, and deduce some formula for the Kostka matrix K = M(s, m) and K^{∗}. For each α = (α^{(0)}, …, α^{(e−1)}) ∈ Z_{n}^{0,0}, we define a function h_{α}(x) by

h_{α}(x) = ∏_{k=0}^{e−1} h_{α^{(k)}}(x^{(k)}),

where h_{α^{(k)}}(x^{(k)}) denotes the complete symmetric function associated to a partition α^{(k)}. As discussed in Remark 4.9, we have

(6.7.1)  Ω(x, y; 0) = ∏_{k=0}^{e−1} ∏_{i,j}(1 − x_{i}^{(k)}y_{j}^{(k)})^{−1} = ∑_{α}z_{α}^{−1}p_{α}(x)p̄_{α}(y),

Moreover, by a similar argument as in [M, I, 4], we have

Ω(x, y; 0) = ∑_{α}h_{α}(x)m_{α}(y) = ∑_{α}m_{α}(x)h_{α}(y),  (6.7.2)
Ω(x, y; 0) = ∑_{α}s_{α}(x)s_{α}(y).

Now one can define a scalar product⟨, ⟩on Ξ, by requiring that⟨h_{α}, m_{β}⟩= δ_{α,β}. Then, by (6.7.2), we have

⟨m_{α}, h_{β}⟩= δ_{α,β},
⟨s_{α}, s_{β}⟩= δ_{α,β}.

In particular, the scalar product is symmetric. It can be extended to a hermitian form on ℂ ⊗_{ℤ} Ξ, which satisfies the property, by (6.7.1), that

⟨p_{α}, p_{β}⟩= z_{α}^{−1}δ_{α,β}.

It follows from this that K^{∗} coincides with the transition matrix M(s, h) between Schur functions {s_{α}(x)} and complete symmetric functions {h_{α}(x)}. Then in view of [M, I, 3.4′′], we have the following.

(6.7.3) K^{∗} coincides with the matrix of the operator ∏_{ν<ν′} ∏_{b(ν′)=b(ν)}(1 − R_{ν,ν′}).

As a corollary to Proposition 6.2, we have the following result for certain type of Kostka functions and Green functions. (Since e = 2, the sign ± is ignored in the subsequent discussion.)

Corollary 6.8. Assume that e = 2 and that Λ = Λ(α) is very special. Then for any β ∈ Z_{n}^{r,s}, we have K_{β,α}^{±}(t) ∈ ℤ[t] with deg K_{β,α}^{±} ≤ a(α) − a(β). (The equality holds only when Λ(β) is a special symbol.) In particular, K̃_{β,α}^{±}(t) is a polynomial in ℤ[t].

Proof. Since K is a matrix with entries in ℤ, it follows from the formula (5.5.1), combined with Proposition 6.2 and Proposition 3.12, we see that K_{β,α}^{±}(t) ∈ ℤ[t]. Hence It is enough to estimate the degree of the polynomial K_{β,α}^{±}(t). A similar argument as in [M, III, (6.3)] implies, by making use of (3.7.1) and (6.7.3), the following statement.

(6.8.1) K_{β,α}^{±}(t) is the coefficient of q_{β,∓} in ∏_{ν<ν′} ∏_{b(ν′)=b(ν)±1}(1 − tR_{ν,ν′})^{−1}q_{α,∓}.

We now estimate the degree of K_{β,α}^{±}(t) following the idea in [M, III, 6, Ex. 4]. Let ε_{1}, …, ε_{M} be the standard basis of ℤ^{M}. We denote by R^{+} the set of positive roots of type A_{M−1}, i.e., R^{+} = {ε_{i} − ε_{j} | 1 ≤ i < j ≤ M}. For any ξ = (ξ_{1}, …, ξ_{M}) ∈ ℤ^{M} such that ∑ ξ_{i} = 0, we define a polynomial P (ξ; t) in t by

P (ξ; t) = ∑_{(m_{γ})} t^{∑ m_{γ}},

where (m_{γ}) runs over all the choices such that ξ = ∑_{γ∈R^{+}} m_{γ}γ with m_{γ} ∈ ℤ_{≥0}. We also define P^{∗}(ξ; t) by a similar formula as P (ξ; t), but this time, (m_{γ}) runs over only the expression such that ξ = ∑_{γ} m_{γ}γ and that γ = ε_{i} − ε_{j} corresponds to the raising operator R_{ν_{i},ν_{j}} occurring in the expression in (6.8.1). Then P (ξ; t) is non-zero only when ξ = ∑ η_{i}(ε_{i} − ε_{i+1}) with η_{i} ≥ 0, and in that case, P (ξ; t) is a monic of degree ∑ η_{i} =⟨ξ, δ⟩. (See [loc. cit.]. Here δ = (M, …, 1, 0) ∈ ℤ^{M} and⟨, ⟩denotes the standard inner product in ℤ^{M}.) Clearly deg P^{∗}(ξ; t) ≤ deg P (ξ; t). By our assumption that Λ is very special, the choice (m_{γ}) = (η_{i}) is allowed in the expression of P^{∗}(ξ; t). Hence P^{∗}(ξ; t) is also a monic of degree ⟨ξ, δ⟩.

Now it follows from (6.8.1) that K_{β,α}^{±}(t) is also expressed as the coefficient of s_{β} in

∏_{ν<ν′} ∏_{b(ν′)=b(ν)±1}(1 − tR_{ν,ν′})^{−1}s_{α} = ∏_{ν<ν′} ∏_{b(ν′)=b(ν)±1}(1 + tR_{ν,ν′} + t^{2}R_{ν,ν′} + ⋯)s_{α}.

Hence, by a similar argument as in [loc. cit.], we see that the degree of K_{β,α}^{±}(t) is equal to the degree of

∑_{w∈𝔖_{m}} ε(w)P^{∗}(w^{−1}(β + δ) − (α + δ); t),

where α, β are regarded as elements in ℤ^{M} in an obvious way. Furthermore, if we put Λ = Λ(α), Λ′ = Λ(β), we have

⟨w^{−1}(β + δ) − (α + δ), δ⟩=⟨β + δ, w(δ)⟩−⟨α + δ, δ⟩
≤⟨β + δ, δ⟩−⟨α + δ, δ⟩
=⟨Λ′, δ⟩−⟨Λ, δ⟩,

since Λ = α + δ′, Λ′ = β + δ′ with δ′ = (mr, (m − 1)r + s, (m − 1)r, (m − 2)r + s, …) as elements in ℤ^{M}. The equality holds only when w = 1. Now let Λ′_{1} be the elements in ℤ^{M} obtained from Λ′ by rearranging the entries in decreasing order. Then we have ⟨Λ′, δ⟩≤⟨Λ′_{1}, δ⟩ and so

⟨Λ′, δ⟩−⟨Λ, δ⟩≤⟨Λ′_{1}, δ⟩−⟨Λ, δ⟩
= n(Λ) − n(Λ′_{1})
= a(Λ) − a(Λ′).

Thus we see that deg K_{β,α}^{±} ≤ a(α) − a(β). The equality holds only when Λ′ = Λ′_{1}, i.e., Λ′ is a special symbol. This proves the corollary.

□

7. Examples

7.1. In this section, we give some examples of Green functions, i.e., the matrix P = K̃_{−}(t) for the case where e and n are small. But before it, we shall discuss some general results on the similarity class having the maximum a-value.

We assume that m = (m_{0}, …, m_{e−1}) is such that m_{0} = m + 1, m_{1} = ⋯ = m_{e−1} for some m > 0. For i = 0, …, e − 1, put β_{i} = (β^{(0)}, …, β^{(e−1)}) with β^{(i)} = (1^{n}) and β^{(j)} = (−) for j ≠ i. Note that χ^{β^{i}} = (det_{V})^{i} for i = 1, … e − 1. In particular, χ^{β^{1}} = det_{V} and χ^{β^{e−1}} = \overline{det}_{V}. Also we put Λ_{i} = Λ(β_{i}). Then it is clear that Λ_{1}, …, Λ_{e−1} fall in the same class. More precisely, it is easy to see that Λ_{0} ∼ Λ_{1} if r = s and Λ_{0} ≁ Λ_{1} if r > s. (The latter follows from the fact that the maximal entry in Λ_{0} is equal to mr + 1, while that in Λ_{1} is ≤ mr and we have a(Λ_{0}) < a(Λ_{1}).) By using a similar argument as in Lemma 4,2, one can show that α ∈ Z_{n}^{0,0} such that a(α) is maximum coincides with one of β_{i}. Hence the class F having the maximum a-value is given as follows

F = \begin{cases}\{Λ_{1}, …, Λ_{e−1}\} & if r > s, \\ \{Λ_{0}, Λ_{1}, …, Λ_{e−1}\} & if r = s. \end{cases}

We identify the class F also as the subset of Z_{n}^{0,0} by the natural bijection α → Λ(α).

In the case of Green polynomials of classical groups, (i.e., e = 2, r = 2, s = 1), the family of maximum a-value corresponds to the identity unipotent class under the Springer correspondence, hence consists of one element {Λ_{1}}, where β_{1} corresponds to the sign character of W. Let P = K̃_{−}(t) be the matrix as given in 1.4. Then it is known that the first column {p_{α,β_{1}}} of P gives rise to the fake degrees {R(χ^{α})} of χ^{α}.

The following result, which is due to Malle [Ma2], is the counterpart of the above result to the general setting.

Lemma 7.2. Let the notations be as above. Put b = a(β_{1}). Then we have

(7.2.1)  ∑_{β_{j}∈F} t^{−b}R(χ^{β^{j}})p_{α,β_{j}} = R(χ^{α}).

Moreover R(χ^{β^{j}}) = t^{n(n−1)/2+jn} for j = 0, …, e − 1.

Proof. Let us consider the equation P Λ^{t}P = Ω in (1.4.2). We write the matrices P, Λ, Ω in a block matrix form, where the first block corresponds the family F.

P = (\begin{matrix}t^{b}I & 0 \\ P_{1} & P_{2} \end{matrix}) ,  Λ = (\begin{matrix}D_{1} & 0 \\ 0 & D_{2} \end{matrix}) ,  Ω = (\begin{matrix}Ω_{1} & ^{t}Ω_{2} \\ Ω_{2} & Ω_{3} \end{matrix}) .

Then we have D_{1} = t^{−2b}Ω_{1} and P_{1} = t^{b}Ω_{2}Ω_{1}^{−1}. Let us denote the entries of Ω_{1}^{−1} by (ω̃_{jk}) with j, k such that β_{j}, β_{k} ∈ F. Then we have

p_{α,β_{j}} = t^{b} ∑_{β_{k}∈F} ω_{α,β_{k}} w̃_{kj},

and so

∑_{β_{j}∈F} t^{−b}R(χ^{β^{j}})p_{α,β_{j}} = ∑_{β_{k}∈F} ω_{α,β_{k}} ∑_{β_{j}∈F} R(χ^{β^{j}})w̃_{kj}

= ∑_{β_{k}∈F} ω_{α,β_{k}} ∑_{β_{j}∈F} t^{−N^{∗}} w̃_{kj}ω_{β_{j},β_{1}},

since R(χ^{β^{j}}) = R(χ^{β^{j}} ⊗ χ^{β^{1}} ⊗ \overline{det}_{V}) = t^{−N^{∗}}ω_{β_{j},β_{1}}. Now the last formula is equal to

t^{−N^{∗}} ∑_{β_{k}∈F} ω_{α,β_{k}}δ_{k,1} = t^{−N^{∗}}ω_{α,β_{1}} = R(χ^{α}).

This prove (7.2.1). The last assertion follows easily from the property of coinvariant algebra R of W since deg χ^{β^{j}} = 1. □

7.3. In the remainder of this section, we shall give explicit examples of the matrices P and Λ associated to various W = G(e, 1, n). These examples support the conjecture 5.5. It would be worthwhile to point out that the entries λ_{α,β}(t) of Λ always factorize the polynomial 𝔾(t).

In the following examples, Z_{n}^{0,0} = Z_{n}^{0,0}(m) are written as α_{1}, …, α_{h}, and the corresponding symbols in Z_{n}^{r,s} are written as Λ_{i} = Λ(α_{i}), with m as in 7.1. In the matrix P of Green functions, the rows and columns are arranged along the order α_{1}, α_{2}, . . .. The matrix Λ is given by the collection of Λ(F), where Λ(F) denotes the diagonal block in Λ corresponding the class F.

First we consider the case where W = G(2, 1, 2), i.e., W is the Weyl group of type B_{2} for the sake of reference. Then Z_{n}^{0,0} is given as

α_{1} = (−; 1^{2}), α_{2} = (1^{2}; −), α_{3} = (1; 1), α_{4} = (−; 2), α_{5} = (2; −).

Assume that r = 2 and s = 1. Then the corresponding symbols and similarity classes are

F_{1} = {Λ_{1} = (420; 42)}, F_{2} = {Λ_{2} = (31; 1)},
F_{3} = {Λ_{3} = (30; 2), Λ_{4} = (20; 3)}, F_{4} = {Λ_{5} = (2; −)}.

GREEN FUNCTIONS 35

The matrices (Λ(F_{i})) are given as follows.

Λ(F_{1}) = (1),  Λ(F_{2}) = ((t^{4} − 1)),
Λ(F_{3}) = (t^{4} − 1) (\begin{matrix}t^{2} & t \\ t & t^{2} \end{matrix}) ,  Λ(F_{4}) = (t^{2}(t^{2} − 1)(t^{4} − 1)).

The matrix P of Green functions is given in Table 1.

Table 1. G(2, 1, 2) (r = 2, s = 1) \begin{matrix}(−; 1^{2}) & t^{4} \\ (1^{2}; −) & t^{2} & t^{2} \\ (1; 1) & t^{3} + t & t & t \\ (−; 2) & t^{2} & & & t \\ (2; −) & 1 & 1 & 1 & & 1 \end{matrix}

7.4. Assume that W = G(3, 1, 2). Then Z_{n}^{0,0} is given as α_{1} = (−; 1^{2}; −), α_{2} = (−; −; 1^{2}), α_{3} = (1^{2}; −; −), α_{4} = (−; 1; 1), α_{5} = (1; 1; −), α_{6} = (1; −; 1), α_{7} = (−; 2; −), α_{8} = (−; −; 2), α_{9} = (2; −; −).

Assume that r = 2, s = 1. Then the corresponding symbols and similarity classes are given as

F_{1} = {Λ_{1} = (420; 42; 31), Λ_{2} = (420; 31; 42)},

F_{2} = {Λ_{3} = (31; 1; 1)}, F_{3} = {Λ_{4} = (20; 2; 2)},

F_{4} = {Λ_{5} = (30; 2; 1), Λ_{6} = (30; 1; 2), Λ_{7} = (20; 3; 1)Λ_{8} = (20; 1; 3)},

F_{5} = {Λ_{9} = (2; −; −)}.

The matrices (Λ(F_{i})) are given as follows.

Λ(F_{1}) = (\begin{matrix}t^{2} & t^{4} \\ t^{4} & 1 \end{matrix}) ,  Λ(F_{2}) = (t^{2}(t^{6} − 1)),  Λ(F_{3}) = (t^{2}(t^{6} − 1)),
Λ(F_{4}) = (t^{3} − 1)(t^{6} − 1) (\begin{matrix}t^{2} & t^{3} & 0 & t^{2} \\ t^{3} & 0 & 0 & 0 \\ 0 & 0 & 0 & t^{3} \\ t^{2} & 0 & t^{3} & 0 \end{matrix}) ,  Λ(F_{5}) = (t^{5}(t^{3} − 1)(t^{6} − 1)).

The matrix P of Green functions is given in Table 2.

Table 2. G(3, 1, 2) (r = 2, s = 1) \begin{matrix}(−; 1^{2}; −) & t^{5} \\ (−; −; 1^{2}) & & t^{5} \\ (1^{2}; −; −) & t^{3} & t^{4} & t^{3} \\ (−; 1; 1) & t^{3} & & & t^{3} \\ (1; 1; −) & t^{4} + t & & t & t & t \\ (1; −; 1) & t^{2} & t^{3} & t^{2} & t^{2} & & t \\ (−; 2; −) & t^{2} & & t^{2} & & & & t \\ (−; −; 2) & & t^{2} & t & & & & & t \\ (2; −; −) & 1 & & 1 & 1 & 1 & & & & 1 \end{matrix}

7.5. Assume that W = G(4, 1, 2). Then Z_{n}^{0,0} is given as

α_{1} = (−; 1^{2}; −; −), α_{2} = (−; −; 1^{2}; −; −), α_{3} = (−; −; −; 1^{2}),

α_{4} = (1^{2}; −; −; −), α_{5} = (−; 1; 1; −), α_{6} = (−; 1; −; 1), α_{7} = (−; −; 1; 1), α_{8} = (1; 1; −; −), α_{9} = (1; −; 1; −), α_{10} = (−; 2; −; −), α_{11} = (1; −; −; 1), α_{12} = (−; −; 2; −), α_{13} = (−; −; −; 2), α_{14} = (2; −; −; −).

Assume that r = 2 and s = 1. Then the corresponding symbols and similarity classes are given as

F_{1} = {Λ_{1} = (420; 42; 31; 31), Λ_{2} = (420; 31; 42; 31), Λ_{3} = (420; 31; 31; 42)}, F_{2} = {Λ_{4} = (31; 1; 1; 1)}, F_{3} = {Λ_{5} = (20; 2; 2; 1), Λ_{6} = (20; 2; 1; 2), Λ_{7} = (20; 1; 2; 2)}, F_{4} = {Λ_{8} = (30; 2; 1; 1), Λ_{9} = (30; 1; 2; 1), Λ_{10} = (20; 3; 1; 1),

Λ_{11} = (30; 1; 1; 2), Λ_{12} = (20; 1; 3; 1), Λ_{13} = (20; 1; 1; 3)} F_{5} = {Λ_{14} = (2; −; −; −)}.

The matrices (Λ(F_{i})) are given as follows.

Λ(F_{1}) = (\begin{matrix}t^{4} & t^{6} & t^{8} \\ t^{6} & t^{8} & t^{2} \\ t^{8} & t^{2} & t^{4} \end{matrix}) , Λ(F_{2}) = (t^{4}(t^{8} − 1)), Λ(F_{3}) = (t^{8} − 1) (\begin{matrix}t^{4} & t^{5} & t^{6} \\ t^{5} & t^{6} & t^{3} \\ t^{6} & t^{3} & t^{4} \end{matrix}) ,

Λ(F_{4}) = (t^{4} − 1)(t^{8} − 1) (\begin{matrix}t^{4} & t^{5} & 0 & t^{6} & 0 & t^{5} \\ t^{5} & t^{6} & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & t^{6} & 0 \\ t^{5} & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & t^{6} & 0 \\ t^{5} & 0 & t^{6} & 0 & 0 & 0 \end{matrix}) ,

Λ(F_{5}) = (t^{8}(t^{4} − 1)(t^{8} − 1)).

The matrix P of Green functions is given in Table 3.

Table 3. G(4, 1, 2) (r = 2, s = 1) \begin{matrix}(−; 1^{2}; −; −) & t^{6} \\ (−; −; 1^{2}; −) & & t^{6} \\ (−; −; −; 1^{2}) & & & t^{6} \\ (1^{2}; −; −; −) & t^{4} & & & t^{4} \\ (−; 1; 1; −) & t^{3} & t^{5} & & & t^{3} \\ (−; 1; −; 1) & t^{4} & & t^{4} & & & t^{3} \\ (−; −; 1; 1) & & t^{3} & t^{5} & & & & t^{3} \\ (1; 1; −; −) & t^{5} + t & & & t & t & & & t \\ (1; −; 1; −) & t^{2} & t^{4} & & t^{2} & t^{2} & & & & t \\ (−; 2; −; −) & t^{2} & & & & t^{2} & & & & & t \\ (1; −; −; 1) & t^{3} & & t^{3} & t^{3} & & t^{2} & & & & & t \\ (−; −; 2; −) & & t^{2} & & & & & t^{2} & & & & & t \\ (−; −; −; 2) & & & t^{2} & & & t & & & & & & & t \\ (2; −; −; −) & 1 & & & 1 & 1 & & & 1 & & & & & & 1 \end{matrix}

7.6. Assume that W = G(3, 1, 3). Then Z_{n}^{0,0} is given as

α_{1} = (−; 1^{3}; −), α_{2} = (−; −; 1^{3}), α_{3} = (1^{3}; −; −), α_{4} = (−; 1^{2}; 1), α_{5} = (−; 1; 1^{2}), α_{6} = (1; 1^{2}; −), α_{7} = (−; 21; −), α_{8} = (1; −; 1^{2}), α_{9} = (−; −; 21), α_{10} = (1^{2}; 1; −), α_{11} = (1^{2}; −; 1), α_{12} = (21; −; −), α_{13} = (1; 1; 1), α_{14} = (−; 2; 1), α_{15} = (−; 1; 2), α_{16} = (1; 2; −), α_{17} = (1; −; 2), α_{18} = (2; 1; −), α_{19} = (2; −; 1), α_{20} = (−; 3; −), α_{21} = (−; −; 3), α_{22} = (3; −; −).

Assume that r = 2 and s = 1. Then the corresponding symbols and similarity classes are given as

F_{1} = {Λ_{1} = (6420; 642; 531), Λ_{2} = (6420; 531; 642)}, F_{2} = {Λ_{3} = (531; 31; 31)}, F_{3} = {Λ_{4} = (420; 42; 41), Λ_{5} = (420; 41; 42)}, F_{4} = {Λ_{6} = (520; 42; 31), Λ_{7} = (420; 52; 31), Λ_{8} = (520; 31; 42), Λ_{9} = (420; 31; 52)}, F_{5} = {Λ_{10} = (31; 2; 1), Λ_{11} = (31; 1; 2)}, F_{6} = {Λ_{12} = (41; 1; 1)}, F_{7} = {Λ_{13} = (30; 2; 2), Λ_{14} = (20; 3; 2), Λ_{15} = (20; 2; 3)}, F_{8} = {Λ_{16} = (30; 3; 1), Λ_{17} = (30; 1; 3)}, F_{9} = {Λ_{18} = 40; 2; 1), Λ_{19} = (40; 1; 2), Λ_{20} = (20; 4; 1), Λ_{21} = (20; 1; 4)}, F_{10} = {Λ_{22} = (3; −; −)}.

38 T. SHOJI

The matrices (Λ(F_{i})) are given as follows.

Λ(F_{1}) = (\begin{matrix}t^{3} & t^{6} \\ t^{6} & 1 \end{matrix}) , Λ(F_{2}) = (t^{3}(t^{9} − 1)), Λ(F_{3}) = (t^{3} + 1)(t^{9} − 1) (\begin{matrix}t^{3} & t^{4} \\ t^{4} & t^{2} \end{matrix}) ,
Λ(F_{4}) = (t^{6} − 1)(t^{9} − 1) (\begin{matrix}t^{3} & 0 & t^{5} & t^{4} \\ 0 & 0 & 0 & t^{5} \\ t^{5} & 0 & 0 & 0 \\ t^{4} & t^{5} & 0 & 0 \end{matrix}) , Λ(F_{5}) = (t^{6} − 1)(t^{9} − 1) (\begin{matrix}t^{6} & t^{7} \\ t^{7} & t^{5} \end{matrix}) ,
Λ(F_{6}) = ((t^{9}(t^{6} − 1)(t^{9} − 1)), Λ(F_{7}) = (t^{6} − 1)(t^{9} − 1) (\begin{matrix}t^{9} & t^{7} & t^{8} \\ t^{7} & t^{8} & t^{9} \\ t^{8} & t^{9} & t^{7} \end{matrix}) ,
Λ(F_{8}) = (t^{3} − 1)(t^{6} − 1)(t^{9} − 1) (\begin{matrix}0 & t^{8} \\ t^{8} & 0 \end{matrix}) ,
Λ(F_{9}) = (t^{3} − 1)(t^{6} − 1)(t^{9} − 1) (\begin{matrix}t^{9} & t^{10} & 0 & 0 \\ t^{10} & 0 & 0 & 0 \\ 0 & 0 & 0 & t^{10} \\ 0 & 0 & t^{10} & 0 \end{matrix}) ,
Λ(F_{10}) = t^{12}(t^{3} − 1)(t^{6} − 1)(t^{9} − 1).

The matrix P of Green functions is given in Table 4.

Table 4. G(3, 1, 3) (r = 2, s = 1) ; \begin{matrix}(−; 1^{3}; −) & t^{12} \\ (−; −; 1^{3}) & & t^{12} \\ (1^{3}; −; −) & t^{9} & & t^{9} \\ (−; 1^{2}; 1) & t^{10} + t^{7} & t^{10} & & t^{7} \\ (−; 1; 1^{2}) & t^{8} & t^{11} + t^{8} & & & t^{7} \\ (1; 1^{2}; −) & t^{11} + t^{8} + t^{5} & & t^{5} & t^{5} & & t^{5} \\ (−; 21; −) & t^{9} + t^{6} & & & t^{6} & & & t^{5} \\ (1; −; 1^{2}) & t^{7} & t^{10} + t^{7} & t^{7} & & t^{6} & & & t^{5} \\ (−; −; 21) & & t^{9} + t^{6} & & & t^{5} & & & & t^{5} \\ (1^{2}; 1; −) & t^{10} + t^{7} + t^{4} & & t^{7} + t^{4} & t^{4} & & t^{4} & & & & t^{4} \\ (1^{2}; −; 1) & t^{8} + t^{5} & t^{8} & t^{8} + t^{5} & t^{5} & & & & t^{3} & & & t^{4} \\ (21; −; −) & t^{6} + t^{3} & & t^{6} + t^{3} & t^{3} & & t^{3} & & & & t^{3} & & t^{3} \\ (1; 1; 1) & t^{9} + 2t^{6} + t^{3} & t^{9} + t^{6} & t^{6} + t^{3} & t^{6} + t^{3} & t^{5} & t^{3} & & t^{4} & & t^{3} & & & t^{3} \\ (−; 2; 1) & t^{7} + t^{4} & t^{7} & & t^{4} & t^{6} & & t^{3} & & & & & & & t^{3} \\ (−; 1; 2) & t^{5} & t^{8} + t^{5} & & t^{5} & t^{4} & & & & t^{4} & & & & & & t^{3} \\ (1; 2; −) & t^{8} + t^{5} + t^{2} & & t^{2} & t^{5} + t^{2} & & t^{2} & t^{4} & & & t^{2} & & & t^{2} & & & t^{2} \\ (1; −; 2) & t^{4} & t^{7} + t^{4} & t^{4} & t^{4} & t^{3} & & & t^{2} & t^{3} & & t^{3} & & & & t^{2} & & t^{2} \\ (2; 1; −) & t^{7} + t^{4} + t & & t^{4} + t & t^{4} + t & & t^{4} + t & & & & t & & t & t & & & t & & t \\ (2; −; 1) & t^{5} + t^{2} & t^{5} & t^{5} + t^{2} & t^{2} & t^{4} & t^{2} & & t^{3} & & t^{2} & & t^{2} & t^{2} & & & & & & t \\ (−; 3; −) & t^{3} & & & t^{3} & & & t^{2} & & & & & & & t^{2} & & & & & & t \\ (−; −; 3) & & t^{3} & & & t^{2} & & & & t^{2} & & & & & & t & & t & & & & t \\ (3; −; −) & 1 & & 1 & 1 & & 1 & & & & 1 & & 1 & 1 & & & 1 & & 1 & & & & 1 \end{matrix}

References

[BMM] M. Broué, G. Malle and J. Michel, Generic blocks of finite reductive groups, Astérisque 212 (1993), 7–92.

[DL] P. Deligne and G. Lusztig; Representations of reductive groups over finite fields, Ann. of Math. 103 (1976), 103–161.

[G] J.A. Green; The characters of the finite general linear groups, Trans. Amer. Math. Soc. 80 (1955), 402–447.

[GM] M. Geck and G. Malle; On special pieces in the unipotent variety, Experimental Math. 8 (1999).

[L1] G. Lusztig; Irreducible representations of finite classical groups, Invent. Math. 43 (1977), 125–175

[L2] G. Lusztig; Intersection cohomology complexes on a reductive group, Invent. Math. 75 (1984), 205–272.

[L3] G. Lusztig; Character sheaves V, Advances in Math. 61 (1986), 103–155.

[L4] G. Lusztig; On the character values of finite Chevalley groups at unipotent elements, J. Algebra, 104 (1986), 146–194.

[L5] G. Lusztig; Green functions and character sheaves, Annals of Mathematics, 131 (1990), 355–408.

[L6] G. Lusztig; Notes on unipotent classes, Asian J. Math. 1 (1997), 194–207.

[LS] G. Lusztig and N. Spaltenstein; On the generalized Springer correspondence for classical groups, in “Advanced Studies in Pure Math.” – Algebraic groups and related topics –, 6, 1985, pp. 289–316.

[M] I.G. Macdonald; Symmetric functions and Hall Polynomials, second edition. Clarendon Press. Oxford 1995.

[Ma1] G. Malle; Unipotent Grade imprimitiver komplexer Spiegelungsgruppen, J. Algebra, 177 (1995), 768–826.

[Ma2] G. Malle; Private communication, 1998.

[S1] T. Shoji; On the Green polynomials of classical groups. Inventiones Math., 74 (1983), 239 - 267.

[S2] T. Shoji; Character sheaves and almost characters of reductive groups, II, Adv. in Math. 111 (1995), 314 - 354.

[S3] T. Shoji; Green functions and a conjecture of Geck and Malle, Beitrag̈e zur Algebra und Geometrie 41, (2000), 115–140.

[S4] T. Shoji; A Frobenius formula for the characters of Ariki-Koike algebras, J. Algebra 226, (2000), 818–856.