4-Homogeneous groups

KANTOR, W. M.

Math. Zeitschr. 103, 67/68 (1968)

4-HomogeneousGroups* WILLIAM M. KANTOR Received April 19, 1967 A k,homogeneous group is a permutation group which acts transitively on the set of k element subsets of the set of permuted points. LIVINGSTONE and WAGNER [4, Theorem 2(b)] have shown that a k-homogeneous group of finite degree > 2 k is k-transitive if k > 5. We shall prove the following Theorem. If G is 4-homogeneous but not 4-transitive on a finite set O, 1s >5, then G is similar to one of the following groups in its usual permutation representation: PSL(2, 5), PGL(2, 5), PSL(2, 23), PFL(2, 23), or PFL(2, 25).

Proof. Let I~l =n. For the cases n = 6 or 7, see [1, pp.215-217]. We now assume that n > 8, in which case G is 3-transitive ([4, Theorem 2(a)]). Let A={e,/3, ~, 6} be an arbitrary set of 4 points of O. We first show that the global stabilizer Ga of A is transitive on A. There is a set S of at least 4 points such that Gz acts on Z as a 3-transitive group in which only the identity fixes 4 points ([4, Theorem 3, Corollary]). From the result in [3] it follows that each such group has a subgroup of order 4 having an orbit of length 4. Thus, G has a subgroup having an orbit o f length 4, and the claim follows from the 4-homogeneity of G. Since G is not 4-transitive, the restriction GAlA of Ga to A has order 4 or 12. G is 3-primitive, that is, G~p is primitive on ~-{~,r For assume that G~p splits f~-{cq/3} into a system of (n-2)/b blocks, each block having b > 1 points ([6, p. 12]). The images of this system under G form (2) block systems, one for each unordered pair of points of O. The number of ways of choosing an ordered pair {e',/3'}, {V', 6'} of pairs of distinct points of f2 so that V' and ~' lie in the same block of G~,~, is

(2) "(n-2)b-l'(b2) =n(n-1)(n-2)(b-1)/4" On the other hand, if c~',/8', ~' and ~' satisfy the above condition then so does

eachofthe (4) kimagesunderGoftheorderedpair{c~',~'},{7',6'}.Here k is the number of images of {c~',~'} under GA,, where A' ={e',/3', ~', 6'}; * Research supported by the National Aeronautics and Space Administration. 5*

68

W.M. KANTOR: 4-Homogeneous Groups

k = 2 or 6 according to whether I Ga'la,[ = 4 or 12. It follows that 2(:)

n(n-1)(n-2)(b-1)/4.

Together with the conditions 1 < b [(n - 2) and n > 8 this implies that b = n - 2, as claimed. If G~p had an involution interchanging ~ and 6 then Gala would contain an odd permutation, whereas GA]a is not $4. Thus, G~a has odd order and hence is solvable by the Feit-Thompson Theorem [2]. Since G~a is primitive on f 2 - { e , fl}, G,a has a characteristic elementary abelian subgroup N of order n - 2 which is transitive and regular on f 2 - { e , fl} ([6, Theorem 11.5]). G has an involution i interchanging c~ and fl and normalizing N. Since n is odd, i fixes exactly one point and hence centralizes no non-trivial element of N. Thus, i inverts N ([1, p.90]). In particular, distinct involutions having a common cycle of length 2 can have neither a second common cycle of length 2 nor a common fixed point. G, has precisely n - 2 involutions. Suppose that i = ( ~ fl)(7 6) .... The transitivity of GA[Aimplies that there is an element of the form j = ( ~ V)(fl 6) .... Since j2 has odd order there is even an involution i' =(a 7)(fl 6) .... As i is the only involution of the form (a fi)(7 6) ..., i'ii'=(~ fi)(~ 6) . . . . i and i and i' generate the only four group (that is, non-cyclic group of order 4) fixing A. Thus, each set of 4 points of ~2 is fixed by a unique four group, and each four group in G is uniquely determined by any of its ( n - 1 ) / 4 non-trivial orbits. Since every pair of distinct commuting involutions generates a four group, and each four group contains 3 such pairs, the total number of pairs of distinct commuting involutions of G is

3 (n4) { ( n - 1 ) / 4 } - l = n ( n - 2 ) ( n - 3 ) / 2 . As commuting involutions have the same fixed point, G~ contains ( n - 2 ) ( n - 3 ) / 2 pairs of distinct commuting involutions. The n - 2 involutions of G, thus generate an elementary abelian 2-subgroup of G, of order n - 1 , so that n - 2 is prime (see [5, p.334], for example). G,p[o_~,,~ is then a solvable, transitive permutation group of prime degree. Thus, G~p r ~ = 1 and the Theorem follows from [3] and [4, p.403]. References 1. BURNSIDE,W.: Theory of groups of finite order. Cambridge: University Press 1911.

2. FELT,W., and J.G. THOMPSON:Solvability of groups of odd order. Pacific J. Math. 13, 775-- 1029 (1963). 3. GORENSTmN,D., and D.R. HUGH~S:Triply transitive groups in which only the identity fixes four letters. Ill. J. Math. 5, 486--491 (1961). 4. LIVINGSTONE,D., and A. WAGNER:Transitivity of finite permutation groups on unordered sets. Math. Zeitschr. 90, 393--403 (1965). 5. SIERPINSKI,W.: Elementary theory of numbers. Warsaw: Monografje Matematyczne 1964. 6. WIELANDT,H. : Finite permutation groups. New York: Academic Press 1964.

Department of Mathematics, Universityof Wisconsin,Madison, Wis. 53706, U.S.A.