School Leadership Effects Revisited: Review and Meta-Analysis of Empirical Studies (SpringerBriefs in Education)

SpringerBriefs in Education For further volumes: http://www.springer.com/series/8914 Jaap Scheerens Editor School L...

Author: Jaap Scheerens

8 downloads 621 Views 1MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

SpringerBriefs in Education

For further volumes: http://www.springer.com/series/8914

Jaap Scheerens Editor

School Leadership Effects Revisited Review and Meta-Analysis of Empirical Studies

123

Prof. Dr. Jaap Scheerens University of Twente Drienerlolaan 5 7522 NB Enschede The Netherlands e-mail: [email protected]

ISSN 2211-1921 ISBN 978-94-007-2767-0 DOI 10.1007/978-94-007-2768-7

e-ISSN 2211-193X e-ISBN 978-94-007-2768-7

Springer Dordrecht Heidelberg London New York Library of Congress Control Number: 2011940816 Ó The Author(s) 2012 No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Cover design: eStudio Calamar, Berlin/Figueres Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

School leaders are expected to play a pivotal role in educational systems, in which state policies regarding decentralization and accountability provide new challenges. The same applies to expectations about innovation, the improvement of quality in education and the finding of solutions for problems that arise from important changes in the profiles and background of students entering the school. But schools are a particular kind of organization as far as leadership roles are concerned. In this report the development of altering concepts of school leadership over a period of about 4 decades is sketched. This development started out with instructional leadership as an apparently strong break with the limited role of leadership in schools, seen as professional bureaucracies. But gradually, leadership thinking evolved to the recognition that school leadership can be devolved over staff and other organizational ‘‘substitutes for leadership’’. Individual, hierarchical leadership seems to have almost disappeared from the scene in some recent studies of leadership effectiveness. The study goes on in an attempt to clarify the theoretical background of these developments, arriving at a proposal to think of a ‘‘lean’’ form of school leadership that is comparable to the concept of metacontrol. The bulk of the study is dedicated to an analysis of the empirical research literature on leadership effects. This includes the presentation of results from an earlier meta-analysis carried out by the authors, a summary of other meta-analyses and a new meta-analysis based upon 25 studies carried out between 2005 and 2010. Interestingly the older reviews and meta-analyses were predominantly based on so-called direct effect studies, while the majority of more recent studies looked at indirect effects of leadership, mediated by other school variables. The report makes up the balance about the importance of the, on average, relatively small total effect of leadership on student outcomes and identifies promising intermediary factors which, stimulated by specific leadership behaviours, impact on student performance. In the final chapter implications for educational practice and policy are sketched under the headings: ‘‘schools need leadership’’, ‘‘the toolkit of the school leaders as a meta-controller’’, ‘‘the special case of turning around failing schools’’ and ‘‘efficiency of school leadership’’. In passing several suggestions are given about interesting next steps in school leadership effects research. v

Acknowledgments

The research published in this book was supported by a grant from the Knowledge Directorate of the Dutch Ministry of Education. The organizational and administrative part of this project was carried out by Mrs. Carola Groeneweg.

vii

Contents

1

Conceptual Perspectives on School Leadership . . . . . . . . . . . . . . . Meta Krüger and Jaap Scheerens

1

2

Earlier Meta-Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jaap Scheerens and Rien Steen

31

3

Anatomy of Some Representative School Leaders’ Effectiveness Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jaap Scheerens

45

Results from School Leadership Effectiveness Studies (2005–2010) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maria Hendriks and Rien Steen

65

Summary and Conclusion: Instructional Leadership in Schools as Loosely Coupled Organizations . . . . . . . . . . . . . . . . . . . . . . . . Jaap Scheerens

131

4

5

ix

Chapter 1

Conceptual Perspectives on School Leadership Meta Krüger and Jaap Scheerens

Leadership has become a concept of increasing importance in the education literature. Stacks of books and articles have been written about leadership—about how to define the concept, what it should comprise and what effects it has. Despite the many researchers and the many definitions of leadership that appear in the literature, there remains very little consensus concerning what leadership is and what it comprises. In an attempt to map the field of leadership studies, Ribbins and Gunter (2002) compared and contrasted the concepts of administration, management and leadership, demonstrating the magnitude of the confusion. Administration was sometimes understood to consist of three successive processes: vision, planning and policy. The term administration thus subsumed management and leadership. In contrast, others have viewed leadership as an overarching concept; leadership is understood to affect policy, values and vision. In an overview of leadership theories, Richmon and Allison (2003) argue that the search for an unambiguous definition of leadership is in vain, as it simply does not exist. Krüger (2010) agrees, stating that leadership may be conceived as a process of influence, as a process of leading and following, as a matter of personality, as a way of

Part of this chapter is a slightly re-edited version of a chapter by Meta Krüger and Bob Witziers , titled ‘‘Developments in Thinking about Instructional leadership’’, in Scheerens and Witziers (2005), pp. 19–38. M. Krüger (&) Department of Social and Behavioural Sciences, University of Amsterdam, P.O. Box 94208, 1090 GE Amsterdam, The Netherlands e-mail: [email protected] J. Scheerens Department of Educational Organization and Management, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands e-mail: [email protected]

J. Scheerens (ed.), School Leadership Effects Revisited, SpringerBriefs in Education, DOI: 10.1007/978-94-007-2768-7_1, The Author(s) 2012

1

2

M. Krüger and J. Scheerens

persuasion, as a manner of interacting, as a process of goal attainment, as a way of creating structure, as negotiation in power relations and as stimulating change. To provide a detailed summary of the literature in this chapter would be impossible considering the stack of literature. Hence, we try to examine and discuss the most important approaches and concepts from the literature in a more global manner. To this end, we have divided the literature into theories emphasising personality traits, behaviours and actions, and leadership styles. Thereafter, we discuss the literature on situational leadership, roles and competences. This will be followed by a discussion of direct and indirect effect models of leadership used in leadership research, in an attempt to develop a model of leadership based on the theories discussed before. In the following section we discuss the concepts of distributed leadership and substitutes for leadership. In the concluding section we zoom in on instructional, transformational and integral leadership in order to summarize and discuss the literature.

Personality Traits of Leaders The earliest research into leadership focused primarily on the personality traits of leaders. Theorists in the 1920s wondered why some individuals were more capable of exercising leadership than were others. For example, Bingham (1927) defined a leader as a person having the greatest number of desirable personality or character traits (cited in Bass and Stogdill 1990). Leaders were seen as individuals who possessed qualities that distinguished them from their followers. Authority, diligence and charisma were among the first traits to be considered. Because these theories were based chiefly on men and not on women, they are sometimes known as ‘great man theories’. As Bass and Stogdill (1990, p. 37) observed, ‘Despite the examples of Joan Arc, Elizabeth I and Catherine the Great, great women were ignored’. Researchers attempted to identify the specific characteristics that these leaders possessed. Bass and Stogdill provide an overview of the numerous studies that are directed toward the personality traits of leaders. Taken together, studies prior to 1950 tend to argue that leaders differ on six points: • Constitution (intelligence, alertness, verbal skills, originality and judgement) • Achievement (intellect, knowledge and athletic skills) • Responsibility (reliability, initiative, persistence, aggression, self confidence and the desire to excel) • Participation (activity, sociability, cooperation, adaptability and sense of humour) • Status (socio-economic position and popularity) • Situation (mental level, status, skills, needs and interests of followers, goals to be achieved, etc.).

1 Conceptual Perspectives on School Leadership

3

Also of interest are personal characteristics that apparently have no influence on leadership. These factors include age, height, weight, body structure, energy, appearance, dominance and control over moods. In a follow-up study, Stogdill compared the results of similar studies written between 1950 and 1970 with earlier findings. While these studies confirmed earlier research, they also combined personality and situational factors. One practical application of this research can be seen in the rise of assessment centers since the mid-1970s. In assessment centers, potential leaders are observed for several days, interviewed and tested in order to ascertain their capacity for leadership. According to Bass and Stogdill (ibid), results of these studies characterized leaders as having strong feelings of responsibility, the drive to complete tasks, energy and persistence in pursuing goals, willingness to take risks, originality in solving problems, initiative, self- confidence and a sense of personal identity, willingness to bear the consequences of their decisions and actions, ability to overcome interpersonal stress, tolerance for frustration and delays, ability to influence the behaviour of others and the capacity to structure social interaction toward the goals that are to be pursued. As stated above, none of the early leadership studies addressed the influence of gender on leadership. It was not until the 1970s that this variable began to be included in international research on school supervision. Initially, much of this research was still primarily oriented toward the careers of men and women and paid little or no attention to differences in actual supervision. In 1990, Eagly and Johnson presented a meta-analysis concerning gender and the leadership styles of managers in general, followed 2 years later by a study concerning gender and the leadership style of principals (Eagly et al. 1992). The conclusion from both studies is that gender differences in leadership style do exist. As far as leadership is concerned, therefore, gender does matter. This result would be confirmed in later research, as the following section will show. An overview of the literature concerning personality and leadership (Judge et al. 2002) organises the literature according to the ‘Five Factor Model’, also known as ‘the Big Five’. Extraversion appeared to have the strongest connection to leadership. In addition, there appeared to be a relationship between leadership and neuroticism, openness, accommodation and conscientiousness. In short, all five of the key factors in personality research appear to be associated with leadership. Leaders are thus different from their followers. In a more recent contribution to this field of study, Leithwood et al. (2006) mention general intelligence, problem solving skills, self-confidence, emotional stability, extraversion, and internal locus of control, openness to experience, conscientiousness and self-efficacy as relevant personality traits for school leaders. The question of how much these personality characteristics contribute to the development of leadership and its effectiveness has persisted since the advent of leadership studies, as has the question of whether personality characteristics really have a greater impact than situational factors. In short, the personality approach appears to provide a rather deterministic view of leadership.

4


Leadership Behaviour and Actions A key assumption of the personality approach is that leadership depends primarily on the question of who leaders are and not on what they do. Closely related to this point is research that examines the behaviour of school principals in relation to a number of variables, including success, motivation and work satisfaction. Such research has provided a foundation for studies that attempt to identify exactly what it is that effective principals do, primarily through observation and the subsequent interpretation of the observations. Although the ultimate goal of these studies is obviously to improve education, the behaviour and performance of principals formed the focus of the research. School effectiveness researchers also maintain a primary focus on the tasks and actions of principals in their search for the characteristics of effective principals. Such research, which takes the form of behavioural observations, provides insight into the profusion of tasks and activities in the everyday work of a principal. Various typologies of the tasks of school principals have been derived from such research. Sergiovanni (1984) provides an early example of these typologies. He describes five distinct behavioural dimensions: technical management activities, interpersonal support and encouragement of personnel, educational intervention, serving as a role model for important goals and behaviours (symbolic leadership) and developing an appropriate and unique school culture. Each dimension involves a series of tasks. This approach distinguishes primarily among tasks that are related to the primary process (e.g., instruction) and tasks that are directed toward secondary processes (e.g., the continued operation of the organization and the shaping of preconditions for providing education). In writing about the dual role of the school principal, Hughes (1985) observes that the principal is simultaneously a ‘chief executive’ (administrative leader) and a ‘leading professional’ (educational leader). Such task typologies have played an important role in research concerning the amount of time principals spend on various tasks. Information gathered from self-reports, diaries and direct observations has made it clear that the work of principals is complex and that their work agendas are fragmented into a variety of tasks, each of which involves a considerable number of differing subtasks. An important result that emerges from Mintzberg’s (1980) research is that managers feel strongly attracted to what he describes as the ‘verbal media’. Managers spend at least 78% of their time on verbal interaction. Mintzberg’s portrait of the manager corresponds closely with the image of the busy principal, such as the description that appears in Cuban’s analysis of studies since the beginning of the twentieth century regarding the work agendas of school directors. Principals must constantly move from task to task, react to questions as they arise, take whatever action is necessary at a given moment and spend between two-thirds and three-fourths of their time talking with and listening to others (Cuban 1988). According to Cuban, principals spend most of their time on administrative tasks; instructional leadership clearly takes second place. This observation has been confirmed in a number of recent studies. Leithwood et al. (1990) report that,


5

in practice, principals are primarily involved with administrative tasks, even though they tend to list the development of activities directed toward actual instructional decisions among their favourite tasks. Such results have also been found in many studies in the Netherlands (e.g., Stoel 1995). Research results show that principals spend their time mainly on keeping the organization running, through which they do little to developing and carrying out an educational vision. The differences that they report themselves between their actual and preferred work agendas suggest that they are unsatisfied with this state of affairs. The result applies strongly to male than to female school directors. In light of these results, it is not surprising that many researchers have emphasised the importance of instructional leadership, assuming that learning results will improve if principals are able to spend more time on tasks that are directed toward the primary processes. School effectiveness research is particularly relevant in this regard. This type of research has led not only to the further conceptualisation of the concept of instructional leadership (Hallinger 1983), but also to a series of studies focusing on the central question of which behaviours are demonstrated by the principals of schools whose achievements exceed expectations. These results suggest that a number of leadership characteristics are important for school effectiveness, including the establishment of high expectations for students and their learning achievements, emphasis on basic skills, involvement with instructional methods, coordination of instructional programmes, evaluation of student progress, provision of support and guidance for teachers and the creation of an orderly and learning-oriented climate (Van Vilsteren 1999). Meta-analyses of research investigating the relation between instructional leadership and learning achievement, however, puts such results sharply into perspective (Witziers et al. 2003). Yet these results did not apparently lead to a decline in the belief in the capacity of principals for improving the school organisation, and thus the concept of transformational leadership arose in the 1990s (Fullan 1992; Leithwood 1992; Sergiovanni 1990). A transformational leader has the task of developing the school organisation by bringing about a cultural shift within the school. The idea behind the transformational approach is that a culture of autonomous, isolated professionals remains dominant in many schools. This impedes the introduction of necessary educational innovation and improvement (Geijsel et al. 1999; Leithwood and Jantzi 2006; Nguni et al. 2006). The cultural shift therefore involves the notion that schools must grow toward a culture of collegiality, shared planning and a continuous effort to bring about improvement. This means that principals have a role in the creation of a working environment in which teachers work together and identify strongly with the mission of the school. Means for achieving such an environment include the delegation of tasks, expansion of teacher qualifications (‘empowerment’) and the (shared) development of a vision for the direction in which the school must develop. The transformational leader steers the behaviors of the staff by initiating a vision for the future, by inspiring, motivating, giving individual support and by setting intellectual challenges (Geijsel 2003). In addition, the importance of

6


creating a learning school organisation by transforming teachers into learning teachers must be emphasized (Geijsel et al. 2009). Transformational leadership is often contrasted with transactional leadership. Whereas transformational leadership primarily seeks to increase the intrinsic motivation of the teachers, in transactional leadership, teachers are stimulated primarily to change on the basis of external rewards. Staff members receive a reward for achievements, whereas undesirable behavior is corrected by negative sanctions. Leadership is based on intrinsic motivation.

Leadership Styles In addition to an orientation toward personal characteristics and the tasks and activities of principals, the concept of leadership style has also received considerable attention. Leadership style can be described as the consistent line that can be recognised in a leader (De Jong and Van Doorne-Huiskes 1992). A leader does not consciously choose a leadership style; it is related to such factors as the leader’s personality and his or her dominant pattern of values. The origin of research into leadership styles can be traced to the beginning of the 1960s. The Ohio State Leadership Studies developed a concept of leadership based on two dimensions. The first dimension (task orientation) involves the achievement of organisational goals. Task orientation emphasises the creation of an organisational structure and channels of communication, the establishment of work processes, the crystallization of procedures and similar activities. All of this takes place with the goal of making the organisation function in a manner that is as goal-directed and as effective as possible. The second dimension (relationship orientation or ‘consideration’) seeks to increase the goodwill and morale of the members of the organisation. Concrete examples of such behaviours include showing warmth and respect, helping and giving the appearance of trust. These dimensions are considered independent of each other. Crossing the two dimensions yields four quadrants, each representing a different style of leadership: (A) high on both dimensions, (B) high on relation orientation and low on task orientation, (C) low on relation orientation and high on task orientation and (D) low on both dimensions. Thinking in terms of these two style dimensions has also been influential in research specifically involving school principals. Such studies usually involve the use of the Leader Behaviour Description Questionnaire (LBDQ). With this instrument, teachers are asked about their perceptions concerning the leadership style of their principals, and results obtained with this instrument have been discussed in detail in various publications. Leithwood et al. (1990) trace the various studies of principals back to four leadership styles in which the two dimensions mentioned above are clearly present. Their description of these styles provides a good overview of the various ways in which principals can fulfil their tasks. Style A is characterised by an emphasis on personal relationships, on bringing about cooperation among teachers and the


7

pursuit of good working relationships with various regional and governmental agencies. Principals who adhere to this style assume that such relations form a necessary pre-condition for more task-oriented activities in the school. Style B leaders direct themselves toward the learning achievements and goodwill of their students. They use a series of means to reach their goals in this regard, including many of the interpersonal, administrative and governance behaviours that are characteristic of the other styles. Style C is less consistent than the previous two. The primary goal of principals with this style includes ensuring the availability of effective instructional programmes, raising the skills of teachers and developing procedures for fulfilling tasks that contribute to a successful instructional programme. In comparison to Style A, Style C leaders are primarily task-directed, and the development of good interpersonal relations is seen as a means to achieving better performance. Compared with Style B, Style C principals have a tendency to adopt and implement apparently effective procedures with the goal of increasing learning achievement, rather than focusing on learning performance itself. Style D is characterised by a nearly exclusive focus on administrative matters. Budgeting, schedules, personnel management and the distribution of information are the tasks that get the most attention, while little time is spent on educational matters. Only in crisis situations do these matters receive the attention of Style D principals. Leithwood et al. (ibid) report that a high degree of both task and relationship orientation raise the satisfaction and performance of the teachers and increase their willingness to support the principal’s decisions. McPherson et al. (1986, p. 227) also report that the combination of task-oriented and relationship-oriented styles has a positive effect on the work satisfaction of teachers. Despite their predilection for working independently and professional autonomy, teachers actually prefer principals who set clear guidelines and work together with them in matters having to do with instruction—at least to a point. Beyond this turning point, teachers prefer to work under the umbrella of a taskoriented leadership style, without the direct, personal interference of the principal. An approach developed by Blake, Mouton and Williams (1981) is closely related to the Ohio State Studies. In their book, The Managerial Grid, they present a typology of leadership styles based on two dimensions: concern for organisational functioning and the achievement of goals (‘performance concerned’) and concern for employees (‘people concerned’). The task and relationship-oriented styles are also recognisable in this typology, as is the assumption that effective leaders tend to integrate both styles to a considerable degree. Krüger (1994) also made use of the Ohio State research to investigate the relationship and task orientation of school principals. She examined the extent of relationship and task orientation in the supervisory (e.g., instruction, guidance, motivation, delegation) and conflict management (competition, adaptation, cooperation, avoidance) styles of school directors. The leadership style differences that were identified apparently have more to do with the school culture than with the gender of the school director. For example, only in continuing education programs are female directors both more task-oriented and relationship-oriented than their male colleagues. Men in secondary and vocational education appear more likely

8


than women to compete in cases of conflict, and women are more prone to avoidance. It is interesting to note that, in general, female teachers are more positive than male teachers about the leadership styles of women, while male teachers are more positive than women about the leadership styles of men. The other aspect of leadership style that is frequently investigated is the degree to which leaders demonstrate democratic (allowing subordinates to participate in decision making), or autocratic (discouraging the participation of subordinates in decision making) behaviour. The dimension of democratic or autocratic leadership (i.e., participatory or directive leadership) emerges from early experimental studies in leadership style (e.g., Lewin and Lippitt 1938) and has been developed by a number of later researchers (e.g., Likert 1961; Vroom and Yetton 1973). In contrast to the relationship-oriented and task-oriented styles, this approach involves a bipolar dimension of decision-making style. The two styles are mutually exclusive. The distinction between democratic and autocratic leadership is sometimes linked to the distinction between relationship and task orientation. This approach contrasts the democratic or participative, relationship-oriented leader with the autocratic or directive, task-oriented leader. Eagly and Johnson (1990) specify this link further: relationship-oriented leadership should facilitate or encourage the democratic or participatory style of decision making. Studies by Eagly and Johnson (1990) and Eagly et al. (1992) conclude that women tend to provide more democratic and less autocratic leadership than men. Research in the Netherlands by Krüger (1994, 1996) does not show this effect; most principals adhere to a democratic style, and there is no difference between the decision-making styles of men and women.

Situational Leadership, Roles and Competencies Situational Leadership Research into personal characteristics, research into behaviours and actions and research into leadership styles have been subjects of frequent criticism. The most important problem is that researchers merely assume that certain personality traits make for a certain effective leadership style, independent of the type of organisation and the circumstances in which a leader must operate. The implausibility of this assumption can be illustrated by considering cultural differences. For example, in countries in which the power distance between individuals at various hierarchical layers is relatively small, typical ‘bossy’ behaviour will have hardly any positive effect (Hofstede 1997). This example shows that leadership style must match the situation in order to generate effective leadership. As early as the 1920s, some theorists were making the same claims about the personality traits of leaders; the situation is of at least as much importance as personality traits in explaining the origins of leadership. Leadership is not


9

anchored in the person; circumstances make the leader. The personal–situational theories arose from this assertion. The foundation for these theories is the assumption that personal talents must match the situation if leadership is to emerge (Bass and Stogdill 1990). Although these ideas had existed for some time, the major breakthrough did not take place until the 1970s. In this period, Fiedler (1967) developed ‘contingency theory’. At its base, this theory relates to the distinction made in the Ohio State Studies, in that it distinguishes between taskoriented and socio-emotionally oriented leadership styles. Rather than assuming that one of the two (or a combination) of the styles is most effective, however, contingency theory proposes that the context determines which leadership style will be best. Three important context factors emerge from this perspective. The first factor involves the relationship that exists between leader and subordinate, which primarily involves mutual trust between the two parties. The second factor is task structure, which primarily involves the question of whether and to what extent a leader knows what must be done in order to maximise performance. The last factor has to do with the means available to a leader for administering rewards and sanctions. Fiedler proposes that the taskoriented leadership style is most appropriate for situations involving considerable trust, a variety of means for administering rewards and sanctions and ample opportunity for the leader to direct the work of subordinates, as well as for situations that are exactly opposite. The socio-emotionally oriented style is most appropriate for situations falling between these two extremes. An interesting question is, which of Fiedler’s three leadership factors would predict situational theory to be most appropriate for educational organisations. Teachers enjoy a high degree of autonomy within the field of education. Van Vilsteren (1999) argues that this phenomenon hinders the development of good working relationships between leaders and teachers, as the interference of the principal into educational matters can quickly generate mistrust among teachers. With regard to Fiedler’s context factor of task structure, education arguably involves a ‘weak’ technology. How it leads to particular results is not known, obviously making it difficult for the principal to intervene. Fiedler’s third and final factor involves the question of whether principals have means for administering rewards and punishments. Such does indeed appear to be the case. For example, principals can offer such rewards as increased instructional hours, time for further training and compliments. Examples of punishments include assigning additional tasks (or taking away other tasks) or scheduling teachers for certain hours against their will. In the Netherlands, as in many other countries, however, the legal position of teachers is such that few means are actually available for forcing a teacher to go along with certain decisions. According to Van Vilsteren (1999), a task-oriented style of leadership is most advisable for this context. Some evidence that this proposition is correct can be found in the school improvement literature. Studies by Fullan (1991) and by Leithwood and Montgomery (1986) reach the conclusion that task-oriented principals are the most effective for initiating and implementing change. Fiedler

10


ultimately proposes that leadership style is a stable characteristic. That also implies that the best option for principals who are thus mismatched with their school organisations is to abandon ship. After all, they are not capable of changing their leadership styles. Vroom and Yetton (1973) propose instead that leaders can indeed adapt their behaviour to suit the situation at hand, and that they are thus capable of operating more effectively. For Vroom and Yetton, the only important issue is decision making, another aspect of leadership style that has been the subject of much of the research we have described. In addition, the perception that democratic leadership is the most effective appears frequently. In light of their situational starting point, Vroom and Yetton argue that effective leaders know when, how and to what degree subordinates must participate in decision making. Depending upon the demands of the situation, this implies that leaders must take action in some cases while in other cases allowing subordinates to have an important role in the decision-making process. Van Vilsteren (1999) describes the implications of this theory for school organisations by distinguishing between the domains of instruction and administration. In the instructional domain, power rests with the teachers, and ‘ill-structured’ problems (problems in which the root cause is often difficult to determine) occur frequently in this area. The administrative domain belongs almost exclusively to the responsibility of the principal, and problems tend to be of a more routine nature. From this, we can see that different situations require principals to contend with different problems. Van Vilsteren therefore argues that principals are well advised to adopt a participative style in the instructional domain. In the administrative domain, principals must seek their refuge in a more autocratic style. Many other theories offer variations on the theme described above. Other essential matters addressed by these theories primarily involve conditions that have an influence on leadership style. In addition to workplace characteristics (e.g., the extent to which tasks are structured), House (1971) distinguishes several employee characteristics that are important in determining the desired style of leadership. Examples of such characteristics include employees’ needs, skills and feelings of self-worth. House derives four styles of leadership, which are described briefly below. Directive style A style that is necessary whenever employees are uncertain, have little experience (or both) with the tasks that they must fulfil and when the work environment is not formalised to any great extent. Under such circumstances, leaders must make clear to the employees what is expected of them and must pay considerable attention to planning, organising and monitoring their work efforts. Supportive style This style consists of giving support to employees, considering their needs and taking care to provide a pleasant working climate. This leadership style is


11

especially important when employees lack self-confidence and when tasks are fairly routine. Participative style This style is necessary when employees are capable of carrying out their tasks independently and when tasks are loosely structured. The most important characteristics of this style are cooperation and the use of ideas and suggestions from the employees as a means of reaching group decisions. Performance-oriented style The most important characteristics of this style are the high standards that employees must meet and the continual active pursuit of increased work performance. This style of leadership is most appropriate for situations involving unstructured tasks. House’s typology implies that leadership style must match the skills of the employees. This implication also appears in the literature concerning self-managing teams, which assumes as well that leaders must adapt to the situations in which their teams operate. In addition, teams are assumed to develop along two lines. In the social dimension of team development, the development process is set into motion with the goal of increasing cohesion within and commitment to the team. The task dimension involves development that ultimately results in a situation in which team members are utilised optimally, the team can make decisions on its own and is able to resolve conflicts on its own, and the team is capable of taking the initiative to raise the level of its performance. Leaders of such teams must therefore use a different leadership style in each phase of team development, and must adopt a style of leadership that ultimately supports the team in its development toward self-sufficiency. The initial phase of team development calls for a traditional, directive style of leadership. This style is then gradually replaced by a style that is more reminiscent of coaching. The role of the leader is primarily that of a facilitator, directing only where necessary and giving credit to the team for its achievements. Team leadership amounts to a sort of advisory function, in which the leader stands beside the team, so to speak. It must further be noted that the type of leadership does not always depend completely on the team’s phase of development. Team composition may also be of influence. The assumption that the culture of the organisation must match the type of leadership also appears in the work of Handy (1981). He refers back to Greek mythology in order to illustrate certain leadership styles. For example, a high degree of creativity and problem-solving ability are characteristic of Athenaleadership, in which influence is exercised primarily on the basis of expertise and the ability to define problems in a different way. Such leadership is most appropriate for organisations that are undergoing a process of innovation and for organisations that require a high degree of expertise. The Dionysus type of leadership is most appropriate for schools, which are loosely coupled organisations in which professionals work primarily independently and individually. For this type of leadership, the most important way of

12


exercising influence involves working together with professionals to reach consensus concerning the details of how tasks are to be accomplished.

Roles A final example of leadership seen as contingent on the organizational culture is the integrated leadership model of Quinn, Cameron and others. These authors assume that leaders must match the culture of their organisations and emphasise the roles of leaders from this perspective. Quinn and Cameron (1983), assert that effective leadership depends on the life phase of the organisation and its attendant values within the organisation. It is thus conceivable that the value orientations that prevail in the beginning phases of a business differ from those of an organisation that has been successfully established in the market for a number of years. For example, consider the case of new departments. A creative and innovation-oriented pattern of values dominates in such departments, and the enthusiasm of the employees and their willingness to contribute to the mission are the lifeblood of the organisation. Once a department has established itself, attention for innovation and creativity fades, turning instead toward formalisation over the course of time. Only when compelled by external developments to undergo a process of innovation in order to survive (for example in the case of decreasing numbers of students or new consumer markets) does a pattern of values re-emerge that is directed toward change and adaptation to the changing reality. Different organisational life stages and cultures call for different types of leaders. The various patterns of values suggest the existence of various cultures within organisations. Faerman and Quinn (1985) describe four different types of culture. Actions within the rational culture are directed primarily toward achieving the mission of the organisation, and productivity and efficiency are important values. In the hierarchical culture, rules and procedures are of eminent importance. Working according to the rules is an important value in this culture. The chief goal in the consensus culture is to optimise interpersonal relationships. Connection, cohesion, morale and similar values play an important role in the actions of organisational members. Finally, the development culture is directed toward momentum, growth and expansion. Values that stimulate these aspects play an important role. This perspective also implies that each culture calls for a different type of leadership. Instead of styles, Quinn et al. (1996) adopt the concept of management roles, of which two fit within each culture at the same time (see Fig. 1.1). For example, the roles of director and producer fit within the rational culture. The role of the director involves carrying out a vision, providing direction, stimulating actions and providing people with the opportunity to realise the leading vision by providing them with the means and the space necessary to achieving the mission. It is also important in such organisational cultures that work be carried out according to demands and goals. The role of the producer therefore also involves the accomplishment of tasks that are compatible with optimising production.


13

Rational model

Internal model

Human relations model

Open system model

Criteria for effectiveness

Productivity

Stability, continuity

Cohesion, commitment

Change and adaptation, external legitimacy

Mean-ends theory

Clear goals and direction lead to more productivity

Establishing routines leads to stability

Commitment leads to hard work

Adaptation and innovation lead to acquiring (external) resources needed for survival

Emphasis on

Goal clarification, rational analysis, Strong, directive leadership

Rules & procedures Clarifying responsibilities, measurement, documentation

Participation, communication, cooperation, reaching consensus, solving conflicts

Adaptation to (political) environment, creative solutions, management of change

Climate

Rational economy, final results

Hierarchical

Teamwork

Innovative, flexible

Role of manager

Director and producer

Controller, coordinator

Mentor, stimulator

Innovator, mediator

Fig. 1.1 The Quinn model

The roles of the coordinator and controller are compatible with an organisational culture that emphasises rules and procedures. The coordinator determines what will happen and when, and planning, organising, giving feedback and similar skills are of eminent importance. The role of the controller is an extension of this, albeit at some distance from the primary process. The controller’s role primarily involves the management of information. The mentor role and the stimulator role are important for a culture that is directed toward consensus. The former involves such skills as coaching, effective communication and increasing personal insight, while the latter involves skills directed toward stimulating cooperation. In more concrete terms, skills contribute to team cohesion (‘team building’), conflict management and increasing the problem-solving ability of teams. Finally, a development-oriented culture calls for the roles of innovator and mediator. The innovator role is the most obvious, as development necessarily implies change and adaptation. The innovator must be capable of generating enthusiasm and support for the changes at hand. The role of the mediator involves mediation between the internal and external environments. For example, one of the tasks is to create and secure the availability of means for giving shape to changes. Another task is to make sure that all internal parties are reading from the same page.

14


Whether principals must manage all of these roles is unclear. On the one hand, there must arguably be a match between culture and the roles that a (school) leader must fulfil. This also implies that things can go wrong when the two are not in balance. For example, a principal focused on optimising school performance, working in a school that is directed toward educational innovation, would have to shift the school culture in his or her direction or bear the consequences of this friction and flee. It can nevertheless be argued that all roles be fulfilled in order to safeguard the effectiveness of the (school) organisation. This standpoint thus approaches the propositions offered by Leithwood and Montgomery (1986) that principals are especially effective when they work in an integrative way. They must not only be good instructional leaders, but must also possess a high level of interpersonal and administrative skills. This does not mean, however, that one person must fulfil all of these roles. It is also possible for a management team, as a collective, to fulfil these roles.

Competencies Quinn and colleagues make a transition from roles to competencies by proposing that certain skills are necessary for each role. A leader must therefore be capable of taking the initiative, setting goals, delegating effectively and similar skills in order to fulfill the director role effectively. A skill implies the presence of knowledge and the capacity for adequate action. This concept can also be found in the competency approach. A competency can be described as a person’s capacity to connect knowledge, skills, attitudes and professional identity relevant for a certain professional situation to personality characteristics and to deploy these in an integrated way to enable adequate acting in specific professional situations (Krüger 2009, p. 120). The emphasis on acting and accomplishing tasks does not mean that skills are the only important factors. On the contrary, the concept of competency refers to an integrated package of knowledge, skills, attitudes and personal characteristics. A successful principal must possess a variety of competencies in order to be able to conduct particular ‘successful’ transactions. Where the transactions are essential for determining whether a principal is indeed effective—in other words, a principal’s competence is apparent in observable behaviour (Aitken 2001; Bollington 1999). The setting of competencies occurs by first determining the roles out of which the competencies flow. ‘Key roles’ must then be identified and subsequently broken down into smaller subunits (‘key units’). The behavioural criteria or actions (‘performance criteria’) that are suitable indicators are subsequently investigated in order to determine whether a principal is effective. One example of this approach can be found in the competency model developed in Australia (Australian Principal Association 2000). In this approach, one of the key roles principals must fulfil is to relate to teachers effectively; they must be capable of communicating effectively with the staff, developing the teaching staff professionally, delegating tasks and responsibilities, handling conflict and showing


15

respect for the personnel. These elements are subsequently translated into specific and observable behavioural criteria. Another approach stems from the Netherlands. A professional standard for school leaders has been developed lately. This standard consists of five general competences: vision orientation; context awareness; deployment of strategies that match new forms of leadership; organization awareness; and higher order thinking (Krüger 2009). In recent days, there is an appeal for inquiry-based working in schools, internationally. The main characteristic of the inquiry-based school is making use of data for school improvement and enhancing student outcomes. In inquirybased schools, the inquiry habit of mind of school leaders, teachers and students is highly valued (Earl and Fullan 2003; Geijsel et al. 2010; Krüger 2010). School leaders and teachers are searching for research evidence in order to make decisions at school and classroom level, and research is conducted by small groups of teachers, school leaders and students. Inquiry-based schools search for external data that indicate ‘what works and what does not work’ in educational innovation: they work evidence based. Next, internal research data are gathered that give more insight into matters such as effective teaching, learning strategies and student achievements. Finally, the inquiry-based school leaders utilize data for decision making in education policy and school policy. The demand laid on school leaders is that they have the ability to stimulate teachers’ inquiry habit of mind, to motivate teachers in doing research and in working with data while innovating curricula and didactics. This approach requires higher order thinking and an inquiry- based orientation from school leaders.

Direct and Indirect Effect Models of School Leadership Within the context of empirical school effectiveness research, direct effects of instructional leadership on student achievement in basic studies are either not found or are present only in certain national contexts; when they are present, they are relatively small (cf. Hallinger and Heck 1996a, b). Concerning the Netherlands, Van de Grift and Houtveen (1999) established that, instructional leadership in primary schools, as measured according to teacher perceptions, had no effect in 1989, while significant positive effects were found in 1993 and 1998. Witziers and Bosker (in Scheerens and Bosker 1997) reported positive effects for the USA, but not for other countries. An overall meta-analysis by Witziers et al. (2003) reveals a small positive effect, which essentially amounts to a slight correlation (\0.10) between instructional leadership and student achievement. Considering the indirect ways in which instructional leadership affects student achievement makes sense conceptually. This assertion has been expressed in a number of conceptual models, including one developed by Bossert et al. (1982), which is shown in Fig. 1.2.

16

M. Krüger and J. Scheerens Context

School leader in action

Instruments

Student outcomes

School environment Educational (pedagogical) climate

(School population, parents’ demands etc.)

Learning results School leader’s vision and experience

School leader behaviour School organisation

Legal context

Fig. 1.2 Conceptual model of instructional leadership, from Bossert et al. (1982)

Fig. 1.3 Mulford’s path-diagram of the impact of transformational leadership

The model by Bossert and others distinguishes two intermediary factors: pedagogical climate and school organisation. A more recent modelling effort, which


17

was also empirically verified by Mulford (2003), proposes teacher leadership and organisational learning as intermediary variables (see Fig. 1.3). Hallinger and Heck (1998) conclude that studies that consider context factors and school characteristics in investigating the effectiveness of the leadership in a school yield more positive results regarding the influence of the principal on learning outcomes. Witziers et al. (2003) only found five studies investigating the indirect effects of instructional leadership on student achievement. Taking such intermediary variables as distributive leadership and organisational learning into account is consistent with a gradual broadening of the concept of leadership that is used. For example, Mulford (ibid) explicitly considers transformational leadership. Figure 1.4 provides an overview of the subsequent stages in broadening the scope of school management concepts.

Instructional leadership

Curriculum and instruction

Extended instructional leadership

School mission Managing the curriculum Providing learning climate

Transformational leadership

Models organisational values Develops shared mission Provides intellectual stimulation Builds consensus Redesigns organizational structure

Integrated leadership

Conditions supporting school improvement Instructional leadership

Competing values model

Productivity Stability, continuity Cohesion, commitment Adaptation

Fig. 1.4 Concepts of leadership at school

Figure 1.5 combines various aspects of instructional leadership and the leadership style factors that were discussed earlier with the state-of-the-art indicators of school and instructional effectiveness (Scheerens 2008). The part in bold in the second column represents the core of instructional leadership. The categories included in the third column of Fig. 1.5 include the four core practices of successful school leadership, as distinguished by Leithwood et al. (2006): setting directions, developing people, managing the teaching and learning program, and redesigning the organization. The third column of Fig. 1.5 shows a concept of leadership that is considerably broader than instructional leadership; it largely coincides with the Quinn framework.

18 Relevant personality traits


Competencies

Extraversion social appraisal skills

context awareness

intelligence motivation internal locus of control domain specific knowledge conscientiousness

vision orientation

Extraversion Social appraisal skills Self confidence

deployment of strategies that match new forms of leadership

Basic human values General moral beliefs Role responsibility

deployment of strategies that match new forms of

Leadership style

Leadership behaviour

Effectiveness enhancing factors

External contacts

Enhanced teaching time

Buffering

Task-related

Direction setting (vision, goals, standards Monitors curriculum and instruction (managing the instructional program) Redesigning the organization

Clear goals and standards Opportunity to learn Student monitoring & feedback Structured teaching Active teaching Active learning

Person-related

HRM & HRD Coaches teachers Recruits teachers Builds consensus Individual support Intellectual stimulation

Cohesion among teachers Professionalization Teacher competency Teachers’ sense of self efficacy

Sets values Creates climate

Shared sense of purpose among teachers

organization awareness

leadership

High expectations Disciplinary climate Supportive climate

Higher order thinking

Fig. 1.5 Intermediary causal structure of leadership at school

Distributed Leadership and Substitutes for Leadership Some authors, e.g., Hulpia (2009, p. 4) connect hierarchical leadership to instructional leadership, while transformational leadership is being associated with bottom-up decision making. This participative, bottom up, collaborative practice has, in the education context, been referred to as ‘‘distributed leadership’’. Other related terms are teacher leadership, shared leadership and democratic leadership. Youngs (2009) distinguishes ‘shared’ leadership and


19

‘distributed’ leadership. According to Youngs, shared leadership is a form of distributed leadership. Shared leadership is a construct positioned at group level, whereas distributed leadership takes place at organizational level. Both ‘‘planned’’ and ‘‘emergent’’ interpretations are given to this distributed leadership. In the case of planned distribution, the concept is close to delegation. When authors speak of a more dynamic, interactive or emergent participation, any kind of hierarchical or focused leadership disappears to the background. A critical comment with the latter interpretation is the statement that when everyone leads, nobody leads (Leithwood et al. 2006). A further conceptual distinction is whether the distribution is limited to personnel with a leadership function, such as deputy or coordinator, or is also extended to personnel without a formal leadership role (e.g., teacher leadership). In a more recent contribution by Heck and Hallinger (2009) leadership is not only distributed to other personnel but, figuratively speaking, to organizational structures as well: Dynamic theories of organizational processes seek to describe how changes in organizational structures (e.g., size, hierarchy, staffing) and social–cultural interactions (e.g., organizational culture, decision-making structures, leadership, social networks) influence organizational outcomes over a period time (Langlois and Robertson 1993; Nonaka and Toyama 2002; Ogawa and Bossert 1995; Williams and Podsakoff 1989; Heck and Hallinger 2009, p. 14)’’. Ogawa and Bossert (1995), cited by Heck and Hallinger (2009, ibid) say that: ‘‘If leadership is treated as an organizational quality, then studies of leadership must have as their unit of analysis the organization.

School organizational improvement is seen as an interactive process over time, in which (distributed) leadership is just one of the interacting elements, next to personal composition, relations with stakeholders and organizational climate. This view on school leadership brings us close to a concept that was proposed much earlier: the concept of ‘‘substitutes for leadership’’. The conceptions of leadership presented so far assume leaders to have both practical and formal means for guiding their subordinates appropriately. The validity of this assumption is uncertain. Some researchers put this into perspective, arguing that, while the role of the leader is not unimportant in an organisation, it must also not be overestimated. The potential influence of leaders on organisational processes and their outcomes are subject to neutralisation by other actors or processes within the organisation, implying that leadership is completely unnecessary in some situations. Kerr and Jermier (1978) offer an example. They propose that there is actually no need for leadership when employees know what must be done and how it must be done. In this case, goals and ‘know–how’ serve as substitutes for leadership. They argue further that there are a number of conditions within an organisation that can neutralise or even work against the influence of a leader. These substitution or neutralising conditions involve three aspects: characteristics of individuals, characteristics of the work situation and characteristics of the organisation. Figure 1.6 provides an overview of what each of these characteristics involves.

20 Fig. 1.6 Substitution or neutralising conditions for the exercise of leadership

M. Krüger and J. Scheerens Characteristics of individuals Experience, educational level and competence Professional orientation Indifference toward rewards and punishments Characteristics of the work situation Structure and routinisation of tasks Feedback to workers inherent in task completion Work satisfaction provided by tasks Characteristics of the organisation Formalisation Inflexibility of rules and procedures Specialisation of staff functions Possibility of isolated leadership function Task cohesion Capacity of leader to distribute rewards

Kerr and Jermier propose that these conditions have various substitution or neutralising effects on leadership. For example, a high degree of employee competence neutralises the necessity of task-oriented leadership behaviour. An intrinsically motivating task has the same effect on relationship-oriented leadership. Further, some conditions render leadership completely unnecessary. For example, the existence of cohesive task groups renders both task-oriented and relationship-oriented leadership unnecessary. The ideas of Kerr and Jermier have also been applied to educational research (Pitner 1986). Pitner concludes that ‘strong’ subject departments can serve as substitutes for the leadership of principals in secondary schools. Another conclusion is that the ‘potential’ influence of the head of a subject department is rendered largely harmless when teachers appeal to their professional orientation and refer to their professional competencies.

Zooming in on Instructional, Transformational and Integral Leadership Instructional Leadership: The Narrow Approach, Focussing on Instruction and Curriculum A sharp increase in attention for instructional leadership can be found in the literature of the late 1970s. The first push for an increase in the interest of school leaders in the teaching and learning processes of their schools arose from research and development activities associated with the Effective Schools movement. British and North American researchers observed that some primary schools were achieving better learning results than were others. These differences in student


21

outcomes could not sufficiently be explained by the individual and social background characteristics of the students. One of the differences they found between more effective and less effective schools concerned the attitudes and the activities of school leaders. According to these researchers, the leaders of effective schools: • Are directly involved in their school’s education and with students; • Repeatedly evaluate the classroom work of their teachers, as well as the learning progress and results of the students; • Promote an orderly working environment and a school climate in which every child is expected to be able to learn. These conclusions led to further research into this topic, in which researchers focused primarily on what school leaders do and how their actions relate to student achievement. Moreover, researchers made a sharp distinction between instructional leadership and administrative leadership. Reviews summarising these studies indicate that effective school leaders are characterised by the engaging in the following activities: • • • • • • •

Promoting an orderly and stimulating work climate Emphasising basic skills Performing student monitoring Co-operating with teachers on curricular and instructional issues Encouraging and rewarding teachers Supervising and controlling teachers Advancing the skills, expertise and professionalism of teachers

In summary, the picture of the instructional leader that emerged from early school effectiveness research consisted mostly of tasks related to curriculum and instruction. The appearance of such concepts as ‘curricular leadership’ (Glatthorn 1987, 1997) and ‘instructional leadership’ (Hallinger 1983), clearly reflect this image. The narrow view of instructional leadership is concentrated on only one of the four core leadership practices, mentioned by Leithwood et al. (2006), namely managing the instructional program.

Extended Instructional Leadership: Emphasising Structural and Cultural Change In addition to these relatively narrow conceptualisations of instructional leadership, other approaches sought to broaden the scope of the concept gradually. In these approaches, actions and strategies other than those closely related to the primary process of teaching and learning come into view as well. The archetypal example of this approach is the work of Hallinger (1983), who asserts that instructional leadership is related to defining a mission for the school, managing curriculum and instruction and promoting a learning climate favourable

22


for student learning. More specifically, this approach to instructional leadership entails the following ten dimensions: A. Defining the school mission 1. Setting school goals 2. Communicating school goals B. 1. 2. 3. 4.

Managing the curriculum Evaluating and monitoring curriculum and instruction Guiding teachers Co-ordinating the curriculum Promoting collaboration and consensus among teachers

C. 1. 2. 3. 4.

Promoting a task-oriented learning climate Visibility Rewarding teachers Promoting professional development Promoting an orderly and task-oriented climate

The first category (defining the school mission) stresses the need for school principals to develop school tactical and strategic goals. These goals should provide direction to teachers’ work in their classrooms and function as guidelines for decision making. It also conveys the notion that these goals should be communicated to all relevant stakeholders within the school. The second category (managing the curriculum) obviously refers to the need for school leaders to manage the curriculum. More specifically, it entails activities related to evaluating and monitoring curriculum and instruction through such activities as observing teachers in their classrooms, monitoring student progress and giving teachers suggestions on how they might improve their work. In addition to these activities, this category also entails activities related to the co-ordination and guidance of teachers. Co-ordination implies activities that ensure consistency and cohesion in a school’s educational programme, either informally or formally (e.g., by playing an important role in creating school development plans). Finally, leaders should encourage collaboration among teachers through both structural and cultural changes in the organisation. The third category (‘promoting a climate conducive for learning’) consists of activities that are thought to enhance student learning more directly. It entails specific activities related to the creation of an orderly, safe and task-oriented school culture (e.g., by developing clear rules for student behaviour), rewarding well-functioning teachers (either materially or otherwise), promoting and initiating teachers’ professional development and being visible. Visibility is understood primarily in terms of the accessibility of principals to their teachers and students.


23

Transformational Leadership Another aspect of Hallinger’s approach (the idea that the school leader can have an impact on student achievement through developing the school’s organisation and culture) is reflected in the concept of transformational leadership. This concept emphasises that one of the main tasks of school leaders is to initiate processes and structures within the school that enable teacher collaboration and participative decision making. The concept is fuelled by the notion that, in many schools, teachers are autonomous and isolated, implying that school leaders should not intervene directly with curricular and instructional affairs, but rather indirectly by transforming the school culture to facilitate collegial planning, collaboration and experimentation aimed at school improvement. In other words, the main tasks of the school leader should be to create a working environment in which teachers collaborate and identify themselves strongly with the school’s mission. Figure 1.7 provides an overview of the characteristics assigned to transformational leadership.

Leithwood, Jantzi & Steinbach (1999)

Bass & Avolio (1993)

Holds high expectations

Intellectual stimulation

Provides intellectual stimulation

Individualised consideration

Models organizational values

Inspirational motivation

Provides individual support

Idealised influence

Builds collaborative culture Strengthens productive school culture Develops shared vision Creates structure for participation in decision-making Builds consensus about school goals

Fig. 1.7 Aspects of transformational leadership

According to Leithwood and colleagues, transformational leadership achieves superior results by operating in keeping with ‘the four I’s’: 1. Idealised influence (being a role model for followers) 2. Inspirational motivation (motivating and inspiring followers by providing meaning and challenge to their work) 3. Intellectual stimulation (stimulating follower’s efforts to be innovative and creative) 4. Individualised consideration (paying attention to each person’s needs for achievement and growth)

24


This conceptualisation builds upon the work of Bass and Avolio, although it also resembles the concept of instructional leadership as developed by Hallinger. An important difference, however, lies in the style of leadership that each approach promotes. Instructional leadership has always been criticised by scholars because of its supposed emphasis on a rather autocratic style of leadership in schools. The concept of transformational leadership, on the other hand, is associated with explicitly promoting such phenomena as democratic leadership, leaders as coaches, and teacher participation in decision making and distributed leadership. This last concept, shared or distributed leadership, denotes that leadership in a school organisation should not depend solely upon the role or position of the school principal; others within the school organisation are also able to exercise leadership. This kind of leadership extends throughout the school organisation through carefully designed structures that permit co-ordinated action (Ogawa and Bossert 1995; Pounder et al. 1995). In summing up, the broader view on instructional leadership has ‘‘direction setting’’ and ‘‘promoting a productive teaching and learning climate’’ in common with transformational leadership. Transformational leadership is more explicitly focused on organisation and people development. Whereas instructional leadership has a bias towards the primary process of teaching and learning, transformational leadership is more oriented on secondary processes that are aimed at improving organisational structures, organisational culture and organisational processes. Structural forms of teacher cooperation are a central focus for transformational leadership; shared or distributed leadership can be seen as an off spring of this central orientation of transformational leadership. Instructional leadership models point towards the role of leadership in the development of curriculum, teaching and learning (Hallinger 2003; Southworth 2002). In contrast, transformational leadership models focus on the means by which leadership builds the school’s broader capacity for change and learning (Cuban 1990; Fullan 2001, 2002, 2006; Leithwood 1994; Mulford and Bishop 1997; Mulford and Silins 2003; Stoll and Fink 1996); Heck and Hallinger 2009, p. 8). Recent efforts to reconcile these approaches to school improvement leadership have begun to emphasize the similarities of the models more than their differences (Hallinger 2003, 2005; Leithwood et al. 2004). Researchers have documented ways in which school leaders develop conditions that support teacher learning, school change, and improvement in learning. This research highlights leadership that develops a shared vision for the school, supports teacher development, builds a collaborative culture, creates an academic press, and obtains broad involvement in decision-making (Barth 1990; Blase 1993; Hallinger 2003, 2005; Jackson 2000; Kleine-Kracht 1993; Leithwood 1994; Leithwood et al. 2004; Marks and Printy 2003; Louis and Marks 1998; Mulford and Silins 2003) (ibid).

Integral Leadership In the concept of ‘‘integral leadership’’ the overlap between extended instructional and transformational leadership is recognised. The rise of this concept is sometimes related to expansions in scale and the increased autonomy of schools.


25

The basic assumption of integral leadership is that distinguishing between instructional leadership and administrative leadership is not very effective, primarily because it leads to fragmentation and segmentation. For this reason, school leaders should integrate all domains (e.g., education, personnel, finance) within their schools, mostly by developing school-wide strategic perspectives that integrate them all. A proponent of this approach, Leithwood (1992) stresses the importance of an integral orientation on the part of school leaders to the development of schools as efficient and effective organisations. Leithwood states that school leadership should be considered from an integral, school-wide perspective, which should form the foundation for questions concerning what to do to improve the school and how to do it. Imants (1996) also emphasises the importance of a strategic perspective, arguing on the basis of a literature review that the ways in which school leaders view their own roles, the functioning of their schools and the schools’ (future) goals, as well as of their capacity for communicating this vision to other stakeholders are crucial conditions for leading the school effectively. In this view, vision is related to an integral view of a school’s goals and the elaboration of the norms and objectives guiding school improvement. In other words, this perspective apparently argues that administrative leadership and instructional leadership can be distinguished in theory, but not in practice. The integral perspective also appears in the approach to leadership developed by Quinn and Cameron (1983) as discussed earlier in this chapter. The core of this approach is that managers fulfil different roles according to their values, norms, goals and task orientations. As we saw, Quinn (ibid) distinguishes among four different models in which managers have different values and goals, fulfil other roles and, consequently, steer their organisation differently. One important element in this approach is that these models are competitive; the overall conceptual framework is described as the conflicting values model. In other words, managers tend to prefer one model (and consequently focus on particular values and roles) over all other models. However, organisational effectiveness benefits from the integration of these different models. In other words, effective managers are managers who are able to integrate all models into their thinking and acting. Finally, the approach developed by Marks and Printy (2003) is also important in this context. These authors argue that transformational leadership is of great importance for school reform, but that the practices related to this type of leadership do not lead to improvements in student outcomes, as they lack a clear focus on teaching and learning. They are therefore insufficient to ensure high-quality teaching and learning. In this respect, Marks and Printy call for integrated leadership. School leaders should be both transformational and instructional leaders. Transformational leadership provides the conditions that support school improvement, and instructional leadership attends to the issues that actually matter in improving student achievement. In summing up the ‘‘conceptual space’’ of school leadership is depicted in Fig. 1.8 as a set of partly overlapping circles.

26


Fig. 1.8 Decomposition of school leadership concepts

A D

B E

C

A stands for transformational leadership, D stands for the broader perception of instructional leadership encompassing direction setting, developing a task-oriented climate and managing the teaching and learning program; B represents the intersection of transformational leadership and instructional leadership, encompassing direction setting and developing a task-oriented culture; C is instructional leadership in the narrow sense (managing the teaching and learning program); E stands for distributed leadership as a subset of transformational leadership. The union of A and D (transformational leadership and instructional leadership) is integrated leadership. Given the considerable overlap between the two main concepts: extended instructional leadership and transformational leadership, two practices will be followed in presenting the results of meta-analyses and reviews in the subsequent chapters: they will either be taken together indiscriminately under the heading of ‘‘school leadership’’; or they will be differentiated under the subheadings: instructional leadership, transformational leadership and integrated leadership. Studies that are specifically labelled as addressing distributed leadership will be subsumed under transformational leadership.

References Aitken, J. (2001). Core competencies for School Principals. http://www.ero.govt.nz/Publication/ eers1995/95no6hl.htm. Australian Principal Association. (2000). Leadership Framework. http://www.apcentre.edu.au/ aboutapc/leadfram.htm. Barth, R. S. (1990). Improving Schools from Within. San Francisco: Jossey-Bass. Bass, B. M., & Avolio, B. J. (1993). Transformational leadership and organizational culture. Public Administration Quarterly, 17(1), 112–121. Bass, B. M., & Stogdill, R. M. (1990). Bass & Stogdill’s handbook of leadership (3rd ed.). New York/London: The Free Press/Collier MacMillan. Bingham, W. V. (1927). Leadership. In H. C. Metcalf (Ed.), The psychological foundations of management. New York: Shaw. Blake, R. R., Mouton, J. S., & Wiliams, M. S. (1981). The academic managerial grid. San Francisco: Jossey-Bass Publishers.


27

Blase, J. (1993). The micropolitics of effective school-based leadership: Teachers’perspectives. Educational Administration Quarterly, 29(2), 142–163. Bollington, R. (1999). Leadership. In M. Brundrett (Ed.), Principles of school leadership. Peter Francis: The Old School House, Norfolk. Bossert, S. T., Dwyer, D. C., Rowan, R., & Lee, G. V. (1982). The instructional management role of the principal. Educational Administration Quarterly, 18(3), 34–64. Cuban, L. (1988). The managerial imperative and the practice of leadership in schools. Albany: State University of New York Press. Cuban, L. (1990). Reforming again, and again, and again. Educational Researcher, 19(1), 3–13. De Jong, A. M., & van Doorne-Huiskes, J. (1992). Leiderschapsstijl: verschillen tussen mannen en vrouwen? In M. I. Demenint & C. E. Disselen (Eds.), Vrouwen, leiderschap en management (pp. 23–36). Lemma BV: Utrecht. Eagly, A. H., & Johnson, B. T. (1990). Gender and leadership style: A meta-analysis. Psychological Bulletin, 108(2), 233–256. Eagly, A. H., Karau, S. J., & Johnson, B. T. (1992). Gender and leadership style among school principals: A meta-analysis. Educational Administration Quarterly, 28(1), 76–102. Earl, L., & Fullan, M. (2003). Using data in leadership for learning. Cambridge Journal of Education, 33(3), 383–394. Faerman, S. R., & Quinn, R. E. (1985). Effectiveness: the perspective from organization theory. Review of Higher Education, 9, 83–100. Fiedler, F. E. (1967). A theory of leadership effectiveness. New York: McGraw-Hill Fullan, M. (1991). The new meaning of educational change. London: Cassell. Fullan, M. (1992). Visions that blind. Instructional Leadership, 49(5), 19–20. Fullan, M. (2001). Leading in a culture of change. San Francisco: Jossey-Bass. Fullan, M. (2002, December). Leadership and sustainability. Principal Leadership, 3, 14–17. Fullan, M. (2006). Turnaround leadership. San Francisco: Jossey-Bass. Geijsel, F. P., Krüger, M. L., & Sleegers, P. J. C. (2010). Data feedback for school improvement: the role of researchers and school leaders. Australian Educational Researcher, 37(2), 59–75. Geijsel, F., Sleegers, P., Leithwood, K., & Jantzi, D. (2003). Transformational leadership effects on teachers’ commitment and effort toward school reform. Journal of Educational Administration, 41(3), 228–256. Geijsel, F. P., Sleegers, P. J. C., Stoel, R. D., & Krüger, M. L. (2009). The effect of teacher psychological, school organizational and leadership factors on teachers’ professional learning in Dutch schools. The Elementary School Journal, 109(4), 406–427. Geijsel, F. P., Sleegers, P. J. C., & Van den Berg, R. M. (1999). Transformational leadership and the implementation of large-scale innovation programs. Journal of Educational Administration, 37(4), 309–328. Glatthorn, A. A. (1987). Curriculum Leadership. London: Glenview. Glatthorn, A. A. (1997). The principal as curriculum leader. Thousand Oaks: Corwin Press. Hallinger, P. (1983). Assessing the instructional management behavior of principals. Unpublished Doctoral Dissertation, Stanford University, Stanford, CA. Hallinger, P. (2003). Leading educational change. Reflections on the practice of instructional and instructional leadership. Cambridge Journal of Education, 33(3), 329–351. Hallinger, P. (2005). Instructional leadership and the school principal: A passing fancy that refuses to fade away. Leadership and Policy in Schools, 4, 1–20. Hallinger, P., & Heck, R. H. (1996a). Reassessing the principals role in school effectiveness: A review of empirical research, 1980–1995. Educational Administration Quarterly, 32, 5–44. Hallinger, P., & Heck, R. H. (1996b). The principal’s role in school effectiveness: an assessment of methodological progress, 1980–1995. In K. Leithwood, et al. (Eds.), International Handbook of Instructional leadership and Administration. Dordrecht: Kluwer. Hallinger, P., & Heck, R. H. (1998). Exploring the principal’s contribution to school effectiveness: 1980–1995. School Effectiveness and School Improvement, 9(2), 157–191. Handy, C. H. (1981). Managementgoden. Deventer: Kluwer.

28


Heck, R. H. & Hallinger, P. (2009). Testing a dynamic model of organizational leadership and school improvement effects on growth in learning. Paper presented at the Annual Meeting of American Educational Research Association: Disciplined Inquiry: Education Research in the Circle of Knowledge, San Diego, CA. Hofstede, G. (1997). Culture and organizations: software of the mind. New York: McGraw-Hill. Hughes, M. (1985). Leadership in professionally staffed organizations. In M. Hughes, P. Ribbins, & H. Thomas (Eds.), Managing Education (262–290). London: Holt Rinehart & Winston. Hulpia, H. (2009). Distributed leadership and organizational outcomes in secondary schools. Universiteit van Gent (dissertation), Gent. Imants, J. (1996). Leiding geven aan onderwijs! Onderwijskundig leiderschap en taakdifferentiatie in basisscholen. Rijksuniversiteit Leiden: DSWO Press. Jackson, D. (2000). The school improvement journey: Perspectives on leadership. School Leadership & Management, 20(1), 61–78. Judge, T. A., Bono, J. E., Ilies, R., & Gerhardt, M. W. (2002). Personality and Leadership: A qualitative and quantitative review. Journal of Applied Psychology, 87(4), 765–780. Kerr, St., & Jermier, J. (1978). Substitutes for leadership: Their meaning and measurement. Organizational Behaviour and Human Performance, 22, 375–403. Kleine-Kracht, P. (1993). Indirect instructional leadership: An administrator’s choice. Educational Administration Quarterly, 29(2), 187–212. Krüger, M. L. (1994). Sekseverschillen in schoolleiderschap. Academisch proefschrift, UvA. Alphen aan den Rijn: Samsom H.D. Tjeenk Willink. Krüger, M. L. (1996). Sekse van de schooldirecteur in relatie tot de gerichtheid op het primaire proces. Pedagogische Studiën, 73(1), 25–41. Krüger, M. L. (2009). The big five of school leadership competencies in the Netherlands. School Leadership & Management, 29(2), 109–127. Krüger, M. L. (2010). Leading schools in the knowledge society: On the way to leaders of learning in inquiry-based schools. In A. H. Normore (Ed.), Global Perspectives on Instructional leadership Reform: The Development and Preperation of Leaders of Learning and Learners of Leadership. Book series Advances in Educational Administration (Vol. 11, pp. 397–417). Bingley, UK: Emerald Group Publishing Limited. Langlois, R., & Robertson, P. L. (1993). Business organisation as a coordination problem: toward a dynamic theory of the boundaries of the firm. Business and Economic History, 22(1), 31–41. Leithwood, K. (1992). Transformational leadership: Where does it stand? Instructional Leadership, 49, 8–12. Leithwood, K. (1994). Leadership for school restructuring. Educational Administration Quarterly, 30(4), 498–518. Leithwood, K., Begley, P. T., & Cousins, J. B. (1990). The nature, causes and consequences of prinicpals’ practice: an agenda for future research. Journal of Educational Administration, 28(4), 5–30. Leithwood, K., & Jantzi, D. (2006). Transformational school leadership for large-scale reform: Effects on students, teachers, and their classroom practices. School Effectiveness and School Improvement, 17(2), 201–227. Leithwood, K., Jantzi, D., & McElheron-Hopkins, Ch. (2006). The development and testing of a school improvement model. School Effectiveness and School Improvement, 17(4), 441–464. Leithwood, K., Jantzi, D., & Steinbach, (1999). Changing leadership for changing times. Philadelphia: Open university press. Leithwood, K., & Montgomery, D. J. (1986). Improving principal effectiveness: The principal profile. OISE Press: Toronto. Leithwood, K., Seashore Louis, K. S., Anderson, S., & Wahlstrom, K. (2004). Review of research: How leadership influences learning. New York: The Wallace Foundation. Lewin, K., & Lippitt, R. (1938). An experimental approach to the study of autocracy and democracy: A preliminary note. Sociometry, 1, 292–300. Likert, R. (1961). New patterns of management. New York: McGraw-Hill.


29

Louis, K. S., & Marks, H. (1998). Does professional community affect the classroom? Teacher’ work and student work in restructuring schools. American Journal of Education, 106(4), 532–575. Marks, H. M., & Printy, S. M. (2003). Principal leadership and school performance: An integration of transformational and instructional leadership. Educational Administration Quarterly, 39(3), 370–397. McPherson, R. B., Crowson, R. L., & Pitner, N. J. (1986). Managing uncertainty, administrative theory and practice in education. Columbus: Charles E. Merrill Publ. Comp. Mintzberg, H. (1980). The nature of managerial work. New York: Prentice Hall. Mulford, B. (2003). School leaders: changing roles and impact on teacher and school effectiveness. OECD, Paris, paper commissioned by the Education and Training Policy Division for the activity Attracting, Developing and Retaining Effective Teachers. Ontleend aan: http://www.oecd.org/dataoecd/61/61/2635399.pdf. Mulford, B., & Bishop, P. (1997). Leadership in organisational learning and student outcomes (LOLSO) Project. Launceston, Australia: University of Tasmania. Mulford, B., & Silins, H. (2003). Leadership for organisational learning and improved student outcomes-What do we know? Cambridge Journal of Education, 33(2), 175–195. Nguni, S., Sleegers, P., & Denessen, E. (2006). Transformational and transactional leadership effects on teachers’ job satisfaction, organizational commitment, and organizational citizenship behavior in primary schools: The Tanzanian case. School Effectiveness and School Improvement, 17(2), 145–177. Nonaka, I., & Toyama, R. (2002). A firm as a dialectical being: Towards a dynamic theory of a firm. Industrial and Corporate Change, 11(5), 995–1009. Ogawa, R., & Bossert, S. (1995). Leadership as an organizational quality. Educational Administration Quarterly, 31, 224–243. Pitner, N. J. (1986). Substitutes for principal leader behavior: An exploratory study. Educational Administration Quarterly, 22(2), 23–42. Pounder, D. G., Ogawa, R. T., & Adams, E. A. (1995). Leadership as an organization-wide phenomena: Its impact on school performance. Educational Administration Quarterly, 31(4), 564–588. Quinn, R. E., & Cameron, K. S. (1983). Organizational life cycles and shifting criteria of effectiveness: Some preliminary evidence. Management Science, 19, 33–51. Quinn, R. E., Faerman, S. R., Thompson, M. P., & McGrath, M. R. (1996). Becoming a master manager: A competency framework. New York: Wiley. Ribbins, P., & Gunter, H. (2002). Mapping leadership studies in education. Educational Management & Administration, 30(4), 359–385. Richmon, M., & Allison, D. J. (2003). Toward a conceptual framework of leadership inquiry. Educational Management & Administration, 31(1), 31–50. Scheerens, J. (2008). Review of research on school and instructional effectiveness. Enschede: University of Twente, Department of Educational Organization and Management. Scheerens, J., & Bosker, R. (1997). The foundations of educational effectiveness. Oxford: Pergamon. Scheerens, J., & Witziers, B. (2005). Educational leadership and student performance. Enschede: University of Twente, Department of Educational Organisation and Management. Sergiovanni, T. (1984). Leadership and excellence in schooling. In Educational leadership, February, 4-3. Sergiovanni, T. J. (1990). Advances in leadership theory and practice. In P. W. Thurston & L. S. Lotto (Eds.), Advances in educational administration, vol 1 (Part A): perspectives on educational reform (pp. 1–35). Thousand Oakes: Corwin Press. Southworth, G. (2002). Instructional leadership in schools: Reflections and empirical evidence. School Leadership and Management, 22(1), 73–92. Stoel, W. G. R. (1995). De taakinhoud, taakomvang en taakbelasting van schoolleiders in het basisonderwijs. Tijdschrift voor Onderwijsonderzoek, 20, 133–154.

30


Stoll, L., & Fink, D. (1996). Changing our schools: Linking school effectiveness and school improvement. Buckingham: Open University Press. Van de Grift, W., & Houtveen, A. A. M. (1999). Instructional leadership and pupil achievement in primary education. School Effectiveness and School Improvement, 10(4), 373–389. Van Vilsteren, C. A. (1999). Onderwijskundig leiderschap van schoolleiders in het voortgezet onderwijs: schoolleiding en vaksecties (proefschrift). Enschede: Universiteit Twente. Vroom, V. H., & Yetton, P. W. (1973). Leadership and decision-making. Pittsburgh: University of Pittsburgh Press. Williams, L., & Podsakoff, P. (1989). Longitudinal field methods for studying reciprocal relationships in organizational behavior research: Toward improved causal analysis. Research in Organizational Behavior, 11, 247–292. Witziers, B., Bosker, R. J., & Krüger, M. L. (2003). Instructional leadership and student achievement: The elusive search for an association. Educational Administrative Quarterly, 39(3), 398–425. Youngs, H. (2009). (Un)Critical times? Situated distributed leadership in the field. Journal of Educational Administration and History, 41(4), 377–389.

Chapter 2

Earlier Meta-Analyses Jaap Scheerens and Rien Steen

Introduction In this chapter we will summarise meta-analyses that we conducted earlier (Scheerens and Bosker 1997; Scheerens et al. 2007). In the final section a brief overview of the results of other meta-analyses will be given.

Description of Earlier Conducted Meta-Analyses by the Authors In this section a description is given of quantitative analyses conducted on the basis of a data set that combined studies from two previous meta-analyses, namely those by Scheerens and Bosker (1997) and by Scheerens et al. (2005). Combined results are presented in Scheerens et al. (2007). Both data sets were re-checked and scrutinised, which in some cases led to a different ‘‘scoring’’ of the basic effect sizes. Only associations of school and instructional variables with cognitive educational achievement were used in the analyses. The meta-analyses were focused on school and instructional conditions expected to be positively associated with student achievement. Only results on instructional leadership will be presented here.

J. Scheerens (&) R. Steen Department of Educational Organization and Management, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands e-mail: [email protected] R. Steen e-mail: [email protected]

J. Scheerens (ed.), School Leadership Effects Revisited, SpringerBriefs in Education, DOI: 10.1007/978-94-007-2768-7_2, Ó The Author(s) 2012

31

32

J. Scheerens and R. Steen

The size of the data base for the school level factors is shown in Table 2.1. Table 2.1 Numbers of studies and replications for the school level variables (‘‘replications’’ are associations between the school variable in question and student achievement) Publications Replications ‘‘Old’’ studies 1985–1994 ‘‘New’’ studies 1995–2005 Total

83 72 155

511 700 1,211

Methods Literature Search A meta-analysis relies on collecting as many studies as possible regarding the topic of interest. The search methods included searches on the Web of Science, and the ERIC and ERA databases. The search was focused at articles published between 1985 and 2005. In addition, the literature database of ECER conferences was examined. In the search the following key words were used: school effectiveness, learning results, effectiveness, effective teaching, effective instruction, teacher effectiveness, educational effectiveness, school effectiveness and student achievement. Finally, recent reviews and books on school effectiveness were checked in order to find additional relevant literature (‘snowball method’). The first step of this search resulted in several hundreds of publications. From these publications, about one-third appeared not to be useful for our purposes, while from one-sixth of all publications it could not be determined whether or not they contained useful information. These were articles that appeared to be inaccessible. This left us with 155 articles that contained information relevant for the purposes of our study in the domain of school effectiveness and 177 articles in the domain of teaching effectiveness. These articles were analysed with regard to effect size presented on student achievement outcomes and relevant school and teaching effectiveness variables, while at the same time data were collected on particular study characteristics.

Type of Meta-Analyses A meta-analysis can be conducted in different ways. In this study two types of meta-analyses were carried out. First, a so called ‘‘vote-counting’’ procedure took place applied to the school level variables. Vote counting comes down to

2 Earlier Meta-Analyses

33

counting the number of significant (positive and negative) associations between a dependent variable and a specific independent variable of interest from a given set of studies. More specifically it was examined for each selected study whether or not the test statistic concerning a particular variable of interest exceeded a conventional critical value at a given significance level. In our case a level of a = 0.05 was used. It might be considered to take the individual study as the unit of analysis. However, it must be noted that some studies deal with multiple outcome indicators and/or multiple indicators of concepts relating to teacher and/ or school effectiveness. For example, when a (hypothetical) study is using two indicators on the factor time (e.g. time spent on mathematics and time spent on reading) and assesses the impact of each indicator on two outcomes (e.g. mathematic test and reading test), there are four relationships (2 process indicators 9 2 outcomes). These four relationships were combined into two relationships by averaging the effects of both indicators for each of the outcomes. The resulting two relationships were examined for their direction and significance and, consequently, these two results (or replications) were included in our final data set. A similar approach was used when one indicator is used in studies carried out in several countries, i.e. the relationship for each country is entered as a separate replication. The conventional vote-counting procedure has been criticised on several grounds. First, it does not incorporate sample size into the vote. As sample sizes increases, the probability of obtaining statistically significant results increases. Second, the procedure does not allow the researcher to determine which treatment is the best in an absolute sense. Although information is found about the best treatment, it is unknown what the margin of superiority is; it does not provide an effect size estimate. Third, the procedure has a very low power for the range of sample sizes and effect sizes most common in the social sciences. When effect sizes are medium to small, the conventional votecounting procedure frequently fails to detect the effects. Moreover, for medium to small effect sizes, the power of the conventional vote-counting procedure tends to zero as the number of studies to be included increases (Bushman 1994). Finally, when a vote-counting procedure is followed, associating counts with moderator variables (study or content characteristics that might influence whether a replication is significant or not) can only be done in a relatively crude way, as compared to procedures in which effect sizes for each replication are calculated. Given this critique, a more refined approach has additional value. Such an approach makes it possible to estimate the ‘true’ effect sizes concerning the association of particular independent variables (in our case process indicators that represent a particular effectiveness-enhancing condition e.g. parental involvement) and dependent variables (school effects, in our case, e.g. student achievement scores in mathematics). Moreover, such an approach makes use of the fact that effect sizes vary among studies. Analysing this variation makes it possible to establish whether specific study characteristics may account for it. We have applied such an approach to our set of studies and replications as well.

34


In fact, next to the quantitative approach, the vote-counting procedure was only used for the meta-analysis on school factors, whereas for the meta-analysis of teaching effectiveness studies we used the multi-level quantitative technique only. More specifically, a multilevel approach to meta-analysis (Raudenbush and Bryk 1985; Hox 2002) was applied. In this approach the selected studies are considered to be a sample from the population of studies, in our case this regards the relationship between specific school effectiveness indicators and student outcomes. Nested under each study are the secondary units: the schools. Each study can then be viewed as an independent replication. This concept could be used but would not solve the problem of biased estimates due to unidentified dependencies when applying multiple results from one study, e.g. when effects are reported for mathematics and language achievement in one study while using the same sample of schools and students. To deal with this problem, instead of the two-level model for meta-analysis a three-level model was used, in which the highest level of the studies is referred to as the acrossreplication level, and the multiple results within a study as the within-replication level. The principal advantages of the statistical meta-analysis employed here are threefold: first, the information from each study is weighted by the reliability of the information, in this case the sample size and second, dependencies within study replications are controlled for. Third, the method applied enables us to examine which study characteristics (or moderators) are responsible for the variation in effect sizes.

Results on School Leadership Dependent and moderator variables Dependent variables used in our study were student outcomes in the cognitive domain, namely student achievement results in mathematics, language, and other subjects, including science. As it was stated in the above, our method allows us to model effect sizes as a function of study characteristics. A first relevant characteristic deals with the question of whether studies have used a language, a mathematics test score, or another score to assess student achievement. This moderator provides insight into the question as to which learning outcomes are most ‘malleable’ by school characteristics. Previous studies (Scheerens and Bosker 1997) suggest that schools have more impact in the area of mathematics than in the area of language. In our study 45.3% of our data relate to the use of a math test, 33.8% of all results to a language test. Apart from examining the impact of the type of test employed, we also investigated the effects of the country in which the study was conducted (the United States of America, the Netherlands, or other countries) and the education


35

level or sector in which the study took place (primary or secondary education). Results regarding these study characteristics provide insight into the question of which context is most ‘susceptible’ for school effectiveness indicators. Studies from the past show that, by and large, effect sizes are higher in US-schools and in primary schools (Scheerens and Bosker 1997, Chap. 6). In our study 33.5% of all effect sizes relate to studies conducted in the US, 24.5% to studies carried out in the Netherlands and 42% to studies conducted in other countries. With regard to school type 63.7% of all results relate to studies carried out in primary schools (36.3% in secondary schools). The other moderator variables relate to the quality of the studies involved. One of them relates to the issue whether or not studies control for student intake characteristics. Effect sizes are by definition less accurate in case outcomes are not corrected for student intake characteristics. Almost all studies in our database include characteristics such as socio-economic status, age, gender, ethnicity and, in a minority of cases, prior achievement, implying that only in rare cases the dependent variable represents learning gain. In this study 73.2% of our results have a ‘value added’-character. Another feature of school effectiveness research is its reliance on correlation design and consequently on statistical techniques suited to analyse data produced by such a design. With regard to statistical techniques in the nineties the school effectiveness community has witnessed the rise of multi-level analysis, a technique which takes into account the ‘nested’ character of data and therefore yields more precise and accurate results than studies relying on, for example, aggregate data. Nowadays this technique is prevalent in school effectiveness research, although other more traditional techniques are still employed as well. This raises the question of whether studies applying ‘high quality’ techniques such as multi-level analysis yield different results from studies in which more traditional techniques have been used. We therefore included ‘‘multi-level/not multi-level’’ as an additional moderator variable in our analyses. In our study 57.6% of all results are based upon multi-level techniques, the other 42.4% on other techniques. To indicate the effect of school effectiveness variables, Fisher’s Z transformation of the correlation coefficient was used. Not all studies presented their results in terms of correlations, and therefore all other effect size measures were transformed into correlations, using formulae presented by Rosenthal (1994). For small values of the correlation coefficient, Zr and r do not differ much, but it should be remembered that all figures presented in the following and indicating effect sizes refer to Zr. More information about the statistical procedure followed can be found in the appendix.

Vote Counting The results representing vote counts concerning school leadership are shown in Table 2.2.

36


Table 2.2 Vote counts, based on replications (%) Primary Secondary Subject math Subject schools schools language -

n.s.

+

Instructional 7 84 9 leadership Country USA

-

n.s.

+

Instructional 5 84 11 leadership Technique multi-level Instructional 1 leadership

Subject other than math and language

-

n.s.

+

-

n.s.

+

-

n.s.

+

-

n.s.

+

2

94

4

5

88

7

0

98

2

13

71

17

Country the Netherlands

Country other than USA and NLD

Design gross

Design value added

-

n.s.

+

-

n.s.

+

-

n.s.

+

-

n.s.

+

10

88

3

2

92

6

8

89

3

4

86

11

-

n.s.

+

5

87

8

Technique not multi-level

School level

Teacher level -

n.s.

+

-

n.s.

+

-

n.s.

+

94

4

8

82

10

5

87

8

n.s.

Total +

- Negative effect, significant at 0.05; n.s. non-significant effect; + positive effect, significant at 0.05

The results across all school level variables (not shown in the table above) indicate that in most cases relationships between school effectiveness indicators and student outcome variables are not significant. This is the case for about 70% of all relationships examined in the study. Only in a minority of cases the relationships between school effectiveness indicators and outcome variables are negative. In total about 4% of all relationships are negatively significant. Around 25% of all relationship examined are positively significant. As far as instructional leadership is concerned, only in 8% of all cases positive, significant relationships are found. The overall vote-counting results indicate that positive findings are mostly found for studies carried out in the United States and for studies using mathematic tests, studies not employing multi-level techniques and, as was to be expected, for studies using gross, unadjusted outcomes. These results apply for (almost) every school effectiveness indicator examined in this study. More mixed results are found with respect to the moderator variable type of school. Depending on the school effectiveness indicator, sometimes more positive findings are found in studies conducted in primary schools, other times in studies in secondary schools. The same conclusion can be reached with regard to the moderator level of measurement and the question whether or not multi-level analyses were used. As far as instructional leadership is concerned; more significant effects are found in the USA than in other countries, there are more


37

Table 2.3 Multilevel, empty model Number of cases Instructional leadership

Across replications

Within replications

Mean effect size

53

170

0.046*

Effect-sizes marked as * are significant at the 0.10 level

Table 2.4 Multilevel, with moderators as predictors Variance Intercept Secondary Arithmetic/ Language USA math Instructional 0.052 leadership

-0.002

0.009

0.011

The Value Netherlands added

0.050 -0.095

0.012

Not Class/ multilevel teacher level -0.018

–

positive significant effects in primary than in secondary education and more positive significant effects for other subjects than math and language (17% positive vs. respectively 7 and 2%).

Multi-Level Approach The results of the multi-level approach to meta-analysis at school level are presented in Tables 2.3 and 2.4. Table 2.3 shows the average effect sizes for all independent variables, and results are generalised over all moderator variables, including assessment domains (mathematics, reading and other). Table 2.4 shows the impact of the various moderator variables. These results show that the overall effect size found in our meta-analyses, based on 170 replications, embedded in 53 studies is 0.046. This is a very low effect size. None of the moderator variables had a statistically significant effect.

Other Meta-Analyses In this section the results of six meta-analyses are summarised: Witziers et al. (2003), Marzano et al. (2005), Chin (2007), Robinson et al. (2008), Creemers and Kyriakides (2008) and Hattie (2009). Witziers et al. (2003), used the same data base as Scheerens and Bosker (1997), but added results from IEA studies. Doing so, they obtained an effect size of r = 0.02. When taking out the IEA studies this effect size was doubled to r = 0.04.

38


Marzano et al. (2005) conducted a meta-analysis based on 69 studies, all from the USA. They investigated the average (direct) effect of general leadership, a term suggesting that they put all types of school leadership under one head, using one general label. In addition they looked at the effect of 21 more specific leadership responsibilities on student achievement. Expressed as a correlation the average effect size that they found for general leadership was 0.25. The authors compare their results to those of Witziers et al. (2003), who, following a similar approach, found an average effect size of r = 0.02. As possible explanations for these strongly different outcomes they offer three lines of reasoning. First, they note that the sample of studies used by Witziers et al. comprised studies from different countries, noting that effect sizes for studies from the USA were higher than those for other countries (r = 0.11 for the studies from the USA). As indicated above the Marzano analysis used only studies from the USA. Next they considered that two corrections made by them might have contributed to the lower effect sizes in the study by Witziers et al. (2003). Marzano et al. (2005) followed a procedure that removed outliers and corrected measures of school leadership and student achievement for attenuation, in other words for unreliability in these measures. The Marzano study has not looked at specific characteristics of studies as possible moderators of the effect sizes found, e.g. subject matter orientation, primary or secondary schools or methodological study characteristics (e.g. correction of student background characteristics, application of multi-level analysis). Chin (2007), presents a meta-analysis based on 28 studies from Taiwan and the United States. The studies address transformational leadership. Of the 28 studies only 11 have student achievement as the dependent variable. Teachers’ job satisfaction and teachers’ perception of effectiveness are also used as dependent variables. Effect sizes are expressed in terms of correlations (Fisher’s Z). The average effect size for the 11 studies that have included student achievement is 0.49. Effect sizes vary from 0.010 to 0.89. Median effect sizes are about 0.45. Robinson et al. (2008) conducted a meta-analyses on 22 published studies. Of these 22 studies 12 addressed instructional leadership, five transformational leadership and another five other kind of school leadership concepts. The average effect size, found for instructional leadership was 0.42, expressed in terms of a correlation this would be 0.21. For transformational leadership the effect size was 0.11 (r = 0.055), and for the other types of leadership 0.30 (r = 0.15). Among the studies that addressed instructional leadership high effect sizes were found for five studies that used between group designs; the other studies showed low to moderate effect sizes. Typically such intergroup comparisons would compare high and low performing schools. Next, effect sizes across studies are reported of five leadership dimensions, namely ‘‘establishing goals and expectations’’ (ES = 0.42), ‘‘strategic resourcing’’ (ES = 0.31), ‘‘planning coordinating and evaluating teaching and the curriculum’’ (ES = 0.42). ‘‘promoting and participating in teacher learning and development’’ (ES = 0.84) and ‘‘Ensuring an orderly and supportive environment’’ (ES = 0.27). Creemers and Kyriakides (2008, pp. 201–203) computed the average effect size of leadership on the basis of 29 studies, both primary and secondary schools. The


39

Table 2.5 Summary of results from meta-analyses on school leadership; effect sizes are rendered as correlations between school leadership and student achievement Meta-analysis by Leadership concept Effect size (correlation) (r) Witziers et al. (2003) Marzano et al. (2005) Chin (2007) Robinson et al. (2008) (1) Robinson et al. (2008) (2) Creemers and Kyriakides (2008) Hattie (2009)

School leadership Generalised school leadership Transformational leadership Instructional leadership Transformational leadership School leadership School leadership

0.02 0.25 0.49 0.21 0.06 0.07 0.18

dependent variable is indicated as student achievement. The average effect size (Fisher’s Z) they found was 0.068. This low effect size is in line with the effect size found in Scheerens and Bosker (1997), Witziers et al. (2003) and Scheerens et al. (2007), reported previously. Hattie (2009) summarises the results of 11 meta-analyses (Neumann et al. 1989; Pantili et al. 1991; Gasper 1992; Bosker and Witziers 1995; Brown 2001; Wiseman 2002; Witziers et al. 2003; Waters et al. 2003; Waters and Marzano 2006; Chin 2007; Robinson et al. 2008). Among this list there are some duplications, the two references to Witziers and Bosker, are based on the same data set, which is also the same data set as used for the analyses of Scheerens and Bosker (1997), as referred to in an earlier section of this section. The meta-analysis by Waters and Marzano, has not looked at school principals but at district superintendents. Finally, the study by Neumann at all looked at job satisfaction and other teacher outcomes as dependent variables. The average effect size found is 0.36, which comes down to a correlation of 0.18 of leadership with the dependent variable. Hattie makes much of emphasising that instructional leadership has a much higher effect size than transformational leadership. His basis for doing so is mainly the study by Robinson et al. (2008), summarised in the above, while he reasons away the relatively high effect size found in Chin’s meta-analysis of transformational leadership by saying that her conceptualisation of transformational leadership had important elements in common with instructional leadership (Hattie 2009, p. 84). In Table 2.5 the results of the six meta-analyses are summarised Witziers et al. (2003), Marzano et al. (2005), Chin (2007), Robinson et al. (2008), Creemers and Kyriakides (2008) and Hattie (2009). Not including our own recent meta-analysis (with an effect size rounded to 0.05), the average effect size across these meta-analyses comes down to 0.17. When leaving out the outlying value of the meta-analysis by Chin the average effect size would become r = 0.12.

40


References Bosker, R. J., & Witziers, B. (1995, January). School effects, problems, solution and a metaanalysis. Paper presented at the International Congress for School Effectiveness and School Improvement, Leeuwarden, The Netherlands. Brown, L. I. (2001). A meta-analysis of research on the influence of leadership on student outcomes. Unpublished Ph.D., Virginia Polytechnic Institute and State University, VA. Bushman, B. J. (1994). Vote-counting procedures in meta-analysis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 193–213). New York:Russell Sage Foundation. Chin, J. M.-C. (2007). Meta-analysis of transformational school leadership effects on school outcomes in Taiwan and the USA. Asia Pacific Education Review, 8(2), 166–177. Creemers, B., & Kyriakides, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. New York: Routledge. Gasper, S. (1992). Transformational leadership. In integrative review of the literature (Doctoral dissertation, Western Michigan University, 1992). Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge. Hox, J. J. (2002). Multilevel analysis: Techniques and applications. Mahwah, N. J.: Lawrence Erlbaum Associates. Marzano, R. J., Waters, T., & McNulty, B. A. (2005). School leadership that works: From research to results. Alexandria, VA: Association for Supervision and Curriculum Development. Neumann, G. A., Edwards, J. E., & Raju, N. S. (1989). Organizational development interventions: A meta-analysis of their effects on satisfaction and other attitudes. Personnel Psychology, 42(3), 461–489. Pantili, L., Williams, J., & Fortune, J. (1991, April). Principal assessment: Effective or not? A meta-analytic model. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. Raudenbush, S., & Bryk, A. (1985). Empirical Bayes meta-analysis. Journal of Educational Statistics, 10(2), 75–98. Robinson, V. M. J., Lloyd, C., & Rowe, K. J. (2008). The impact of leadership on student outcomes: An analysis of the differential effects of leadership type. Educational Administration Quarterly, 44(5), 635–674. Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231–244). New York: Russell Sage Foundation. Scheerens, J., & Bosker, R. (1997). The foundations of educational effectiveness. Oxford: Pergamon. Scheerens, J., Luyten, H., Steen, R., & Luyten-de Thouars, Y. (2007). Review and meta-analyses of school and teaching effectiveness. Enschede: Department of Educational Organisation and Management, University of Twente. Scheerens, J., Seidel, T., Witziers, B., Hendriks, M., & Doornekamp, G. (2005). Positioning the supervision frameworks for primary and secondary education of the Dutch Educational Inspectorate in current educational discourse and validating core indicators against the knowledge base of educational effectiveness research. Enschede/Kiel: University of Twente/ Institute for Science Education (IPN). Waters, T. J., & Marzano, R. J. (2006). School district leadership that works: The effect of superintendent leadership on student achievement. Denver, CO: Mid-continent Research for Education and Learning. Waters, T., Marzano, R. J., & McNulty, B. (2003). Balanced leadership: What 30 years of research tells us about the effect of leadership on student achievement. A working paper. Midcontinent Regional Educational Lab., Aurora, CO. [BBB23081]. Research for Education and Learning.


41

Wiseman, A. W. (2002, February). Principals’ instructional management activity and student achievement: A meta-analysis. Paper presented at the Annual Meeting of the Southwestern Educational Research Association, Austin, TX. Witziers, B., Bosker, R. J., & Krüger, M. L. (2003). Instructional leadership and student achievement: The elusive search for an association. Educational Administrative Quarterly, 39(3), 398–425.

Annex: Research Articles Used for the Meta-Analyses by the Authors Articles on School Leadership (1985–1995) Bamburg, J. D., & Andrews, R. L. (1991). School Goals, Principals and Achievement. School Effectiveness and School Improvement, 2(3), 175–191. Bedford, B. (1993). School effectiveness characteristics and student achievement: a study of relationships in Georgia Middle Schools. ERIC document No. EA 020 722. Blank, R. K. (1987). The role of principal as leader: Analysis of variation in leadership of urban high schools. Journal of Educational Administration, 81(2), 69–80. Brandsma, H., & Knuver, A. (1989). De invloed van school-en klaskenmerken op rekenprestaties in het onderwijs. In J. Scheerens & J. C. Verhoeven (red.), Schoolorganisatie, beleid en onderwijskwaliteit. Brewer, D. J. (1993). Principals and student outcomes. Economics of Education Review, 12(4), 281–292. Cantu, M. (1994). A study of principal instructional leadership behaviors in successful and nonsuccessful urban elementary schools. Unpublished doctoral thesis. Dissertation Abstract International, 55(7), January 1995. Austin: University of Texas. Couch, J. C. (1991). A study of student achievement and how it relates to the principal in the role of instructional leader. Paper presented at the Annual Meeting of the Mid-South Educational Research Association, Lexington, New York. Durland, M., & Teddlie, Ch. (1996). A network analysis of the structural dimensions of principal leadership in differentially effective schools. Paper presented at the annual meeting of the AERA, New York. Eberts, R. W., & Stone, J. A. (1988). Student achievement in public schools: do principals make a difference. Economics of Education Review, 7(3), 291–299. Ellet, C. D., & Logan, C. S. (1990). Analyses of school learning environments: organizational coupling, robustness and effectiveness. Paper presented at the AERA, Boston. Friedkin, N. E., & Slater, M. R. (1994). School leadership and performance: a social network approach. Sociology of Education, 67(2), 139–157. Gouldring, E. B., & Pasternak, R. (1994). Principals’ coordinating strategies and school effectiveness. School Effectiveness and School Improvement, 5(3), 239–253. Grift, W. van de (1987). Zelfpercepties van onderwijskundig leiderschap en gemiddelde leerprestaties. In J. Scheerens & W. G. R. Stoel (Eds.), Effectiviteit van onderwijsorganisaties. Lisse: Swets & Zeitlinger. Grift, W. van de (1990). Instructional leadership and academic achievement in elementary education. School Effectiveness and School Improvement, 1(3), 26–40. Grisay, A. (1995). Effective and less effective junior schools in France. Paper presented at the annual meeting of the ICSEI, Leeuwarden. Guillemard, L., Palmer, D. J., & Wilson, V. L. (1994). Discrimination of average, effective and ineffective intermediate and secondary schools from ratings of school characteristics by

42


parents, students, teachers and principals. Paper presented at the annual meeting of the AREA, New Orleans. Hallinger, P. (1989). What makes a difference? School context, principal leadership and student achievement. Paper presented at the annual meeting of the AERA, San Francisco. Heck, R., Marcoulides, G. A., & Lang, P. (1991). Principal instructional leadership and school achievement: The application of discriminant techniques. School Effectiveness and School Improvement, 2(2), 115–135. Hill, P. W., Rowe, K. J., & Holmes-Smith, P. (1995). Factors affecting students’ educational progress: Multilevel modeling of educational effectiveness. Paper presented at the annual meeting of the ICSEI, Leeuwarden. Hofman, R. H. (1993). Effectief schoolbestuur: een studie naar de bijdrage van schoolbesturen aan de effectiviteit van scholen. Groningen: Rijksuniversiteit Groningen, dissertatie. Houtveen, Th., Vermeulen, C., & Van der Grift, W. (1993). Bouwstenen voor onderzoek naar de kwaliteit van scholen. Utrecht: ISOR. Hoy, W. K., Tater, J. C., & Bliss, J. R. (1990). Organizational climate, school health, and effectiveness: a comparative analysis. Educational Administration Quarterly, 26(3), 260–279. Hunter, C. (1994). Los Angeles unified school district middle school principals’ instructional behaviors and academic achievement. Unpublished doctoral dissertation. Dissertation Abstracts International, 55(12), June 1995. Los Angeles: Peppardine University. Krug, S. E. (1992). Instructional leadership, school instructional climate and student learning outcomes. Department of Education, Washington, DC: Office of Educational Research and Improvement. Leitner, D. (1994). Do principals affect student outcomes: an organizational perspective. School Effectiveness and School Improvement, 5(3), 219–238. Mandeville, G. K., & Kennedy, E. (1991). The relationship of effective school indicators and changes in the social distribution of achievement. School Effectiveness and School Improvement, 2, 14–33. Mortimore, P., Sammons, P., Stoll, L., Lewis, D., & Ecob, R. (1988). School matters: the junior years. Somerset: Open Books. Postlewaithe, N., & Ross, K. (1992). Effective schools in reading; implications for educational planners. Den Haag: IEA. Reezigt, G., Guldemond, H., & Creemers, B. (1995). The empirical validity of the school effectiveness model. Paper presented at the annual meeting of the ECER, Bath. Rowan, B. P., Chiang, F. S., & Miller, R. J. (1996). Using research on employee performance to study teaching effectiveness: results from an analysis of teachers effects on student achievement in mathematics using NELS: 88 data. Paper presented at the annual meeting of the AERA, New York. Rymenans, R., Geudens, V., Coucke, H., Van den Bergh, H., & Daems, F. (1996). Effectiviteit van Vlaamse secundaire scholen: een onderzoek naar de effecten van het onderwijsaanbod, de tijdsbesteding aan het Nederlands en de schoolkenmerken op de lees- en schrijfprestaties. Antwerpen: Universiteit van Antwerpen. Sontag, L., & Meijnen, G. W. (1995). De invloed van school- en klaskenmerken op de cognitieve en lingu vaardigheden van kleuters. Pedagogische Studiën, 72(4), 258–272. Tam, W. M., & Cheng, Y. C. (1995). School environment and student performance. Paper presented at the annual meeting of the ICSEI, Leeuwarden. Teddlie, C., & Stringfield, S. (1993). Schools make a difference. Lessons learned from a 10 year study of school effects. New York: Teachers College Press. Vermeulen, C. J. (1987). De effectiviteit van 17 Rotterdamse stimuleringsscholen. Pedagogische Studiën, 64, 49–58. Werf, M. P. C. van der, & Weide, M. G. (1991). Effectief onderwijs voor allochtone leerlingen. Tijdschrift voor Onderwijsresearch, 16(4), 231–243. Werf, M. P. C. van der, & Weide, M. G. (1993). Effectieve voorrangskenmerken in de school en klas. Pedagogische Studiën, 70, 108–221.


43

Witziers, B. (1992). Coördinatie binnen scholen voor voortgezet onderwijs. Enschede: Universiteit Twente, dissertatie. Yelton, B. T., Miller, K., & Ruscoe, G. C. (1994). The stability of school effectiveness: comparative path models. Paper presented at the annual meeting of the AERA, New Orleans, 4–8 April.

Articles on School Leadership (1995–2005) Creemers, B., & Werf, G. v. d. (2000). Economic viewpoints in educational effectiveness: Costeffectiveness analysis of an educational improvement project. School Effectiveness and School Improvement, 11(3), 361–384. D’Agostino, J. V. (2000). Instructional and school effects on students’ longitudinal reading and mathematics achievements. School Effectiveness and School Improvement, 11(2), 197–235. Deinum, J. F. (2000). Schoolbeleid, instructie en leerresultaten. Groningen: GION. Griffith, J. (2003). Schools as organizational models: Implications for examining school effectiveness. Elementary School Journal, 104(1), 29–47. Grift, W. van de, & Houtveen, A. A. M. (1999). Instructional leadership and pupil achievement in primary education. School Effectiveness and School Improvement, 10(4), 373–389. Grift, W. van de, Houtveen, T., & Vermeulen, C. (1997). Instructional climate in Dutch secondary education. School Effectiveness and School Improvement, 8(4), 449–462. Hallinger, P., Bickman, L., & Davis, K. (1996). School context, principal leadership, and student reading achievement. Elementary School Journal, 96(5), 527–549. Heck, R. H., & Marcoulides, G. A. (1996). School culture and performance: Testing the invariance of an organizational model. School Effectiveness and School Improvement, 7(1), 76–95. Hill, P. W., & Rowe, K. J. (1998). Modelling student progress in studies of educational effectiveness. School Effectiveness and School Improvement, 9(3), 310–333. Hofman, R. H., Hofman, W. H. A., & Guldemond, H. (1999). Social and cognitive outcomes: A comparison of contexts of learning. School Effectiveness and School Improvement, 10(3), 352–366. Hofman, R. H., Hofman, W. H. A., & Guldemond, H. (2001). The effectiveness of cohesive schools. International Journal of Leadership in Education, 4(2), 115–135. Jacob, B. A., & Lefgren, L. (2004). The impact of teacher training on student achievement- quasiexperimental evidence from school reform efforts in Chicago. Journal Of Human Resources, 39(1), 50–79. Leithwood, K., & Jantzi, D. (2000). Principal and teacher leadership effects: A replication. School Leadership and Management, 20(4), 415–434. Levacˇic´, R., Steele, F., Smees, R., & Malmberg, L. (2003, 11–13 September). The relationship between school climate & head teacher leadership, and pupil attainment: Evidence from a sample of English secondary schools. Paper presented at the British Educational Research Association Annual Conference, Edinburgh. Marks, H. M., & Printy, S. M. (2003). Principal leadership and school performance: An integration of transformational and instructional leadership. Educational Administration Quarterly, 39(3), 370–397. Reezigt, G. J., Guldemond, H., & Creemers, B. P. M. (1999). Empirical validity for a comprehensive model on educational effectiveness. School Effectiveness and School Improvement, 10(2), 24. Trautwein, U., Koller, O., Schmitz, B., & Baumert, J. (2002). Do homework assignments enhance achievement? A multilevel analysis in 7th-grade mathematics. Contemporary Educational Psychology, 27(1), 26–50.

44


Werf, G. van der (1997). Differences in school and instruction characteristics between high-, average-, and low-effective schools. School Effectiveness and School Improvement, 8(4), 430–448. Werf, G. van der, Creemers, B., Jong, R. de, & Klaver, E. (2000). Evaluation of school improvement through an educational effectiveness model: The case of Indonesia’s PEQIP project. Comparative Education Review, 44(3), 329–355.

Chapter 3

Anatomy of Some Representative School Leaders’ Effectiveness Studies Jaap Scheerens

Introduction In this chapter five recent studies which have addressed indirect effect models of school leadership, are reviewed in somewhat more detail. The studies have been carried out in different countries, Australia, the United Kingdom, 17 OECD countries and the United States, respectively.

The Leadership for Organizational Learning and Student Outcomes Study (Silins and Mulford 2004) The leadership for organizational learning and student outcomes (LOSLO) project is widely cited as one of the first studies that used an elaborate causal model to study leadership effects. The focus of the study was first of all to elaborate the concept of organizational learning in schools. It was seen in relationship with transformational leadership, teacher leadership and aspects of teachers’ work. In relating these organizational factors with student outcomes, student engagement and participation were chosen as the outcome variables. The LOSLO study was carried out in 96 Australian high schools. Data were collected by means of questionnaires to principals, teachers (2503) and students (3500 grade ten students).

J. Scheerens (&) Department of Educational Organization and Management, University of Twente, Drienerlolaan 5, P.O. Box 217, 7522 NB Enschede, The Netherlands e-mail: [email protected]


45

46

J. Scheerens

Key Constructs The concept of Organizational Learning is the first key construct of this study. Organizational learning is defined as having four main components: 1. A trusting and collaborative climate 2. A shared and monitored school mission (the extent to which teachers participate in all aspects of the school’s functioning) 3. Taking initiatives and risks (the extent to which the school leader and school structures support experimentation and teachers feel supported in taking initiatives) 4. Professional development (the extent to which staff draw on available knowledge and skills and continuously improve their performance). Leadership is the second key construct. It is defined as ‘‘transformational leadership’’, with the following six main components: 1. Vision and goals (the extent to which the principal works toward whole staff consensus in establishing school priorities) 2. Culture (the extent to which the principal promotes an atmosphere of caring and trust among staff) 3. Structure (the extent to which the principal in a school structure promotes participative decision making, supports delegation and distributive leadership and encourages teacher autonomy in making decisions) 4. Intellectual stimulation (stimulate teachers’ refection on their work with students, stimulate staff to learn from each other) 5. Individual support (moral support, showing appreciation and taking that into account when taking decisions) 6. Performance expectation (high expectations for teachers and students and a principal who expects the staff to be effective and innovative). The underlying message in both concepts seems to be the imperative to collaborate, trust one another and be optimistic about performance. The two lists of dimensions are quite similar in carrying this message, and even run the risk of creating a tautology. When depending on ratings from the same respondents, some doubt about the independence of the key measures in a study like this might be raised.

Other Variables in the Study The other categories of variables included in the study are: school context variables, some other internal school variables (next to organizational learning and transformational leadership described above) and student outcome variables.

3 Anatomy of Some Representative School Leaders’ Effectiveness Studies

47

School context variables • Socio economic status (SES) • School size Other internal school variables • Resource: the degree to which teachers feel that resources for improving staff effectiveness are available • Staff valued: the extent to which staff feel valued • Leadership satisfaction: satisfaction with the leadership from all sources in the school • Community focus: the degree of responsiveness to the community • Teacher leadership: the extent to which individual teachers, teams and whole staff work together and are sources of leadership in the school • Teachers’ work. This variable was collected from students’ responses and has the following items: students liking the way they are instructed, variety of instructional activities employed, the extent to which students’ work is discussed with them, organization of the class, expectations of students that they will do their best and the extent to which students are challenged in class. Student outcome variables • Engagement, student engagement with school including students’ perceptions of the way that teachers are relating to them, perceptions of the relationship with their peers, perceptions of the usefulness of their school work for later life and the extent of identification with the school. • Participation, including absences, participation in extra curricular activities, preparedness to do extra school work, involvement in classroom decisions and in setting learning goals, voicing opinions in class Again there is some overlap in some of these concepts: strong emphasis upon all actors in the school valuing one another, sharing responsibility and working together.

Data Analysis The data were analyzed by means of a partial least square path analysis. A fully recursive model was fitted, using 12 variables: socio economic status, school size, resource, leader (i.e., transformational leadership), staff valued, leadership satisfaction, community focus, teacher leadership, organizational learning, teachers’ work, participation and student engagement. For most latent variables depending on several items the item consistency was fairly high. Organizational learning came out as the spider in the web of relationships between the organizational variables. It mediated the effect of transformational leadership on teachers’ work. Relatively small indirect effects of transformational leadership on student

48

J. Scheerens

engagement and participation were found. The total of indirect effects on participation was a path coefficient of 0.09. The total of indirect effects on engagement is reported as 0.16. It should be noted that the total of indirect effects was based on the sum of 4–5 rather complicated indirect effects, and that the model did not show a more straightforward pattern of indirect effects, e.g., from leadership via organizational learning to student engagement.

Concluding Remarks Although the study was led by an elaborate conceptualisation, some doubts about tautological elements were raised. The conceptual work does not relate to the empirical results and multilevel models of educational effectiveness research. In fact in that literature, cooperation and consensus, the cornerstone of the LOSLO model, usually has only a relatively modest effect on student achievement (e.g., Scheerens and Bosker 1997). Although interesting, the variable ‘‘teachers’ work’’, can only be seen as a far proxy of instruction at classroom level. Participation and engagement are conceptualized and operationalized well in this study, but it could be argued that they are rather intermediary effects than ultimate student outcomes. No publications could be detected on the LOSLO study where academic outcomes were incorporated, although, Mulford (2006), hints that this has been done (without providing actual figures). In the same publication he also says that student participation was directly and engagement indirectly associated with academic achievement. Substantively, and assuming for a moment that engagement and participation can be used as proxies for educational outcomes, the LOSLO study shows small total effects of leadership on outcomes, based on a maze of complicated indirect effects.

The Study by Day et al. (2009) The study by Day et al. is titled: ‘‘The impact of school leadership on pupil outcomes’’. It is a major, labor-intensive study, on a relatively large sample of British primary and secondary schools which improved their performance over a 3 year period (2003–2005). The authors present a set of strong causal claims about the importance of school leadership in promoting school improvement and school effectiveness. They conclude by saying that their study ‘‘… provides evidence of an increasingly sophisticated understanding of the strong links that have traditionally been thought to exist between school leadership and student achievement’’ (p. 191). Yet, the actual findings of their quantitative analyses hardly show any effect, direct or indirect, on pupil outcomes. It is worthwhile to follow the design and analysis in more detail. No attention will be given to the extensive case study material that is also rendered in the report. Suggestive and interesting as this material may be, it does not directly address the impact of leadership on student achievement.


49

Sampling and Data Collection From national census data all primary and secondary schools that ‘‘were effective and improving over a 3 year period’’ were indentified. School performance data consisted of pupils’ cognitive outcomes from the Key Stage national assessment and CSCE results. At a response rate of about 20% this resulted in the participation of 378 primary schools and 362 secondary schools. Schools were categorized as low starters (change from low to moderate performance), moderate (change form moderate to high performance) or stable high. Survey data were collected in two waves, one immediately after the 2003–2005 period over which the improvement had occurred, and a second wave, 1 year later. Data were collected by means of surveys (questionnaires) to school heads, key staff members (3–5 per school), other teachers and students. For the quantitative analysis that will be discussed in this review of the study only the response from the school head survey at wave 1 was used. Change in average school performance in mathematics and reading over the 3-year period in question (2003–2005) was used as the dependent variable. From the report it can be gathered that change was measured as a change between cohorts, and not in the sense of individual growth of students over a 3-year period. The main categories of variables that were used in the first wave head teacher survey, and that were used as the main variable categories in the subsequent data analyses (structural equation modeling) were: leadership practice, leaders’ internal states, leadership distribution, school conditions and classroom conditions. It is important to note that all data consisted of relatively high inference ratings by school heads, and that no objectively measured factual data were included. For example, class size was measured by asking the head teachers about the degree to which class size was ‘‘considered excessive’’. As far as leadership activities were concerned, the data consisted of head teacher self-reports and not of ratings by teachers, which is often used as a more objective approach in research on school leaders.

Concept Development and Measurement The following major variable categories were used in the study: • • • • •

Leadership practices Leaders’ internal states Leadership distribution Improvement in school conditions and disciplinary climate Characteristics of school conditions academic emphasis, school culture, teaching policies and practices and extra curricular activities • Classroom conditions, workload volume and workload complexity

50

J. Scheerens

Leadership Practices Questionnaire development started out from an a priori conceptualization of four main leadership practices: setting directions, developing people, redesigning the organization, and managing the teaching and learning program. Factor analyses conducted on the primary school head teacher data identified four factors, three in line with the first three a priori factors, setting directions, developing people and redesigning the organization; the fourth considerably reduced to ‘‘use of data’’. This means that the relatively large issue of managing the teaching and learning program, in this study came to be reduced to use of data. (Which is a very relevant and interesting phenomenon but just a facet of the teaching and learning program.) The factor analyses for secondary schools identified five factors, the same four as for the primary schools, and an additional category named use of classroom observation. All factors were based on about four items. The nature of the items provides further illustration on what was actually registered in this study. Setting directions was, among others, represented by the item: ‘‘demonstrating high expectations for students’’; developing people by ‘‘encouraging staff to think of learning beyond the academic curriculum’’; redesigning the organization by ‘‘increasing dialogue about school improvement between pupils and adults’’, and use of data by the item ‘‘encouraging staff to use data for their work’’. The items refer to rather general kinds of behaviour, are often literal statements of the factor labels (such as the item on use of data), and are not very precise and factual. Sometimes the set of three to four items that load on a factor are a rather narrow interpretation of the label, for instance redesigning the organization has no items that refer to any kind of structural reform of the school organization but just mention various kinds of cooperation that could well be seen as instances of normal day-to-day functioning, without any kind of redesign.

Leaders’ Internal States Three dimensions were identified, leader trust in teachers, leader efficacy with respect to their ability to improve teaching and learning and leader efficacy with respect to the belief in the sustainability of their own motivation and commitment. Exemplary items are, respectively: ‘‘I feel a strong loyalty to my teachers’’, ‘‘Head feels able to create a positive learning environment in the school’’ and ‘‘Head feels able to sustain own job satisfaction in leadership role’’.

Leadership Distribution For this general category seven subdimensions were identified: leadership provision by school management team (SMT), leadership provision by staff, leadership provision by individual groups, distributed leadership, SLM collaboration,


51

SLM impact on learning and teaching standards, SLM impact on policies in relation to homework and lesson planning. Distributed leadership was represented by two items: ‘‘Most leadership tasks are not carried out by the head or the SLM’’ and ‘‘Many others take on leadership tasks’’.

Improvement in School Conditions and Disciplinary Climate Under this heading improvement in school conditions, pupil behaviour and pupil attendance were subsumed. Order and discipline issues are predominant in each of these school conditions. Enhanced commitment of teachers was included as well. For the primary school data reduced teacher mobility and increased teacher attendance was added as a dimension.

Characteristics of School Conditions, Academic Emphasis, School Culture, Teaching Polices and Practices and Extracurricular Activities From the secondary school data four dimensions were identified with respect to the teaching and learning culture: • Positive learner motivation and learning culture (‘‘Pupils respect others who get good marks’’) • High academic standards (‘‘This school sets high standards’’) • Assessment for learning (‘‘The class teacher uses pupil data to set academic standards’’) • Teacher collaborative culture (‘‘Teachers in our school mostly work together to improve their practice’’) Next, again based on the secondary school data three additional factors were identified, associated with the ‘‘coherence in school wide learning and teaching programs and extra curricular activities’’: • External collaboration and learning opportunities • Extra curricular activities (‘‘most of our pupils participate in extra curricular activities’’ • School wide coherent learning and teaching program this comprised an item on ‘‘ability grouping’’ and the following restatement of the issue as an item: ‘‘we are able to provide a coherent teaching and learning program’’ Practically the same dimensions were identified on the basis of the primary school data.

52

J. Scheerens

Classroom Conditions: Workload Volume and Workload Complexity Here the following dimensions were identified: • Class size (‘‘Teachers in this school do not teach an excessive number of pupils’’) • Teacher workload volume (‘‘Teachers’ workload is quite fair, compared to other teachers in other schools’’) • Teacher workload complexity (‘‘Most pupils in the school are cooperative’’) • Teacher autonomy and positive learning atmosphere (‘‘The atmosphere throughout my school encourages pupils to learn’’) It is quite striking that actual teaching strategies and curricular emphases are exempt from these ‘‘classroom conditions’’. Three dimensions are a weak approach of teachers’ working conditions; the fourth includes ‘‘a positive atmosphere’’. This is a general weak point in school leadership modeling efforts. Heck and Hallinger (2010), refer to the treatment of this category as a black box in school leadership effect research. In another widely cited piece of research, the LOSLO project, teachers’ work is relatively narrowly operationalized as the way students perceive the relationship with their teachers. An interesting attempt at covering instructional strategy in a structural equation model of school leadership, is made in the study by Louis et al. (2010), who introduces a category ‘‘focused teaching’’, which encompasses direct teaching and constructivist approaches. In the studies by Heck and Moryiama (2010) and Ten Bruggencate et al. (2010), which will be discussed further on, an even more explicit use is made of the teaching effectiveness literature.

Structural Equation Modeling Out of the seven categories of dimensions that were discussed in the above four return in structural models, differentiated for secondary and primary schools. The factors that are represented in the model as ‘‘levels’’ are key dimensions of leadership based on activities and leaders internal states (six factors, level 1), leadership distribution (four factors, level 2), mediating factors in the structural model (four factors, level 3) and factors representing intermediary outcomes that seem to be directly or indirectly associated with change in student achievement (five factors, level 4). The factors within the four levels for secondary schools are specified below: Level 1: key dimensions of leadership • Setting directions • Redesigning the organization • Head trust (of teachers)


53

• Use of data • Developing people • Use of observation Level 2: leadership distribution • • • •

Distributed leadership Staff collaboration SMT collaboration SMTs impact on learning and teaching

Level 3: mediating school factors • • • •

Teacher collaborative culture Assessment for learning Improvement in school conditions External collaboration and learning opportunities

Level 4: intermediary outcomes • • • • •

High academic standards Pupil motivation and responsibility for learning Reduction in staff mobility and absence Change in pupil behaviour Change in pupil attendance

The dependent variable is indicated as ‘‘pupil academic outcomes’’, which actually reflects change in academic outcomes over a 3-year period.

Results of the Structural Equation Modeling In the secondary school model direct effects on student achievement are found for the factors: SMTs impact on student and learning (0.11), improvement in student behaviour (0.14) and staff collaboration (0.12). None of the key dimensions of leadership has a direct effect on academic outcomes. The following indirect effects are found: • An indirect effect of Trust, via staff collaboration on achievement (0.20 9 0.12) of 0.02 • An indirect effect of Trust via SMT supporting teaching and learning on achievement (0.17 9 0.11) of 0.02 • An indirect effect of Redesigning the organization, via Improvement in School conditions and Improvement in pupil behaviour on student achievement (0.33 9 0.24 9 0.14) of 0.01. This means that the analyses of the effect size of the secondary school models leads up to a total effect of leadership on achievement of 0.05.

54

J. Scheerens

When turning to the primary school model (p. 98 of the report) again there is no direct effect of any of the key leadership dimensions. From the path diagram only one indirect effect can be discerned, namely from Trust, via Teacher collaborative culture and Pupil motivation to academic outcomes (0.40 9 0.21 9 0.16) of 0.01.

Final Comments The results of the quantitative part of this study stand in sharp contrast to the interpretation that is given by the authors and was cited at the beginning of this review. When they speak of ‘‘strong links’’ they have actually demonstrated small to negligent indirect effects of their key leadership dimensions and no direct effect on student outcomes whatsoever. Question marks can be put behind practically every one of their claims about the importance of leadership for the improvement of learning outcomes. For example their first claim, in the last chapter is as follows: ‘‘Their … (head teachers J.S.’’)... educational values and leadership practices shape the internal processes and pedagogical practices that result in improved pupil outcomes’’. (p. 183). Very little is actually shown in terms of improved pupil outcomes attributable to leadership dimensions. And: ‘‘School leaders improve teaching and learning indirectly and most powerfully through their influence on staff motivation, commitment and working conditions.’’ (p. 185). These statements are puzzling in the face of the actual indirect associations that were shown, or rather failed to show up, in the structural equation modeling.

Ten Bruggencate et al. (2010) ‘‘From School Leadership to Student Achievement, Analyses Based on TIMSS, 2007’’ The TIMSS surveys focus on student achievement in mathematics and science at the fourth and eight grades. In total 59 countries participated in TIMSS 2007, but the analyses presented in the current study focused on the 14 participating OECD countries (and Slovenia) whose eighth grade students were assessed. Data were collected in October–December 2006 (in the Southern Hemisphere countries) and March–June 2007 (in the Northern Hemisphere countries). The study made use of the fact that TIMSS contains one questionnaire item which is directed at priorities in principal’s work. A structural equation model was tested, with leadership, contextual variables, various kinds of intermediary variables and mathematics achievement as the outcome variable.


55

Conceptual Basis The study was placed in the tradition of school effectiveness research, assuming that leadership conditions facilitate effective teaching conditions. Thus teachers’ beliefs and practices were used as a first category of intermediary variables. Student attitudes were expected to function as a second category of intermediary variables. Next a number of school context variables were included. Among them were the ‘‘normal’’ background characteristics such as school size, school SES composition and urbanity of the school’s location. A striking choice in this study was to consider school climate variables and school resources as context variables as well. Mostly these variables are used as intermediary variables in indirect effect modeling studies of school leadership. In the case of the current study these variables were used as antecedent conditions of school leadership, rather than factors affected by leadership behaviour.

Variables School Leadership The item from the TIMSS principal questionnaire to measure school leadership is presented in Table 3.1.

Table 3.1 Leadership item TIMSS, 2007. Cited from Ten Bruggencate et al. (2010) By the end of this school year, approximately what percentage of time in your role as principal will you have spent on these activities? Write in the percent The total should add to 100% a. Administrative duties (e.g., hiring, budgeting, scheduling) b. Instructional leadership (e.g., developing curriculum and pedagogy) c. Supervising and evaluating teachers and other staff d. Teaching e. Public relations and fundraising f. Other

Teachers’ Beliefs and Practices In this category the following variables were used: • The amount of professional development of teachers, estimated by principals • The amount of professional development of teachers, indicated by teachers

56

J. Scheerens

• Teachers’ job satisfaction, as rated by teachers (teacher responses) • Topic coverage (the degree to which a range of subject matter topics of math, addressed in the mathematic test, had actually been taught)

Student Attitudes In this category of intermediary variables two issues were covered: • Student attitudes towards mathematics • Students attitudes toward school

Context Variables Based on responses from the principal the following context variables were used: • Total school enrolment • Size of the community where the school is located • School climate: ‘‘the items on which this index is based include teacher, student and parent behaviours (e.g., teachers’ job satisfaction, parental involvement and students’ desire to do well in school)’’ (ibid) • Frequency of student misbehaviours like classroom disturbance and vandalism Next two variables were constructed on the basis of responses from teachers, which reflect limiting conditions to effective teaching: • Problematic student characteristics like strongly varying academic abilities and lack of interest • The second index relates to a shortage of ICT resources (hardware, software and support) A final context variable was based on results from the student questionnaire, namely a five category variable expressing the number of books available in the students’ homes.

Results of the LISREL Analysis The results of the structural equation modeling, using school level data, are indicated in the path diagram, cited from the original study’s report (Fig. 3.1). The effects of the context variables were not shown in the diagram in the service of making it better legible. Very small direct effects from the leadership variables ‘‘supervising teachers’’ (positive) and ‘‘administrative duties’’ (negative) were noted. The sign of these factors makes sense, as administrative duties might be seen as taking away time


57

No. of books at home; Total school enrolment; Type of community; Disadvantaged students limiting teaching; Student behavior frequency; Shortage of ICT resources limiting teaching; School climate (principal’s perception).

Teachers’job satisfaction (teacher perc.)

0.08 Valuing school

Professional development (principal perc.) Instructional leadership

0.45 0.18 Valuing math.

-0.12 0.08

Administrative duties

0.15

-0.06

-0.21 0.18

Math achievement

-0.04 -0.13

0.04 0.24

Supervising teachers 0.12

Topic coverage

0.15 Public relations Professional development (teacher perc.)

0.09 0.07

Fig. 3.1 Effects of Leadership activities and other TIMSS 2007 school variables on achievement. Only significant relationships are shown. Cited from Ten Bruggencate et al. (2010)

that principals might have spent more effectively on other matters. Next, indirect effects of ‘‘supervising teachers and ‘‘public relations’’, via ‘‘topic coverage’’ were noted of 0.03 and 0.04, respectively. The indirect effect from instructional leadership on mathematics achievement is positive, but tiny 0.02. The total effects (direct and indirect effects together) of the four leadership variables are: instructional leadership: 0.02, administrative duties -0.09, supervising teachers, 0.09 and public relations 0.04. Topic coverage had the strongest direct effect on mathematics achievement. Of the context variables, ‘‘books at home’’ had a relatively strong effect on achievement, namely 0.51. The suppressing climate and resources conditions had sizeable negative effects. A final striking outcome of this analysis was the fact that none of the school organizational variables, professional development and teachers’ job satisfaction, seemed to be affected by leadership.

58

J. Scheerens

Final Comments Interesting aspects of this study are its international basis and the fact that an instructional variable ‘‘pur sang’’ as ‘‘topic coverage’’, also often indicated as ‘‘opportunity to learn’’, appeared to function as a relevant intermediary variable for leadership oriented towards supervision of teachers. A more counterintuitive result was that instructional leadership in the sense of developing the curriculum and pedagogy appeared to have no influence on topic coverage. This latter outcome gives food for thought about the way instructional leadership actually works out.

The Study by Heck and Moriyama (2010) ‘Examining Relationships Among Elementary Schools’ Contexts, Leadership, Instructional Practices, and Added-Year Outcomes: A Regression Discontinuity Approach’ The study by Heck and Moriyama used data from 198 public schools, available from a state Department of Education survey. State assessment data, comprising mathematic and reading skills were used to compute an indicator of academic growth between Grades 4 and 5. The authors used the Spring 2005 administration of the school survey to measure school practices and the 2003 administration to measure leadership. The scales obtained from the survey data were based on the combined responses of teachers, parents and students. Altogether 12,342 fourth grade students and 12,831 fifth grade students participated in the study.

Theoretical Background In quite a few earlier studies that investigated indirect leadership effect models, the overall conceptualization lacked a clear rationale for the selection of intermediary variables. Particularly when leadership was conceptualized as transformational the choice of intermediary variables meandered in all directions, coming down to a broad conception of a ‘‘good’’ organization (clear mission, collaboration, empowerment of teachers), without a specific reasoning about how these general organizational ‘‘goodies’’, would enhance student achievement in targeted areas. Endemic to these studies was the neglect, or superficial treatment of instructional conditions as a key intermediary area of influencing student performance. Heck and Moriayama make an important step forward by drawing on the school effectiveness literature, particularly those authors that have developed conceptual multilevel effectiveness models, in which organizational conditions are seen as supportive of effective instructional arrangements at classroom level (Creemers 1994; Scheerens and Bosker 1997; Creemers and Kyriakides 2008). In the current


59

study they use four dimensions of instructional practices: classroom teaching (including time and opportunity to learn), support for student learning, professional capacity of the school’s teaching staff and focus on sustained learning improvement. Leadership is fully conceptualized as distributed leadership; the authors use the term collaborative leadership and define it on the basis of three subdimensions, governance, collaboration in school improvement decisions, and participation in evaluating the school’s academic development. Perhaps it could be argued that a leadership construct that is more closely aligned to supporting instructional conditions at classroom level is instructional rather than distributed leadership. In this sense a more radical instructional effectiveness-oriented approach to indirect effect models of school leadership might have been considered.

Variables Appendix 1 of Heck and Moryiama (ibid) describes the definition and coding of variables as follows: ‘‘Student background: grade level (Grade 4 = 0, Grade 5 = 1), gender (female = 1, male = 0), low socioeconomic status (participated in the federal free or reduced meal program coded 1, else = 0), English Language Learners (ELL, as opposed to Limited English Language Learners, coded 1; else 0), age [i.e., transformed into a number based on the total months between the child’s birth date and the date of the assessment and then centered (M 0) with respect to the month of April (month of testing)]. School context and student composition: school size (i.e., number of students enrolled for the 2004–2005 school year), student stability (percentage of students who attended the same school for the entire school year), student composition is a weighted composite of percentage of student participation in the federal subsidized lunch program, percentage of students receiving special education (SPED) services and percentage receiving limited English proficiency (ELL) services. Composition was then standardized (M = 0, SD = 1). Larger positive values indicate school settings where these percentages of students are higher. Teaching staff variables: licensing (percentage of full-time teachers who met state license to teach in public schools in the state), staff stability (percentage of teachers with 5 or more years at the school); teacher experience (average number of years of teaching at a school). School instructional practices. The four dimensions (State Department of Education, 2001) used to define instructional capacity are summarized below, with Cronbach’s alphas in parentheses and example items paraphrased from the original survey. Classroom teaching conditions (a = 0.91). This dimension is defined by opportunity to learn (i.e., curriculum is based on state content and performance standards, teaching and learning activities are focused on helping students meet

60

J. Scheerens

state learner outcomes, homework is appropriate and focused on meeting content and performance standards, students learn how to assess their own progress and set own learning goals, teachers and parents are informed about academic expectations and content to be covered, school has high academic standards and expectations for all students), quality of instruction (i.e., instruction oriented toward developing student cognitive skills, teacher uses a variety of teaching strategies and activities, students have diverse ways to show they have learned material, instruction focused on active participation, students receive feedback on progress and suggestions for improvement, teacher uses assessments to adjust instruction), and use of time (instructional time is flexible and organized to support learning). Quality of student support system (a = 0.85). This dimension is defined as creating a learning environment that supports learning (e.g., school environment supports learning, students are well behaved, discipline problems are handled quickly and fairly, safety is emphasized, teachers care about and respect students, administrators and staff treat each other with respect, teachers provide extra help for students who need it, school activities meet student interests and needs, programs meet special needs of students, open communication exists among teachers and administrators, school is well maintained). Professional capacity (a = 0.80). The third dimension is defined by the extent to which the teaching staff has instructional expertise and is able to participate in quality professional development, focused on improving instruction, and also the presence of systematic evaluation in place to help teachers grow professionally. Focused and sustained action on improvement (a = 0.83). The last dimension refers to the school’s focus on academic improvement (i.e., school communicates its goals to staff, parents, and students; school has high academic standards for all students; changes in curriculum and instructional practices are coordinated and school wide; school learner outcomes are widely known; respondent is personally involved in the school improvement process; the school’s vision is reviewed on a regular basis with the involvement of its stakeholders). Collaborative leadership. Leadership was defined by eight items that measure three specific aspects of collaborative leadership (with items paraphrased in parentheses): • Shared school governance which encourages shared vision, broad participation, and shared accountability for student learning (e.g., administrators, teachers, and staff work together effectively to achieve school goals; teachers can freely provide input and express concerns to administrators; school provides opportunities for parents to participate in important decisions about their children’s education, e.g., scheduling, homework, discipline) • Collaborative decisions focusing on academic improvement (e.g., school ensures teachers have a major role in decisions about curriculum development; school provides opportunities for teachers to plan and make school decisions about professional development and curriculum; teachers have needed instructional resources to teach effectively)


61

• Broad participation in efforts to evaluate the school’s academic development (e.g., school provides regular opportunities for all stakeholders to review the school’s vision and purpose; assesses progress in making school changes).’’

Math and Reading Outcomes The April 2005 state assessment to measure students’ achievement based on statedeveloped math and reading content and performance standards was used. Each test consisted of a mix of constructed-response items and standardized test items from the Stanford Achievement Test, 9th edition (Harcourt Brace Educational Measurement, 1997). Student scores (re-scaled to range from 100 to 500) were equated across the 2 years to enable the measurement of academic growth between Grades 4 and 5.

Design and Analyses The authors used a regression discontinuity design to compute an ‘‘added year effect’’ for mathematics and learning. This approach makes use of the condition that in many school systems there is an exact ‘‘cutting score’’, in this case a specific calendar age that students should have to be admitted to school. By examining the achievement differences between students in adjacent grades that have practically the same calendar age, an estimate is made of the effect of 1 year of schooling. This methodology was developed and elaborated by Luyten (2006). Multilevel structural equation modeling was used to assess the indirect effect of collaborative leadership, taking into consideration school contextual conditions.

Results Leadership was positively associated with instructional practice (0.28), and instructional practice, in its turn was positively associated with the added year effect in math (0.49) and reading (0.57). This yielded indirect effects of 0.14 for mathematics and 0.16 for reading.

Final Comments This study is innovative in connecting with the educational effectiveness literature, which gives it a clearer rationale for understanding how indirect effects of leadership are mediated. It is also innovative in using state-of-the-art methodology for

62

J. Scheerens

multilevel structural equation modeling connected to the idea of computing ‘‘added year’’ effects of schooling as the outcome variable. The results of the study seem to indicate that improvement-oriented collaborative leadership is positively associated with effective instructional practices. One of the facets of the study that could be considered as a weakness, and which is also explicitly recognized by the authors, is the fact that the measurements of leadership and instructional conditions are based on ratings from participants and not on more objective data. More objectively measured indicators of school and classroom functioning might be the next step forward in this field. By concentrating on collaborative leadership it could be put up for discussion whether this study is still about leadership, or about something like an ‘‘integrated and achievement oriented functioning of the school organization’’. Somehow the leader, even in the sense of an overall supervisor or meta-controller,1 has disappeared from the scene.

Conclusion When making up the balance of this more in-depth review of studies, four issues jump to the fore: the conceptual basis of the studies, the total leadership effects that were found, the kind of intermediary variables that appear to be most promising and methodological developments.

Conceptual Basis Of the four studies two follow a more organizational orientation in their conceptual models and two are more directly associated with multilevel school effectiveness models. Organizational variables, strongly related to cooperation and consensus, have a central place as mediating variables in the studies by Silins and Mulford, and by Day et al. In the two other studies, those by Ten Bruggencate et al. and Heck and Moryiama, there is more explicit additional attention to instructional conditions.

Total Effect Sizes The total effect sizes of leadership, comprising direct and indirect effects, are quite small in the first three studies (mostly below a correlation of 0.10); the effect sizes in the study by Heck and Moryiama, are slightly higher (0.14 and 0.16). These

1

A meta-controller is controlling controllers.


63

findings are in line with those found in the meta-analytic results presented in the earlier chapters.

What are Effective Intermediary Variables? The range of organizational variables used in the first two studies present a maze of intercorrelations between leadership facets and school organizational variables, but often the crucial link with student outcomes is missing. Only in the study by Heck and Moriyama a straightforward causal path from leadership via instructional conditions to student academic performance of some magnitude was found. Including more objective measures of instructional conditions seems to be a more promising approach to the choice of intermediary variables than remaining with organizational factors expressing positive feelings, cooperation and shared decision making.

Methodological Developments Two of the four studies are purely cross-sectional, while those by Day et al. and Heck and Moryiama have built in more longitudinal aspects. Truly longitudinal designs are clearly the way ahead to further improve leadership studies, and educational effectiveness studies in general.

References Creemers, B. P. M. (1994). The Effective Classroom. London: Cassell. Creemers, B., & Kyriakides, L. (2008). The dynamics of educational effectiveness: a contribution to policy, practice and theory in contemporary schools. New York: Routledge. Day, C., Sammons, P., Hopkins, D., Harris, A., Leithwood, K., Gu, Q., et al. (2009). The Impact of School Leadership on Pupil Outcomes. Nottingham, UK: The National College for School Leadership. Heck, R. H., & Hallinger, P. (2010). Testing a longitudinal model of distributed leadership effects on school improvement. The Leadership Quarterly, 21, 867–885. Heck, R. H., & Moriyama, K. (2010). Examining relationships among elementary schools’ contexts, leadership, instructional practices, and added-year outcomes: a regression discontinuity approach. School Effectiveness and School Improvement, 21(4), 377–408. Louis, K. S., Dretzke, B., & Wahlstrom, K. (2010). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21(3), 315–336. Luyten, H. (2006). An empirical assessment of the absolute effect of schooling: regressiondiscontinuity applied to TIMSS-95. Oxford Review of Education, 32(3), 397–429. Mulford, B. (2006). Leadership for improving the quality of secondary education: Some international developments. New Zealand Journal of Educational Leadership, 21(1), 7–27.

64

J. Scheerens

Scheerens, J., & Bosker, R. (1997). The foundations of educational effectiveness. Oxford: Pergamon. Silins, H., & Mulford, B. (2004). Schools as learning organizations: Effects on teacher leadership and student outcomes. School Effectiveness and School Improvement, 15(3–4), 443–446. Ten Bruggencate, G., Luyten, H., & Scheerens, J. (2010). Quantitative analysis of international data. exploring indrect effect models of school leadership. Enschede: University of Twente.

Chapter 4

Results from School Leadership Effectiveness Studies (2005–2010) Maria Hendriks and Rien Steen

In this chapter an overview is presented of recent studies in which the impact of school leadership on student achievement was investigated. The technique of meta-analysis was used to synthesize the results of 25 studies published between 2005 and 2010. In the meta-analysis both studies exploiting direct and indirect models of school leadership were examined. Next, the conceptual content of the school leadership measures and intermediary variables used in the studies was analyzed. This chapter consists of a first section in which the research method is described, a second section in which the results of the direct effect studies are presented and a third section which presents the results of the studies guided by indirect effect models. The chapter ends with a short concluding section.

Method Literature Search To identify potential relevant studies the following online databases were used: Web of science (www.isiknowledge.com), Scopus (www.scopus.com), ERIC and Psycinfo (provided through Ebscohost). The search was carried out in November 2010 and focused on publications between 2005 and 2010. The databases were M. Hendriks (&) R. Steen Department of Educational Organization and Management, University of Twente, P.O. Box 217 7500 AE, Enschede, The Netherlands e-mail: [email protected] R. Steen e-mail: [email protected]


65

66

M. Hendriks and R. Steen

Table 4.1 Results literature search (November 2010)

Database

Number of hits

ERIC PsycInfo Scopus Web of science Total Duplicates Total number of possible relevant publications

37 140 84 42 303 48 255

searched using the key terms that were also used in the meta-analyses published by Scheerens et al. (2007). The databases were searched using a combination of the following groups of key terms: • ‘‘school effectiveness’’, ‘‘education* effectiveness’’, ‘‘teach* effectiveness’’, effective* teaching’’, ‘‘effective instruction’’, ‘‘instruction* effectiveness’’, ‘‘mastery learning’’, ‘‘constructivist teaching’’, ‘‘mathematics instruction’’, ‘‘reading instruction’’, ‘‘science instruction’’, ‘‘classrooms’’, ‘‘mathematics teaching’’, ‘‘reading teaching’’, ‘‘science teaching’’; • ‘‘value added’’, attainment, achievement, ‘‘learn* result*’’, ‘‘earn* outcome*’’, ‘‘learn* gain’’, ‘‘student* progress’’; • leadership, principal. In total 303 hits were found (see Table 4.1). After removing the duplicate publications 255 unique publications were left. In addition to the search in databases the following journals were searched: • • • • • • • •

American Educational Research Journal Educational Administration Quarterly Educational Management Administration & Leadership International Journal of Leadership in Education Journal of Educational Administration Leadership and Policy in Schools School Leadership and Management School Effectiveness and School Improvement

Finally, recent reviews and books on school leadership and school effectiveness as well as references in recent articles were checked in order to find additional literature.

Inclusion Criteria The first selection of the studies collected was guided by the following selection criteria:

4 Results from School Leadership Effectiveness Studies

67

1. Independent variable: The study is designed explicitly to examine school leadership. 2. Dependent variable: The study had to include an explicit measure of cognitive student achievement. 3. Language of the publication: Publications included had to be written in English or Dutch. Databases and journals other than primarily English were not searched. 4. Study population: The study had to be conducted at primary and/or secondary school level (for students aged 4–18). 5. Year of publication: The study is published or presented not earlier than January 2005 and before January 2011. 6. Methods: Studies had to contain empirical data and outcomes. Titles and abstracts of publications were evaluated on the six selection criteria. Using the above-mentioned selection criteria 80 publications remained for further evaluation. Each of these publications was examined in full. After this second round 25 publications were selected for meta-analysis.

Conceptual Models to Study the Relationship Between School Leadership and Achievement Based on the approaches described by Pitner (1986) and Hallinger and Heck (1998) to study school leadership effects in non- experimental research, De Maeyer et al. (2007) describe five conceptual models that could be used to study the relationship between school leadership and achievement. These models are the direct effects model, the direct effect model with antecedents, the indirect effect model, the indirect effect model with antecedents and the recursive or reciprocal effect model. Direct models assume a direct relationship with achievement, indirect models assume mediated effects, i.e. that school leadership influences pupil achievement through intermediate variables. In 16 of the 25 publications selected for this meta-analysis, researchers used an indirect model to study the relationship between school leadership and student achievement. In nine publications a direct model was used. In this meta-analysis no further distinction is made in models with and without antecedents. Studies that assumed a recursive model were not among the ones selected for the meta-analysis. Two studies (Borden 2010; Kythreotis et al. 2010) started out from an indirect model. In both studies however, the relationship between school leadership and achievement was modeled directly. Therefore these studies are included under the studies that used a direct effects model. The studies by Louis et al. (2010) and by Opdenakker and Van Damme (2007) used different approaches to test the relationship between school leadership and student achievement, and were therefore classified both under the direct and indirect models.

68


Finally, some of the studies using a mediated-effect model also examined direct effects; the outcomes of these studies were classified under the indirect effect models.

Meta-Analysis: Direct Effect Models In this section the results are presented of the 11 studies in which direct effect models were used. In contrary to the previous meta-analyses by Scheerens et al. (2007) (as reported in Chap. 2 of this report) in this meta-analysis no mean effect size over all publications has been computed. The reason is that out of the 11 publications from which direct effects were available only three publications reported standardized effects for all relevant effects. Six publications published only non-standardized effects and two publications published standardised effects for some effects only. Given this state of affairs, instead of conducting a mean effect size, a so called ‘‘vote-count’’ procedure was followed. This comes down to counting the number of positive significant effects, negative significant effects and non-significant effects (all based on a two-sided test with p \ 0.05). A vote count procedure provides an overview of the number of replications (any association between leadership and student achievement) that are statistically significant and in the expected (positive) direction. The limitation of this procedure is, however, that the size of effects is not accounted for. In the vote counting procedure used in this meta-analysis the individual replication in an individual study is taken as the unit of analysis. Some studies examine multiple independent variables with regard to leadership (e.g. Instructional leadership and Shared leadership) and/or multiple dependent variables (e.g. reading and math). In other studies the relationship between the same leadership variable and outcome variable is examined in different cohorts of students or in several countries or different analysis methods are used to investigate the direct effects. In this meta-analysis each relationship between a leadership variable and an outcome measure is entered as a separate replication. In total 94 replications were used in the analysis. Table A.1 (attached at the end of this chapter) provides information on the characteristics of the 11 studies from which a direct relation of school leadership with achievement was examined. As can be seen in Table A.1 studies were conducted in primary education, and four studies in secondary education and in the two studies school leadership was examined at both the primary and secondary level. The countries in which the studies were employed varied. Four studies took place in the US, others in Paraguay, Cyprus, England, Spain and Belgium (Flemish Community). In two studies data were collected in more than one country (four Latin American countries and eight countries in different global regions). Methods of analysis used to investigate the direct relationships were correlation (two studies), regression analysis (three studies), multilevel modeling (six studies) and structural equation modeling (one study).

4 Results from School Leadership Effectiveness Studies Table 4.2 Vote counts direct effect studies based on replications (2005–2010)

Anderson (2008) Borden (2010) Horng et al. (2010) Kythreotis et al. (2010) Leithwood and Jantzi (2006) Louis et al. (2010) Martin et al. (2008) Miller and Rowan (2006) O’Donnell and White (2005) Pearson correlations O’Donnell and White (2005) Regression Opdenakker and Van Damme (2006) Shin et al. (2010) Totals (%)

69 Negative

Not significant

Positive

2 0 0 0 0 1 0 1 0

1 2 9 6 1 1 2 15 6

5 0 1 2 1 0 1 2 6

0

10

2

0

1

0

0 4 4

16 70 74

0 20 21

Note: significant at 0.05

The results of the vote counts for direct effect studies concerning school leadership are presented in Table 4.2. The results indicate that, out of 94 replications 74 of all direct relationships between school leadership and achievement examined are not significant, 20 are positively significant and 4 of the relationships are negatively significant. Compared to the vote counts of direct effect school leadership studies in the period until 2005 the percentage of positively significant relationships found in studies between 2005 and 2010 is relatively higher (21% in 2005–2010 and 8% until 2005) and the percentage of non-significant relationships is relatively lower (74% in 2005–2010 and 87% until 2005). Some technical side remarks with the results in Tables A.1 and 4.2 are in order to point at some intricacies of interpreting the outcomes. These concern the influence of relationships between the independent variables used in some of the studies. In the study by Horng et al. (2010) the independent variables measure the percentage of time spent on specific aspects of school leadership. Since each principal spends by definition 100% of his time on all aspects together, a relatively high percentage spent on one or a few specific aspects leads inevitably to relatively low values on at least one other aspect. This implies that when a positive effect exists on the proportion of time spent on at least one aspect, a negative effect will exist on at least one other aspect, since the average effect is zero. In Horng et al. the time spent by the principal is split up in six different task categories (Administration, Organization management, Day-to-day instruction, Instructional Program, Internal relations and External relations). For each task category an independent variable is defined, but since the time spent on these six

70


categories always add up to 100%, only five variables are used in the analyses (Administration is omitted). The five variables included in the analyses show a positive effect in both models on School performance (with the exception of one small negative effect in one occasion). This implies that the aspect which is not used in the analyses, in this case percentage of time spent on administration has a negative effect on School performance. In the present table we see one significant positive effect for Percentage of time spent on organization management and no other significant effects. When the researcher would have included Percentage of time spent on administration in the analyses and excluded Percentage of time spent on organization management she would likely have found a significant negative effect of Percentage of time spent on administration on School performance and no other significant effects. So the fact that a significant positive effect is found in the table does not mean that the principal has a positive effect on School performance but that it matters how the principal spends his time. A different situation of related variables, which occurs frequently in the field of educational research, is that variables show a positive correlation because they measure merely the same underlying dimension. This happens often when variables are composed from Likert scales. When the correlation between such variables is high regression models may suffer from multicollinearity, which means that in regression analyses it is hard to distinguish the contribution of each of the highly correlated variables. It may result in the situation that when two or three of such variables are involved only one of these happens to show a significant effect. It may also result in a situation where the effect of the underlying dimension is about equally distributed over the two or three variables involved. This may mean that none of these variables shows a significant effect, when all these variables are used in the same regression model, whereas each would have shown a significant effect if the other variables had not been included in the model. This seems to be the case in the study by by O’Donnell and White (2005). Three variables of Instructional Leadership Roles show significant positive correlations with two achievement measures. However, when these variables are entered in a regression model, only one of the three showed a significant effect for both achievement measures. It is likely that these indicators correlate rather highly, but this cannot be verified as the authors did not publish the correlations.

Conceptualizations of School Leadership Used in the Studies In Chap. 1 of this book the development of school leadership models is sketched and a decomposition of school leadership concepts is presented. The chapter ends with an overview of broad categories that will be used in this study to classify the operationalizations of leadership as measured in the direct (and indirect) effect studies.


71

The following categorization was used: • Instructional leadership: Managing the teaching and learning program (instructional leadership in the narrow sense). When direction setting (school mission) and developing a task-oriented culture (providing learning climate) is included next to managing the teaching and learning program it is proposed to speak of an extended perception of instructional leadership. • Transformational leadership: Conditions supporting school improvement: The main elements of this concept are: the modeling of organizational values, the development of a shared mission, the provision of intellectual stimulation, building consensus and redesigning the organizational structure. Distributed leadership is considered as a facet of transformational leadership. • Integrated leadership: Combination of transformational and instructional leadership. Table 4.3 gives an overview of the leadership variables used in the direct effect studies and their operationalization. As can be seen, in each publication a different operationalization of school leadership was applied. Also, as far as information is available, it seems that in each study researchers developed their own instrument to measure school leadership. The exception is the study by O’Donnell and White in which the Principal Instructional Management Rating Scale, developed by Hallinger (1983), cited in Hallinger and Heck (1996a), was used. In four publications the measurement of school leadership focussed on instructional leadership, in two studies an operationalisation of transformational leadership was used to measure school leadership and in three studies integrated leadership was considered. In the study by Louis et al. (2010) instructional and transformational leadership were used as two different variables. The rather limited description of the measurement of leadership in the publication by Martin et al. (2008) did not allow for a classification of their leadership variable used.

Meta-Analysis of Indirect Effect Studies Mediated or indirect effect models hypothesize school leaders to achieve their effect on school performance not only through a direct effect from school leadership to student achievement, but also through intermediate variables such as school organization and school culture (Hallinger and Heck 1998; De Maeyer et al. 2007). Hallinger and Heck (1998) organized the intermediate variables used in four domains: (1) Vision and goal setting, (2) Organizational structure, (3) Human capital and (4) Organizational culture and social networks. In the conceptual map of potential intermediary variables presented in Chap. 2 of this report four groups of effectiveness enhancing intermediate school and teaching variables were distinguished and seen as related to leadership behavior (see Table 4.4).

72


Table 4.3 Leadership variables used in direct effect studies and classification under leadership subheadings Study Leadership Leadership variable(s) subheading Anderson (2008)


Borden (2010)


Horng et al. (2010)


Five factors: Community relations: mainly representing a community relations role, including the variables: number of academic events, number of sports events, parent participation and the principal making community relations a top priority Meeting time: time spent in teachers’ meetings discussing discipline and parents Teacher relations: including variables that reflect relationships among teachers and between teachers and the principal: the amount of time teachers spent with each other discussing students, the principal’s experience as a teacher, suggesting empathy Academic focus: variables included reflected instructional leadership, whether or not teachers were given a preparation period during the day and the principal placing a priority on student evaluation, reflected an academic emphasis Stability in personnel (and indirectly a desirable working environment) with main variables the number of years the principal has been at the school and the percentage of teacher turnovera Principal’s activities’ inside’ the classroom: uses clear criteria to evaluate teacher performance, is knowledgeable about pedagogy, visits classrooms to observe the teaching and learning process, uses exam results to make program change recommendations Principal’s activities’ outside’ the classroom’ (two scales) 1. Expectations: demonstrates a commitment to academic objectives, considers all students capable of learning, has high expectations of teachers, has a long-term vision for the school, maintains high standards of student behavior 2. Support: consults with others, makes teachers proud to be part of the school, recognizes the school’s problems, creates an orderly and disciplined environment, enjoys working with students Principal’s actions (coded in six task categories): Administration (e.g. fulfilling compliance requirements, managing student discipline, managing student attendance, preparing and implementing standardized tests) (continued)

4 Results from School Leadership Effectiveness Studies Table 4.3 (continued) Study Leadership subheading

Kythreotis et al. (2010)


Leithwood and Jantzi (2006)


73

Leadership variable(s) Organization management (e.g. managing budgets, resources, hiring personnel, dealing with concerns from teachers, networking with other principals) Day-to-day instruction (e.g. informally coaching teachers to improve instruction, formally evaluating teachers, conducting classroom observations, implementing required professional development) Instructional program (e.g. developing an educational program across the school, evaluating curriculum, using assessment results for program evaluation and development, planning or directing supplementary of after school instruction) Internal relations (e.g. developing relationships with students, communicating with parents, interacting socially with staff about school-related topic, counseling staff) External relations (e.g. working with local community members or organizations, fundraising, communicating with the district office to obtain resources) Principal’s leadership styles: The structural principals’ leadership style: emphasis on goals, planning and coordination The human resources principals’ leadership style: principal sensitive to the human needs The political principals’ leadership style: recognition of the ways that people seek to advance their own interests The symbolic principals’ leadership style: with a focus on the rituals, myths and ceremonies that give meaning to organizational culture Transformational leadership: Including three broad categories of leadership practices Setting directions: building school vision, developing specific goals and priorities, and holding high performance expectations Developing people: providing intellectual stimulation, offering individualized support, modeling desirable professional practices and values Redesigning the organization: developing a collaborative school culture, creating structures to foster participation in school decisions, creating productive community relationships (continued)

74


Table 4.3 (continued) Study Leadership subheading Louis et al. (2010)



Martin et al. (2008)

Miller and Rowan (2006)


O’Donnell and White (2005)


Leadership variable(s) Instructional leadership: supportive behaviors and direct coaching or modeling. Example items: How often in this school year has your school administrator discussed instructional issues with you? How often in this school year has your school administrator observed your classroom instruction? How often in this school year has your school administrator attended teacher planning meetings? How often in this school year has your school administrator made suggestions to improve classroom behavior or classroom management? Shared leadership: teachers’ influence over and participation in school-wide decisions. Example items: teachers have an effective role in schoolwide decision making. Teachers have significant input into plans for professional development and growth. School’s principal ensures wide participation in decisions about school improvement. How much direct influence do students have on school decisions? Administration’s leadership: the type of procedures that the school administration follows, their organizational environment and their pedagogical coordination. Example items: ‘‘In general, the administration makes appropriate decisions at the right time’’ and ‘‘The principal should listen to the teachers more’’ Organic management (three measures): Supportive leadership (example items: goals/priorities for the school are clear, encouraged to experiment with teaching, administrative behavior is supportive, rules for student behavior are enforced) Teachers’ control over key instructional decisions (example items: influence over discipline policy, influence over grouping students by ability, influence over establishing curriculum) Amount of staff collaboration present in the school (example items: colleagues share beliefs about mission, teachers at this school are continually learning, great deal of cooperative effort among staff) Instructional leadership was defined as the behaviors and tasks included in three dimensions of principal’s instructional leadership: 1. Defining the school mission 2. Managing the instructional program, and 3. Promoting the school learning climate Principal instructional management rating scale (PMRS) (continued)

4 Results from School Leadership Effectiveness Studies Table 4.3 (continued) Study Leadership subheading Opdenakker and Van Integrated Damme (2006) leadership

Shin et al. (2010)


75

Leadership variable(s) Participative professionality-oriented leadership includes two types of tasks: 1. The extent to which the school leader is involved with educational tasks and 2. The extent to which teachers participate in decision making at school Instructional leadership: The time spent on instructional leadership (e.g., developing curriculum and pedagogy) and the time spent on supervising and evaluating teachers and other staff

a This factor is not included in the Table 4.2 as it does not refer to instructional, transformational or integrated leadership

Table 4.4 Conceptual map of leadership behaviors and potential intermediary variables Leadership behavior Effectiveness enhancing school and teaching factors External contacts Buffering Direction setting (goals, standards) Monitors curriculum and instruction (managing the instructional program)

HRM & HRD Coaches teachers Recruits teachers Builds consensus Sets values Creates climate

Enhanced teaching time Clear goals and standards Opportunity to learn Student monitoring and feedback Structured teaching Active teaching Active learning Cohesion among teachers Professionalization Teacher competency Teachers’ sense of self efficacy Shared sense of purpose among teachers High expectations Disciplinary climate Supportive climate

The literature search for indirect effect studies resulted into 15 publications published between 2005 and 2010 (see Tables A.2, A.3 attached at the end of this chapter). Six studies examined indirect effects of leadership in primary school contexts, four in secondary schools and five studies included both primary and secondary schools. Six studies were conducted in the US, four in Canada, two in the Flemish Community of Belgium and one in each of England and the Netherlands. One study was based on data from 14 OECD countries participating in TIMSS 2007. In all studies structural equation modeling was used to examine the direct and indirect effects of leadership on achievement. In almost all studies, the design

76


included control for student background effects, either through the use of gain scores or covariates. In Table A.2 an overview is presented of the variables used in the studies: the independent [leadership variable(s)], the antecedent and contextual variables, the intermediate variables and the dependent variable(s) used in the studies. As can be noticed, in more than half of the studies the indirect effect models include intermediate variables at more than one level. Table A.3 provides an overview of direct and indirect effects. All paths between leadership and achievement for which (in) direct effect size statistics were available or could be calculated are included in Table A.3. In the last column of Table A.3 for each leadership variable the total effect size is presented (see also Table 4.5 for a summary). As stated above, the total effects in Tables A.2 and A.3 were copied from the publication, whenever possible. In case these total effects were not explicitly published, they were computed from the path diagrams in the publications (see Annex 3), by adding the individual effects of each path in the diagram. The individual effect of a single path is computed by multiplying all effects included in that path. There may be (usually minor) differences between published total effects and total effects computed from the diagram, especially where the path diagrams only mention significant effects. Table 4.5 summarizes the total effects of all 34 replications, found in 15 publications. The mean magnitude of the total effects equals 0.031, which does not deviate significantly from 0, given a standard error of 0.20. However, when we exclude the publication from Ten Bruggencate (2009), which at one hand contains some highly negative effects (-0.31, -0.18 and -0.16, respectively, see Table 4.5) and on the other hand more replications (6) than all other publications in the table, the mean would be 0.060, which deviates significantly from 0 with a standard error of 0.18. This shows that including or excluding one publication can largely affect the conclusions. Studies in which relatively high effect sizes were found are the studies by Leithwood and Jantzi (2008), Leithwood and Mascall (2008) and Heck and Moriyama (2010). The study by Leithwood and Jantzi (2008) will be described shortly below. The study by Heck and Moriyama (2010) is presented in Chap. 3. The other study with relatively high effects sizes [the study by Leithwood and Mascall, (2008)] does not include information about the direct and indirect effects from collective leadership to achievement and therefore will not be described below. The study by Leithwood and Jantzi (2008) aimed to improve the understanding of direct and indirect effects of school leader efficacy on student learning. The study was part of the larger project Learning form District Efforts to Strengthen Instructional leadership, a multi-year mixed-methods study. Data were collected by means of a principal and teacher survey (96 principals and 2,706 teachers). In the principal survey principals were asked both about School leader self-efficacy (LSE) for improving instruction and student learning


77

Table 4.5 Summary of total effects sizes in indirect effect studies Author and Year Leadership measure Achievement measure Day et al. (2009)

Heck and Hallinger (2009) Heck and Hallinger (2010)

Integrated leadership (primary level) Integrated leadership (secondary level) Initial distributed leadership

Change in pupil outcomes over 3 years Idem

0.001

Growth rate Math

0.03

Change in leadership Distributed leadership

Idem Initial Reading scores (year 2) Initial Math scores (year 2) Growth rate Reading Growth rate Math Added year effect Reading

0.09 0.02

Idem

Heck and Moriyama (2010) Leithwood and Jantzi (2008)

Change in leadership Idem Collaborative leadership

Idem Integrated leadership: School leadership

Leithwood et al. (2006) Leithwood and Mascall (2008)

School leadership Idem Collective leadership

Leithwood et al. (2010)

Distributed leadership

Louis et al. (2010)

De Maeyer et al. (2007) Opdenakker and Van Damme (2007) Ross and Gray (2006) Supovitz et al. (2010)

Total effect

Idem Instructional leadership

Shared leadership Integrated leadership

0.04

0.02 0.10 0.10 0.16

Added year effect Math Proportion of students reading or exceeding the state’s proficient level 2 year mean achievement score 2 year mean achievement gain Percentage of students meeting or exceeding the proficiency level on language and math tests Percentage of students per school achieving level 3 or higher at math and literacy test Idem Percentage of students at school level meeting or exceeding the proficiency level 2005 math tests Idem Reading

0.14 0.24

0.11 -0.06 0.24

0.11

0.15 0.05

0.03 -0.02

Idem Participative professionally oriented leadership

Math Math

-0.16 0.006

Transformational leadership Principal leadership

Composite school score

0.22

English Language and Arts

0.03

Idem

Math

-0.009 (continued)

78


Table 4.5 (continued) Author and Year Leadership measure Ten Bruggencate (2009)

Ten Bruggencate et al. (2010)

Mean

Leadership style: rational goals (teacher perceptions) Leadership style: internal Process (teacher perceptions) Leadership style: human relations (teacher perceptions) Leadership style: open systems (teacher perceptions) Leadership style: rational goals (principal perceptions) Leadership style: open systems (principal perceptions) Time spent on instructional leadership Time spent on administrative duties Time spent on supervising teachers Time spent on public relations 15 publications; 34 effect measures

SE mean Without Ten Bruggencate (2009) Mean 14 Publications; 28 effect measures SE mean

Achievement measure

Total effect

Average exam mark

-0.16

Idem

0.003

Idem

0.004

Idem

Idem

Idem

Math (TIMSS)

-0.18

0.002

-0.31

0.02

Idem

-0.09

Idem

0.09

Idem

0.04 0.031 (0.020) 0.060 (0.018)

and about the Collective capacity of colleagues across schools in the district to improve student learning (LCE). Next to this, principals were asked about District conditions and District leadership. In the teacher survey three variables were included: School leadership, Class conditions and School conditions. School leadership was measured by asking questions about four categories: setting directions, developing people, redesigning the organization and managing the instructional program. To measure School conditions four variables were chosen from five broad sets of school conditions resulting form a literature review. These variables are:


79

• ‘‘School culture (minimize disruptions to instructional time, shared beliefs and values, students feel safe in the school). • Decision making processes (e.g., teachers’ participation, feedback to teachers about instructional practices, autonomy in the classroom decisions, teacher input into school plans). • Supports for instruction (e.g., adequate time for professional development, sufficient support for students with special needs, adequate curriculum materials). • Professional learning community (four of the five interconnected variables in Louis and Kruse’s (1995) conception of professional learning communities— shared responsibilities, beliefs about teaching and learning, deprivatized practice, and reflective dialogue)’’ (Leithwood and Jantzi 2008, p. 520). To measure class conditions the following four conditions were measured • Workload (e.g., manageable numbers of students, manageable number of subjects taught, access to teaching assistance when needed). • Areas of formal preparation (e.g., teaching subjects in which I am formally prepared, teaching grade levels for which I am formally prepared). • Student grouping (e.g., group students according to need, depends on instructional purposes). • Curriculum and instruction (e.g., sufficient written curricula on which to base lessons, enable students to construct their own knowledge, school has a rigorous core curriculum for most students) The other two conditions that resulted from the literature review (attention to class size and homework) were omitted. The data used to measure achievement were school-wide results on state mandated tests of language and mathematics at several grade levels over 3 years (2003–2005). A school’s student achievement was represented by the percentages of students meeting or exceeding the proficiency level on language and math tests. These percentages were averaged across grades and subjects and resulted in a single achievement score for each school for each of 3 years. Data analysis methods included Pearson product correlations, standard multiple regression, hierarchical multiple regression, analysis of variance and LISREL (linear structural relations). The results of testing the structural model are indicated in the path model in Fig. 4.1, as reprinted from Leithwood and Jantzi (p. 520). In the path model only significant relationships are shown. The total effect from school leadership on student achievement is significant and equals 0.24. The model indicates that School leadership (SLEADER) is positively related with school conditions. Direct effects as well as indirect effects from school leadership to student learning are found. However, the relationship between class conditions and student achievement is not significant and negative (-0.07) and this has not been included in the model presented in Fig. 4.1. The only explanation

80

M. Hendriks and R. Steen LCE .50*

District .77* Leadership

.26*

District Conditions .66*

LSE .32*

School .87* Class Conditions Conditions .66*

Achievement

.40*

SLEADER Fit indices RMSEA .02 RMR .05 AGFI .89 NFI .96 X2 = 13.88 , df= 13, P=.38

Standardized Total Effects on Student Achievement District Leadership .13* District Conditions .17* Leader Collective Efficacy .32* Leader Self Efficacy -.01* School Leader Behavior .24* School Conditions .34* Class Conditions -.07

Fig. 4.1 Modeling the relationship among variables related to leader self efficacy (LSE) and leader collective efficacy (LCE) in the study by Leithwood and Jantzi 2008

the researchers could find for the negative and non-significant relationship between class conditions and achievement was the rather low reliability of the scale used to measure Class conditions (a = 0.60).

Conceptualization of School Leadership in the Indirect Effect Studies Transformational leadership and Integrated leadership are the subheadings under which the operationalizations of leadership variables used in almost all reviewed indirect effect studies could be positioned (see Table 4.6). In almost all reviewed studies the measurement of school leadership includes aspects of transformational leadership. In half of these studies integrated leadership is measured which means that in addition to transformational leadership practices or styles, aspects of instructional leadership are also included in the measurement of leadership.

Promising Paths and Intermediate Variables in Indirect Effect Models In Table 4.7 an overview is given of the most promising paths in the indirect models reviewed in this study. In the remainder of this section these paths are described shortly and the operationalisation of the intermediary variables is referred to as well. The section ends with a comparison of the intermediate variables used in the studies with the (four groups of) effectiveness enhancing


81

Table 4.6 Leadership variables used in indirect effect studies and classification under leadership subheadings Study Leadership Leadership variable(s) used in the study subheading Day et al. (2009)


Heck and Hallinger (2009)




Leadership: ‘setting directions’, ‘redesigning organization’ and ‘head trust’ (plus three other dimensions: ‘use of data’, ‘developing people’ and ‘use of classroom observation’) (Example items are: ‘demonstrating high expectation’s for staff work, promoting a range of continuous professional development experiences among all staff, improving internal review procedures, encouraging staff to use data in their work and regularly observing classroom activities) Distributed leadership: three aspects Make collaborative decisions focusing on educational improvement (i.e., ensure teachers have a major role in decisions about curriculum development in the school; enable administrators, teachers, and staff work together effectively to achieve school goals); Emphasize school governance that empowers staff and students, encourage commitment, broad participation, and shared accountability for student learning (i.e., provide opportunities for parents to participate in important decisions about their children’s education through a variety of venues; ensure teachers can freely express input and concerns to the administrators; provide opportunities for teachers to plan and make school decisions); and: Emphasize participation in efforts to evaluate the school’s academic development (i.e., ensure adequate resources are available to the school to develop its educational programs; provide regular opportunities for all stakeholders to review the school’s vision and purpose) Distributed leadership: three aspects School improvement (Example items: to what extent does school leadership: make decisions to facilitate actions that focus the energies of the school on student achievement and school-wide learner outcomes; empower staff and students; encourage commitment, participation and shared accountability for student learning?); School governance (i.e., adopt governance guidelines which are consistent with the school’s purpose and support the achievement of the state standards and the school-wide learner outcomes?); and (continued)

82


Table 4.6 (continued) Study Leadership subheading

Heck and Moriyama Transformational (2010) leadership





Leadership variable(s) used in the study Resource management and development (i.e., Allocate available resources in a manner that sustains the school program and are used to carry out the school’s purpose; use assessment results as the basis for the allocation and use of resources? Collaborative leadership: three specific aspects (items paraphrased in parentheses): Shared school governance which encourages shared vision, broad participation and shared accountability for student learning (e.g., administrators, teachers and staff work together effectively to achieve school goals; teachers can freely provide input and express concerns to administrators); Collaborative decisions focusing on academic improvement (e.g., school ensures teachers have a major role in decisions about curriculum development; school provides opportunities for teachers to plan and make school decisions about professional development and curriculum); Broad participation in efforts to evaluate the school’s academic development (e.g., school provides regular opportunities for all stakeholders to review the school’s vision and purpose; assesses progress in making school changes) School leadership Enabling school improvement teams to carry out responsibilities by providing resources and support, and by empowering teams Instrumental in initiating school improvement process of and participation in school improvement planning, in ensuring its continuation as a priority. principals who act as facilitative leaders encourage successful implementation of school improvement process Teachers must be energetic and responsibility enthusiastic about school improvement process as they shoulder much of the for planning and implementation. Teachers are in the majority on most school improvement teams. They are very influential in the SIP process School leadership Setting directions Developing people Redesigning the organization Managing the instructional program (continued)

4 Results from School Leadership Effectiveness Studies Table 4.6 (continued) Study Leadership subheading Leithwood and Mascall (2008)




Louis et al. (2010)


83

Leadership variable(s) used in the study Collective leadership (nine items) Each item used to measure collective leadership concerned a single source of influence, including district administrators, principals, other school administrators, some individual teachers, teachers with designated leadership roles, staff teams, some individual parents, parent advisory groups and students. Regarding each source of influence, respondents were asked to rate the extent of direct influence on school decisions Leadership is conceptualized and measured as a set of practices distributed among staff with a focus on managing and leading the instructional program. The 1st item in the scale assessed the extent of leadership distribution in the school (‘‘Leadership in this school is provided by many teachers and administrators’’). One item was about direction-setting practices (‘‘Leaders clearly communicate expected standards for literacy and math’’), two items were about practices aimed at developing people (e.g., ‘‘Leaders provide quality staff development opportunities…’’), and one item concerned organizational design (‘‘Leaders provide us with opportunities to collective discuss student work’’). All remaining items (12) were practices aimed at managing the instructional program (e.g., ‘‘Leaders provide me with useful feedback about my instruction’’; ‘‘Leaders provide me with information about instruction that is useful to me in the classroom’’) Instructional leadership To measure instructional leadership supportive behaviors as well as direct coaching or modeling were chosen. The scale consisted of seven items. Example items are: How often in this school year has your school administrator discussed instructional issues with you? How often in this school year has your school administrator observed your classroom instruction? How often in this school year has your school administrator attended teacher planning meetings? How often in this school year has your school administrator made suggestions to improve classroom behavior or classroom management? (continued)

84


Table 4.6 (continued) Study Leadership subheading De Maeyer et al. (2007)


Opdenakker and Van Integrated Damme (2007) leadership

Ross and Gray (2006)


Supovitz et al. (2010)


Ten Bruggencate (2009)


Leadership variable(s) used in the study Integrated leadership Instructional leadership Managing the instructional program (Evaluation of pupils and teachers) Climate-oriented (learning time, promoting professional development, valuing the expertise of each teacher) Transformational leadership Oriented to a shared vision, achievement-oriented, culture-oriented (creating structures for participation in decision making with regard to school policy) (Based on De Maeyer and Rymenans 2004) The scales concerning School leadership are the Amount of educational tasks the school leader is involved with (e.g. coaching new teachers, stimulating teachers to implement educational innovations and to reflect in group on the educational goals of their school, discussing new teaching methods and material with teachers and supporting school personnel while preparing and implementing school task) and the Amount of teacher participation in the decision-making at school (e.g. the extent to which teachers participate in decision-making at school and the extent to which consultation between school leader and teachers takes place) Transformational leadership: 12 items measuring teacher perceptions that their principal leads by developing the capacity of the organization and its members to adapt to the demands of a changing environment Principal leadership Mission and goals (The extent to which the principal has an instructional mission and related goals) Principal trust (teachers have a trusting relationship with their principal) Focus on instruction (principal expertise and focus on instruction) School leadership styles Rational goals (goal setting) Internal process (maintenance of law and order and establishment of procedures) Human relations (supporting teachers) Open systems (external contacts and acquisition of resources) (continued)

4 Results from School Leadership Effectiveness Studies Table 4.6 (continued) Study Leadership subheading Ten Bruggencate et al. (2010)


85

Leadership variable(s) used in the study Time spent on: Administrative leadership Instructional leadership (developing curriculum and pedagogy) Supervising teachers and other staff Public relations and fundraising

intermediate school and teaching variables considered as promising intermediate variables between school leadership and achievement (Chap. 1 of this report, Fig. 1.4).

Day et al. (2009) (Secondary Level) The most relevant intermediating factor between Integrated leadership and Change in pupil outcomes over 3 years is Leadership distribution in the school. The effect of this path equals 0.035. The dimension Leadership distribution consists of four factors: Distributed leadership, Staff collaboration, School Management Team (SMT) collaboration and SMT’s impact on teaching and learning (Day et al. 2009).

Heck and Hallinger (2009) The most relevant intermediating factor between initial distributed leadership and reading growth rate is capacity for change. The effect of this path equals 0.025. The most relevant intermediating factor between change in leadership and Math growth rate is also capacity for change. The effect of this path equals 0.083. In the study (school academic) capacity was formed by combining four subscales: • Standards emphasis and implementation (example items are: school’s educational programs are aligned to the State content and performance standards; Teaching and learning activities are focused on helping students meet the State content and performance standards; School prepares students well for the next school; Students and parents are informed about what students are expected to learn; School has high academic and performance standards for students); • Focused and sustained action on improvement (example items are: The school clearly communicates goals to staff, parents and students; Vision and purpose are translated into appropriate educational programs for children; School seeks

86


Table 4.7 The most relevant paths found in indirect effect models Author and Year

Leadership measure

Achievement measure

Intermediate variables

Day et al. (2009)

Integrated leadership (secondary level)

Leadership distribution in the school

0.04


Initial distributed leadership Change in leadership Distributed leadership

Change in pupil outcomes over 3 years Growth rate Math

Change in capacity

0.03

Growth rate Math Initial Reading scores (year 2) Initial Math scores (year 2) Growth rate Reading Growth rate Math Added year Effect Reading Added year Effect Math Proportion of students reaching or exceeding the state’s proficient level 2 year mean achievement score 2 year mean achievement gain Percentage of students meeting or exceeding the proficiency level on language and math tests Percentage of students per school achieving level 3 or higher at math and literacy test Idem

Change in capacity School capacity

0.08 0.02

School capacity

0.02

Change in capacity Change in capacity Instructional practices

0.10 0.13 0.16

Instructional practices

0.14

School conditions

0.24

(not reported)

0.11

(not reported)

-0.06

(not reported)

0.24

Not clear from publication (model 1)

0.11

Not clear from publication (model 2) Focused instruction

0.15




Distributed leadership Change in leadership Change in leadership Collaborative leadership Collaborative leadership Integrated leadership: School leadership

School leadership Idem

Leithwood and Mascall (2008)

Collective leadership




Louis et al. (2010)

Instructional leadership Percentage of students at school level meeting or exceeding the proficiency level 2005 math tests

Combined effect via this path

0.03

(continued)


87

Table 4.7 (continued) Author and Year

Leadership measure

Achievement measure

Instructional leadership Idem

Maeyer et al. (2007)



Professional community and Focused instruction Professional community and Focused instruction None

Combined effect via this path 0.02

0.03

Shared leadership

Idem


Reading

Integrated leadership Integrated leadership Transformational leadership Transformational leadership

Reading Academic climate Math None Composite school score Teacher commitment to school mission Composite school score Collective teacher efficacy and Teacher commitment to community partnerships Composite school score Teacher commitment to professional community English Language and Change in instruction Arts Average exam mark None

-0.16

Average exam mark

None

-0.18

Average exam mark

None

-0.32

Math achievement

Valuing Math

Math achievement

None

-0.04

Math achievement

Valuing Math

-0.02

Math achievement

Topic coverage (OTL)

-0.03

Transformational leadership Supovitz et al. (2010) Ten Bruggencate (2009)


Principal leadership Leadership style: Rational goals (teacher perceptions) Leadership style: Open systems (teacher perceptions) Leadership style: Open systems (principal perceptions) Time spent on instructional leadership Time spent on administrative duties Time spent on administrative duties Time spent on administrative duties

-0.27 0.25 -0.15 0.13 0.12

-0.07

0.02

0.02

(continued)

88


Table 4.7 (continued) Author and Year

Leadership measure

Achievement measure



Time spent on supervising teachers Time spent on supervising teachers Time spent on supervising teachers Time spent on public relations

Math achievement

None

0.04

Math achievement

Valuing Math

0.01

Math achievement


0.03

Math achievement


0.04

ways to improve its programs and activities that promote student achievement; Teachers know what the school learner outcomes are); • Quality of student support (example items are: Standards exist for student behavior; Discipline problems are handled quickly and fairly; School environment supports learning; Open communication exists among administrators, teachers, staff and parents; Teachers feel safe at school; Teachers and staff care about students); • Professional capacity of the schooll (example items are: Teachers are well qualified for assignments and responsibilities; Leadership and staff are committed to school’s purpose; Staff development is systematic, coordinated and focused on standards-based education; Systematic evaluation is in place (Heck and Hallinger, p. 671). Capacity for change over time refers to growth on the subscales of academic capacity between year 1 and 3 of the study (Heck and Hallinger 2009).

Heck and Hallinger (2010) The most relevant intermediating factor between Change in leadership and Reading growth rate is also Change in school improvement capacity. The effect of this path equals 0.10. The most relevant intermediating factor between Change in leadership and Math growth rate is also Change in school improvement capacity. The effect of this path equals 0.13. The mediating variable School improvement capacity encompasses 7 subscales. The first four subscales are also used in Heck and Hallinger (2009). The subscales are:


89

• Standards emphasis and implementation (example items: see Heck and Hallinger (2009) above); • Focused and sustained action on improvement (example items: see Heck and Hallinger (2009) above); • Quality of student support (example items: see Heck and Hallinger (2009) above); • Professional capacity of the school (example items: see Heck and Hallinger (2009) above); • School communication (example items are: school employs a wide range of strategies to ensure parent involvement; open communication among staff; open communication exists between school staff and parents; school keeps parents informed; I encourage and welcome parents to come to my classroom); • Stakeholder involvement (example items are: parents participate in important decisions about their children’s education; school involves parents in classrooms such as tutoring students or checking homework; school encourages parent involvement in a variety of ways); and • Student safety and well being (example items are: The school is orderly and supports learning; school staff shows that they respect and care about students; students can receive extra help and support when needed) (Heck and Hallinger 2010).

Heck and Moriyama (2010) The most relevant intermediating factor between Collaborative leadership and Added year effect Reading is change in Instructional practices. The effect of this path equals 0.16. The most relevant intermediating factor between Collaborative Leadership and Added year effect Math is Change in instructional practices. The effect of this path equals 0.14. School instructional practices: The four dimensions used to define Instructional practices are classroom teaching conditions, quality of student support system, • Classroom teaching conditions: This dimension is defined by three subdimensions: Opportunity to learn (i.e. curriculum is based on state content and performance standards, students learn how to assess their own progress and set own learning goals, school has high academic standards and expectations for all students), Quality of instruction (i.e., instruction oriented toward developing student cognitive skills, teacher uses a variety of teaching strategies and activities, students receive feedback on progress and suggestions for improvement, teacher uses assessments to adjust instruction) Use of time (instructional time is flexible and organized to support learning). • Quality of student support system. This dimension is defined as creating a learning environment that supports learning (e.g., school environment supports learning, discipline problems are handled quickly and fairly, teachers provide

90


extra help for students who need it, open communication exists among teachers and administrators, school is well maintained) • Professional capacity: the extent to which the teaching staff has needed instructional expertise and is able to participate in quality professional development focused on improving instruction, and also the presence of systematic evaluation in place to help teachers grow professionally. • Focused and sustained action on improvement: example items are school communicates its goals to staff, parents and students; school has high academic standards for all students; changes in curriculum and instructional practices are coordinated and school wide; school learner outcomes are widely known; respondent is personally involved in the school improvement process) (Heck and Moriyama 2010).

Leithwood and Jantzi (2008) The variable School conditions is the most relevant intermediating factor between Integrated Leadership and Proportion of students reaching or exceeding the state’s proficient level. The effect of this path equals 0.24. The variable ‘School conditions’ encompasses four subvariables: • School culture (minimize disruptions to instructional time, shared beliefs and values, students feel safe in the school). • Decision-making processes (e.g., teachers’ participation, feedback to teachers about instructional practices, autonomy in the classroom decisions, teacher input into school plans). • Supports for instruction (e.g., adequate time for professional development, sufficient support for students with special needs, adequate curriculum materials). • Professional learning community (shared responsibilities, beliefs about teaching and learning, deprivatized practice and reflective dialog) (Leithwood and Jantzi 2008).

Leithwood, Patten and Jantzi (2010) In both models 1 and 2 the total effects from Leadership practices on student achievement are noticeable (for both models 0.11 and 0.15, respectively). The most relevant intermediating variable seems to be factor Rational path. However, the path model diagrams and the total effects on student achievement in the accompanying tables are not clear about the influence of the most relevant intermediating variables. In the study the variables on the Rational path are rooted in the knowledge and skills of school staff members about curriculum, teaching and learning. In model 2


91

of the study Academic press and Disciplinary climate are the variables that represent the Rational path (Leithwood et al. 2010).

Louis et al. (2010) Focused instruction is the most relevant intermediating factor between Instructional Leadership and Mean Math achievement, both with a direct effect from Instructional Leadership and an indirect effect via professional community. The combined effect equals 0.05 (Louis et al. 2010). The variable Focused instruction reflected previous analysis from the researchers that suggested ‘‘that teachers may, in their own work, be bridging the scholarly debates between constructivist and direct instruction by developing strategies that are designed to incorporate elements of both’’ (p. 322). Example items are: My instructional strategies enable students to construct their own knowledge, I maintain a rapid pace of instruction in my classes, I focus on developing a deep knowledge of the core subjects that I teach. The variable Teacher’s professional community examines the nature of teacher’s relationships with each other. Example items are: In our school we have well-defined learning expectations for all students, Our student assessment practices reflect our curriculum standards, How many teachers in this school take responsibility for improving the school outside their own class?, How often in this school year have you visited other teachers’ classrooms to observe instruction?

De Maeyer et al. (2007) For Reading, the direct effect of Integrated leadership on achievement of -0.27 is largely neutralized by the indirect effect via academic cimate of +0.25. So the total effect is -0.02. For Math, the direct effect of Integrated leadership on achievement of -0.15 is not neutralized by indirect effects. In schools scoring high on Academic climate, the school policy lays a strong emphasis on the performance of their pupils. This emphasis is shared by teachers and the principal and results in shared values within the school’s staff (De Maeyer et al. 2007).

Ross and Gray (2006) There are three intermediating factors from almost the same magnitude between Transformational leadership and a composite school score. These are Teacher commitment to school mission, with a contribution of 0.14, Teacher commitment to Community partnerships (mostly indirect via Collective teacher efficacy), with

92


a contribution of 0.16 and Teacher commitment to professional community with a negative contribution of -0.09. The latter negative contribution does not necessarily mean that Teacher commitment to professional community is indeed unproductive, but merely that other ways of Teacher commitment contribute more toward achievement. Teacher commitment to the school mission measures teachers’ acceptance of school goals, their belief that these goals were shared by the staff, and their commitment to reviewing school goals regularly. Teacher commitment to the school as a professional community represents teachers’ commitment to sharing teaching ideas with each other. Teacher commitment to the school-community partnerships measures teacher commitment to including parents in setting directions. Collective teacher efficacy is a specific form of self-efficacy in which the target of the beliefs is the organization to which the individual belongs, i.e., ‘‘the perceptions of teachers in a school that the efforts of the faculty as a whole will have a positive effect on students’’ (Ross and Gray 2006).

Supovitz (2008) The most relevant intermediating factor between Principal leadership and English Language Arts student learning is Change in instruction. The effect of this path equals 0.02. Change in instruction is composed of four items that asked teachers about the degree to which they had changed various aspects of their teaching (i.e. teaching methods, assigned student work, kind of questions asked, understanding of student needs) (Supovitz 2008).

Ten Bruggencate (2009) Leadership style rational goals (teacher perception) and Leadership style open systems (teacher perception) both show a direct negative effect of -0.17 and -0.18 respectively on the average exam mark, while there are only negligible indirect effects for these Leadership styles and only negligible direct and indirect effects for both Leadership style internal process (teacher perception) and Leadership style human relations (teacher perception) (Ten Bruggencate 2009).

Ten Bruggencate et al. (2010) The most relevant effects on Math achievement of how a school leader spends time on specific duties are a positive effect of 0.09 for time spent on supervising


93

Table 4.8 Conceptual map of leadership behaviors, potential intermediary variables and intermediate variables in promising paths Leadership behavior

Effectiveness enhancing school and teaching factors

External contacts buffering Direction setting (goals, standards) Monitors curriculum and instruction (managing the instructional program)

Enhanced teaching time Clear goals and standards

Opportunity to learn

Student monitoring and feedback

Structured teaching

Active teaching

HRM and HRD Coaches teachers Recruits teachers Builds consensus

Active learning Cohesion among teachers

Professionalization

Teacher competency

Teachers’ sense of self efficacy

Intermediate variables in promising paths

Change in school academic capacity (Heck and Hallinger 2009) Change in school improvement capacity (Heck and Hallinger 2010) School instructional practices (Heck and Moriyama 2010) Teacher’s professional community (Louis et al. 2010) Teacher commitment to the school mission (Ross and Gray 2006) School instructional practices (Heck and Moriyama 2010) Topic coverage (Ten Bruggencate et al. 2010) Change in school academic capacity (Heck and Hallinger 2009) Change in school improvement capacity (Heck and Hallinger 2010) School instructional practices (Heck and Moriyama 2010) School conditions (Leithwood and Jantzi 2008) School instructional practices (Heck and Moriyama 2010) Focused instruction (Louis et al. 2010) Change in instruction (Supovitz 2008) School instructional practices (Heck and Moriyama 2010) Focused instruction (Louis et al. 2010) Change in instruction (Supovitz 2008) School conditions (Leithwood and Jantzi 2008) Teacher’s professional community (Louis et al. 2010) Teacher commitment to the school as a professional community (Ross and Gray 2006) Change in school academic capacity (Heck and Hallinger 2009) Change in school improvement capacity (Heck and Hallinger 2010) School instructional practices (Heck and Moriyama 2010) School conditions (Leithwood and Jantzi 2008) Change in school academic capacity (Heck and Hallinger 2009) Change in school improvement capacity (Heck and Hallinger 2010) School instructional practices (Heck and Moriyama 2010) Collective teacher efficacy (Ross and Gray 2006)

(continued)

94


Table 4.8 (continued) Leadership behavior

Effectiveness enhancing school and teaching factors

Intermediate variables in promising paths

Sets values creates climate

Shared sense of purpose among teachers

Change in school academic capacity (Heck and Hallinger 2009) Change in school improvement capacity (Heck and Hallinger 2010) School instructional practices (Heck and Moriyama 2010) School conditions (Leithwood and Jantzi 2008) Teacher’s professional community (Louis et al. 2010) Academic climate (De Maeyer et al. 2007) Teacher commitment to the school mission (Ross and Gray 2006) Change in school academic capacity (Heck and Hallinger 2009) Change in school improvement capacity (Heck and Hallinger 2010) School instructional practices (Heck and Moriyama 2010) Teacher’s professional community (Louis et al. 2010) Academic climate (De Maeyer et al. 2007) Change in school academic capacity (Heck and Hallinger 2009) Change in school improvement capacity (Heck and Hallinger 2010) School instructional practices (Heck and Moriyama 2010) School conditions (Leithwood and Jantzi 2008) Change in school academic capacity (Heck and Hallinger 2009) Change in school improvement capacity (Heck and Hallinger 2010) School instructional practices (Heck and Moriyama 2010) School conditions (Leithwood and Jantzi 2008)

High expectations

Disciplinary climate

Supportive climate

teachers (with a direct effect of 0.04 and combined indirect effects of 0.05) and a negative effect of -0.09 for Time spent on administrative duties (with a direct effect of -0.04 and combined indirect effects of -0.05). Combined indirect effects refer to indirect paths by two variables: Valuing mathematics and Topic coverage (Opportunity to learn) (Ten Bruggencate et al. 2010). In Table 4.8 the intermediate variables in promising paths of the indirect effect models are grouped according to the conceptual map of potential intermediary variables as developed in Chap. 1 of this report. The comparison in Table 4.8 shows that intermediate variables used in the studies by Heck and Hallinger (2009); Heck and Hallinger (2010a); Heck and Moriyama (2010) and Leithwood and Jantzi (2008) cover a broad spectrum of effectiveness enhancing school factors.


95

In other studies the intermediate variables were more focused on specific effectiveness enhancing variables. In the study by De Maeyer et al. (2007) the intermediate variable Academic climate was limited to climate and values (high expectations and shared sense of purpose among teachers). In the study by Ten Bruggencate et al. (2010) the intermediate variable related to effectiveness enhancing school teaching factors is Topic coverage (Opportunity to learn). In the study by Louis et al. (2010) two intermediate variables are used: Focused instruction and Teachers’ professional community: Focused instruction is targeted at aspects of direct and constructivist teaching. Teachers’ professional community is a variable in which several subcategories of intermediary variables with regard to professional capacity of the staff (HRM and HRD) and climate are combined. Intermediate variables covering aspects of teaching are included to a limited extent in the publications examined: only three studies (Heck and Moriyama 2010; Louis et al. 2010 and Supovitz 2008) included teaching variables. In the studies by Heck and Moriyama and Louis et al. in the operationalization of the teaching variable explicit reference is made to aspects of both direct and constructivist teaching.

Conclusion In this chapter an overview of recent studies examining the relative impact of school leadership on student achievement is presented. This was done by means of a metaanalysis of 25 studies published between 2005 and 2010. In the meta-analyses both studies exploiting direct and indirect models of school leadership were examined. In 16 of the 25 publications found, researchers used an indirect model to study the impact of school leadership on student achievement; in nine publications a direct effect model was used. In two indirect model studies the relationship between school leadership and achievement was modeled directly and these studies were therefore classified under the direct effects studies. In two other studies different analysis techniques meant to measure both direct and indirect effects were used. These latter two studies were classified double both under the direct and indirect models. In total, results from 11 publications could be used in the meta-analysis of direct effect models. For the review of indirect effect models results from 15 publications could be used.

Direct Effect Studies In the meta-analysis of direct effect studies only a vote count procedure was used. The reason for this is that from the 11 available publications standardized effects for all relevant effects were reported in only three publications. In total 94 replications were used for the vote count.

96


The results of the vote count indicated that in almost three-quarter (74%) of all direct relationships examined the impact of school leadership on student achievement was not significant. In 20% of the relationships a significant positive effect was found and in 4% of all relationships a negative significant effect. Compared to the vote counts of direct effect school leadership studies in the period until 2005, the percentage of positively significant relationships found in studies between 2005 and 2010 is relatively higher (21% in 2005–2010 vs. 8% until 2005) and the percentage of non-significant relationships is relatively lower (74% in 2005–2010 vs. 87% until 2005). The conceptual analysis of the leadership concept used in the direct effect studies showed that in each publication a different operationalization of school leadership was applied. Taking into account the decomposition of school leadership concepts by Scheerens (see Chap. 1) school leadership was measured as instructional leadership in three studies, as transformational leadership in two studies and as integrated leadership in three studies. In one study both the concepts of instructional and transformational leadership were used. In the last study the limited description of the measurement of leadership in the publication did not allow a classification of the leadership variable used. With the exception of one study in which leadership styles were addressed, the other studies all focussed on different categories of leadership behavior.

Indirect Effect Studies In total 34 replications from the 15 publications in which indirect effect models were applied could be used to calculate the average effect size across all studies. The mean effect size found is 0.031 and does not deviate significantly from 0, given a standard error of 0.20. However, when excluding one publication with highly negative effects and more replications (6) than in the other publications, the mean effect size would be 0.060. The mean then deviates significantly from 0 with a standard error of 0.18. The conceptual analysis of leadership variables in the indirect effect studies showed that it is equal to direct effect studies, also in indirect studies researchers developed their own instruments to measure school leadership. In almost all indirect studies the measurement of school leadership includes aspects of transformational leadership, and in half of these studies also instructional leadership. In all but one publications, the measurement of school leadership addressed leadership behavioural categories. The exception is the study by Ten Bruggencate (2009), in which a more generic model of leadership was used and leadership styles were measured. The intermediate variables used in the studies were analyzed as well. The classification of effectiveness enhancing school and teaching factors was used to categorize the intermediate variables. The first striking result was that intermediate variables used in a number of studies (i.e. the studies by Heck and Hallinger 2009; Heck and Hallinger 2010;


97

Heck and Moriyama 2010 and Leithwood and Jantzi 2008) cover a very broad spectrum of effectiveness enhancing school factors, while in other studies the variables were much more focused. Examples of the latter are the study by De Maeyer et al. (2007) in which the intermediate variable used is Academic climate (related to the factors High expectations and shared sense of purpose among teachers as distinguished by Scheerens) and the study by Ten Bruggencate et al. (2010) in which the intermediate variable used is Topic coverage (related to the factor Opportunity to learn). The first four studies contain intermediate variables as factors related to Direction setting and Managing the instructional program, Professional capacity, Development and commitment of staff and Academic and supportive climate. The second striking result is that intermediate variables with regard to teaching are included in a minority of studies. In three studies (Heck and Moriyama 2010; Louis et al. 2010 and Supovitz 2008) aspects of teaching are included as variables or sub-variables.

Primary

Primary and secondary

Paraguay

US

Borden (2010)

Horng et al. (2010)c

65

256

96

Primary

Mexico Brazil Argentina Chile

Anderson (2008)

No of schools

School type

Countries

Author and year

Day-to-day-instruction Instructional program

Principal’s activities’ outside’ the classroom Organization Management

Meeting time Teacher relations Academic focus Principal’s activities’ inside’ the classroom

Meeting time Teacher relations Academic focus Community relations

Community relations

Leadership variable

Total accountability points earned

Mathematics: Overall average of students’ year-end grades (grade 5 and 6)

UNESCO examination Math

UNESCO examination Language

Achievement measure

Table A.1 Characteristics from direct effect studies on school leadership (2005–2010)

School size Minority enrolment SES ‘‘School improvement zone’’ School grade 3 years prior

Ability SES Gender

Antecedents (control variables)

Correlational design OLS Regression

Structural equation modeling

Correlational design Multilevel

Design and analysis

0.936 0.313

1.651**

-0.23

-1.016*** 1.147** 1.120** 0.53

-1.921*** 1.424*** 0.803 1.567***

2.089***

Effect size reported in original publicationa

+ +

(continued)

+**

-0.23

-*** +** +** 0.53

-*** +*** + +***

+***

Effect for vote countb

98 M. Hendriks and R. Steen

Countries

Cyprus

Author and year

Kythreotis et al. (2010)

Table A.1 (continued)

Primary

School type

22

No of schools

Principals’ leadership style: • Human resource Principals’ leadership style: • Political Principals’ leadership style: • Symbolic Principals’ leadership style: • Structural

Day-to-day-instruction Instructional program Internal relations External relations Principals’ leadership style: • Structural

Internal relations External relations Organization Management

Leadership variable

Student achievement score Math end of school year

Student achievement score Greek language end of school year

Achievement measure

Prior knowledge in Mathematics Gender SES

Prior knowledge in Greek language Gender SES

School grade 3 years prior


Correlational design Multilevel analysis

Correlational design OLS Regression

Design and analysis

(continued)

n.s.

n.s.

nit

nit

n.s.

0.04***

+ + + + n.s.

+ +


nit

0.04***

0.292 0.200 0.368 0.622 nit

-0.0236 0.308 0.869


4 Results from School Leadership Effectiveness Studies 99

England


Louis et al. (2010) US

Countries

Author and year


Primary

Primary

School type

106

258

256

No of schools


Principals’ leadership style: • Human resource Principals’ leadership style: • Political Principals’ leadership style: • Symbolic Transformational leadership practices: Setting directions Developing people Redesigning the organization

Leadership variable

Key stage 2 Achievement gain Numeracy (2 years) Percentages of students at school level meeting or exceeding the proficiency level on 2005 math tests

Key stage 2 Achievement gain Literacy (3 years)

Achievement measure


Stepwise regression

-0.315***

0.14**

Correlation coefficients

(continued)

-0.315***

0.14**

0.06

n.s.

nit

0.06

n.s.

0.04***


nit

0.04***


Correlation coefficients

Design and analysis


Primary (cohort 1)

Secondary

Spain

Martin et al. (2008)d

Miller and Rowan US (2006)

School type

Countries

Author and year


140

27

No of schools

Administrative support

Spanish Language

Shared leadership Leadership of the administration

Mathematics Science Reading (2nd grade school achievement)

Achievement measure

Leadership variable

SES Learning strategies Meta-cognitive ability Rate of grade repeating SES Percentage of female students

Race/ethnicity Gender, Family SES, educational expectations or engagement (motivation), ability grouping, course taking. Students’ average SES, dispersion of family SES among students, total school enrollment

Sex


HLM Growth models

Correlational design Longitudinal Multilevel analysis

Design and analysis

nit nit -3.334

-0.059 1.04***


(continued)

n.s. n.s. -

-0.059 +***



Author and year

Countries

Table A.1 (continued) No of schools

138

138

137

806

806

140

138

138

137

806

School type

Primary (cohort 1)

Primary (cohort 3)

Primary (cohort 3)

High school

High school

Primary (cohort 1)

Primary (cohort 1)

Primary (cohort 3)

Primary (cohort 3)

High school

Teacher control

Leadership variable

Mathematics (2nd grade school achievement) Reading (4th grade school achievement) Mathematics (4th grade school achievement) Reading (10th grade school achievement) Mathematics (10th grade school achievement) Reading (2nd grade school achievement) Mathematics (2nd grade school achievement) Reading (4th grade school achievement) Mathematics (4th grade school achievement) Reading (10th grade school achievement)

Achievement measure


Design and analysis

(continued)

+***

0.297***

-

-2.322

-**

-

-4.637

-3.412**

-

-3.147

-

-

-0.263

-0.357

+

+

+


2.280

0.046

1.130



O’Donnell and White (2005)e

Author and year

US

US

Countries


138

137

806

806

Primary (cohort 3)

Primary (cohort 3)

High school

High school

75

138

Primary (cohort 1)

Secondary Middle level

140

806

High school

Primary (cohort 1)

No of schools

School type

Instructional leadership role: (1) defining the school mission (teacher questionnaire) Instructional leadership role: (2) managing the instructional program (teacher questionnaire)

Staff cooperation

Leadership variable

Mathematics (10th grade school achievement) Reading (2nd grade school achievement) Mathematics (2nd grade school achievement) Reading (4th grade school achievement) Mathematics (4th grade school achievement) Reading (10th grade school achievement) Mathematics (10th grade school achievement) 8th-grade reading components of the 2000–01 PSSA

Achievement measure


Pearson correlation

Design and analysis

(continued)

+**

+**

-

-0.087

+**

+

0.082

+**

-

+

0.455

-1.279

+

+

+***


1.450

3.863

0.929***



Author and year

Countries


School type

No of schools

Achievement measure

Instructional leadership role: (3) promoting the school learning climate (teacher questionnaire) (1) Defining the school 8th-grade mission (teacher mathematics questionnaire) components of the 2000–01 PSSA (2) Managing the instructional program (teacher questionnaire) (3) Promoting the school learning climate (teacher questionnaire) (1) Defining the school 8th-grade reading mission (principal components of questionnaire) the 2000–01 PSSA (2) Managing the instructional program (principal questionnaire) (3) Promoting the school learning climate (principal questionnaire)

Leadership variable


Pearson correlation

Design and analysis

n.s.

n.s.

(continued)

n.s.

+***

+***

n.s.

+**

+**

n.s.

+**

+**

n.s.

+***


+***



Author and year

Countries


School type

No of schools

Achievement measure

(1) Defining the school 8th-grade mathematics mission (principal components of questionnaire) the 2000–01 PSSA (2) Managing the instructional program (principal questionnaire) (3) Promoting the school learning climate (principal questionnaire) (1) Defining the school 8th-grade reading components of mission (teacher the 2000–01 questionnaire) PSSA (2) Managing the instructional program (teacher questionnaire) (3) Promoting the school learning climate (teacher questionnaire) (1) Defining the school 8th-grade mission (teacher mathematics questionnaire) components of the 2000–01 PSSA

Leadership variable

SES


Correlational design Forward Selection Regression

Design and analysis

n.s.

n.s.

n.s.

n.s.

nit

nit

nit

n.s.

n.s.

(continued)

n.s.

0.394***

n.s.

n.s.

0.394***




Author and year

Countries


School type

No of schools

Achievement measure

(2) Managing the instructional program (teacher questionnaire) (3) Promoting the school learning climate (teacher questionnaire) (1) Defining the school 8th-grade reading mission (principal components of questionnaire) the 2000–01 PSSA (2) Managing the instructional program (principal questionnaire) (3) Promoting the school learning climate (principal questionnaire) (1) Defining the school 8th-grade mission (principal mathematics questionnaire) components of the 2000–01 PSSA (2) Managing the instructional program (principal questionnaire)

Leadership variable

SES


Correlational design Forward Selection Regression

Design and analysis

n.s.

n.s.

n.s.

n.s.

nit

nit

nit

nit

(continued)

n.s.

nit

0.344***

n.s.

nit

0.344***




Australia

Shin et al. (2010)

114

164

Egypt

171

47

No of schools

Colombia

Secondary

Belgium (Flemish Secondary Community)

Opdenakker and Van Damme (2006)

School type

Countries

Author and year


Supervising and evaluating teachers Instructional leadership Supervising and evaluating teachers Instructional leadership Supervising and evaluating teachers

Instructional leadership (developing curriculum and pedagogy)

(3) Promoting the school learning climate (principal questionnaire) Participative professionallyoriented leadership

Leadership variable

Mathematics

Mathematics

Achievement measure

Individual level: Ability SES Parental involvement Achievement motivation School level Average ability Average SES Percentage of students in the school from economically disadvantaged homes


Correlational design Multilevel

Correlational design Multilevel analyis

Design and analysis

-

-0.41

+

0.45

-0.11

+

-

-

0.33

-0.23

-0.30

-

(continued)

n.s.

nit

-0.020




111

140

133

182

Hungary

Korea

Turley

United States

No of schools

103

School type

England

Countries

Instructional leadership Supervising and evaluating teachers Instructional leadership Supervising and evaluating teachers Instructional leadership Supervising and evaluating teachers Instructional leadership Supervising and evaluating teachers Instructional leadership Supervising and evaluating teachers

Leadership variable

Achievement measure


Design and analysis

-

-0.06

+

0.59

-0.40

+

+

0.10

0.56

-

+

0.41

-0.09

_

-

-0.29

-0.63

+


0.43


a (*p \ 0.10, ** p \ 0.05, *** p \ 0.01) Nit = not in table, n.s. = not significant b Only standardized effects are reported (*p \ 0.10, ** p \ 0.05, *** p \ 0.01) Nit = not in table, n.s. = not significant c Administration tasks is not included as this category was omitted as the reference category in the analysis d In the meta-analysis only the achievement measures set at the midpoint of the time series in the six models are used. The linear slopes and quadratic slopes reported in the publication are not taken into account e In this table the effects of the Pearson correlation analyses and the effects of the univariate forward selection regression analyses are included. Effects are reported between leadership (as perceived by teachers and principals separately) and mathematics achievement

Author and year



Country

England

England

US

Author and year

Day et al. (2009)

Day et al. (2009)


Primary

Secondary

Primary

School type

195

309

363

No of schools

Change in leadership

Initial distributed leadership (three aspects) • School improvement • School governance • Resource management Initial distributed leadership Change in leadership



Leadership variable

Table A.2 Characteristics of indirect effect studies on school leadership

Principal stability Principal stability

SES

SES

Antecedents

Change in capacity Change in capacity Change in capacity

Leadership distribution: • Distributed leadership • Staff’ • SLT Collaboration • SLT impact on learning and teaching

Leadership distribution: • ‘Distributed leadership • Staff • SMTa • SLTb Collaboration’ • SLT’s Impact on Learning and Teaching’

Intermediate level 1

School Organization

School processes: • Teacher collaborative culture • Assessment for learning • Improvement in school conditions’ • External Collaborations and Learning Opportunities’ School processes: • Teacher collaborative culture • Assessment for Learning • Improvement in school conditions • External collaborations and learning opportunities’


Change in pupil outcomes over 3 years

• High academic standards • Pupil motivation’ and responsibility for learning • Reduction in staff mobility and absence • Change in pupil behavior • Change in pupil attendance • High academic standards • Pupil motivation and learning culture • Change in pupil behavior • Change in pupil attendance

(continued)

Math growth

Math growth

Math growth


Achievement measure



Primary

US

US

Heck and Hallinger (2010) Leadership Quarterly

Heck and Moriyama (2010)

Primary

School type

Country

Author and year


198

197

No of schools

Collaborative leadership: shared school governance, collaborative decisions focusing on academic improvement, broad participation in efforts to evaluate the school’s academic development Collaborative leadership

Student stability Student composition Teacher stability and experience

Teacher stability, principal stability school size



Student composition

Student composition

Antecedents

Initial leadership (year 1)

Distributed leadership (three aspects) •School improvement • School governance • Resource management Initial leadership (year 1)


Leadership variable

Added year effect Math

(continued)

Growth Rate Math

Growth Rate Read

Initial Math level (year 2)

Initial Read level (year 2)

Reading (grade 3–5)

Math (grade 3–5)

Achievement measure



Added year effect Read


Initial school capacity (year 1) Initial school capacity (year 1) Change in school capacity Change in school capacity Instructional practices (four dimensions); Classroom teaching conditions, Quality of student support system Professional capacity Focused and sustained action on improvement

School improvement capacity



Country

US

Canada

Canada

Author and year






Primary

Primary and secondary (K-12)

School type

90

88

No of schools

Transformational school leadership • providing resources and support, empowering teams • facilitative: encourage successful implementation Transformational leadership: Collective leadership

School leadership: 1. Setting directions 2. Developing people 3. Redesigning the organization 4. Managing the instructional program

Leadership variable

SES

Capacity motivation setting

School improvement planning process

Contents of the school Improvement plan Implementation process

Class conditions • Workload • Areas of formal preparation • Student grouping • Curriculum and instruction

School conditions • School culture • Decision making processes • Supports for instruction • Professional learning community

District leadership • District conditions • Leaders’ self efficacy beliefs • Leaders’ collective efficacy beliefs

Co-variables: District size School size School level Principal turnover Principal gender, years of experience, race, ethnicity



Antecedents


(continued)

Percentage of students meeting or exceeding the proficiency level on language and math tests (averaged across grades and subjects— language and mathematics) over 3 years (2003–2005)

Students’ 2 year mean achievement scores (as measured by the provincial tests of literacy and mathematics in Grades 3 and 6) Students’ 2 year mean gain

Percentages of students meeting or exceeding the proficiency level on language and math tests (averaged across grades and subjects— language and mathematics) over 3 years (2003–2005)

Achievement measure


Country

Canada

US

Belgium (Flanders)

Belgium (Flanders)

Author and year


Louis et al. (2010)

De Mayer et al. (2007)



Secondary

Secondary


Primary

School type

57

47

106 (50 primary 53 secondary, 3 K8)

199

No of schools

Integrated leadership Participative professionally oriented leadership


Instructional leadership Shared leadership

Distributed leadership (with focus on managing and leading the instructional program

Leadership variable

Percentage of girls, mean IQ, mean SES, mean linguistic ethnic background (LEB) School size Average intellectual level of students in school

School poverty

Rational path Emotions path Organizational path Family path

Composite SES

Cooperation between teachers School climate: • Learning climate • Relational climate Opportunity to learn

Academic climate

Rational path: • Academic press • Disciplinary climate Emotions path: • Collective teacher efficacy • Teacher trust in others Organizational path • Instructional time • Professional learning community Family path: • Computer at home • Adult help at home Professional community


Antecedents

Focused instruction



(continued)

Mathematics test

Percentages of students at school level meeting or exceeding the proficiency level on 2005 math tests Mathematics Reading

Percentage of students per school achieving level 3 or higher at the grade 3 and 6 math and literacy achievement

Achievement measure


Secondary

Varies from 67 in Cyprus to 239 in USc

52

205

No of schools

Time spent on Instructional leadership Administrative duties Supervising teachers Public relations

School leadership style Rational goals Internal process Human relations Open systems

Principal leadership


Leadership variable

School size Denomination (RC) Percentage of cultural minorities Urbanisation of the environment Valuing education Competition No. of books at home; Total school enrollment; Type of community; Disadvantaged students limiting teaching; Student behavior frequency; Shortage of ICT resources limiting teaching; School climate (principal’s perception)

SES

Antecedents

Student engagement Teacher’s work

Performanceorientation Developmentorientation

Student attitudes towards mathematics (valuing math) Student attitudes towards school (valuing school)

Peer influence

Teacher commitment to school mission Teacher commitment to professional community Teacher commitment to community partnerships Change in instruction


Collective teacher efficacy



Math achievement

Grade 3 and 6 Achievement: residuals from regression 2001 scores over 2000 averaged across grades (3 and 6) en subjects (reading, writing and mathematics) English Language Arts Mathematics Promotion rate Average exam mark

Achievement measure

c

b

SMT: School Management Team SLT: Senior Leadership Team In order to correct for over and underrepresentation of schools a weighting procedure was applied to ensure that within countries all schools are represented appropriately. The weighting procedure also ensures that all countries have the same weight in the overall analyses regardless of their size

a

14 countries (TIMSS 2007)

Secondary

Netherlands



US

Supovitz et al. (2010) Ten Bruggencate (2009)

Primary

Canada


School type

Country

Author and year



Integrated leadership: (secondary level)

Day et al. (2009)b

Heck and Hallinger (2010) Leadership quarterly

Math Growth Rate

Initial distributed leadership

Total

Initial Reading scores (year 2)

School capacity

Change in capacity

Change in capacity

Change in capacity

Change in capacity

Transformational leadership Distributed leadership

0.29

0.28

0.12**

46**

0.46**

14**

0.14**

Contribution of all 0.005 other paths with 3–6 intermediate levels



Initial Read level (year 2)

Math Growth Rate

School organization

Math Growth Rate

School organization

Change in pupil outcomes

Teacher collaborative culture

Label

Label

ES*

Indirect effect level 2


Change in leadership Math Growth Rate

Direct effect ES*a

Change in leadership Math Growth Rate

Total

Math Growth Rate



Achievement measure


Total

Integrated leadership: (primary level)

Day et al. (2010)


Leadership measure

Author and Year

Table A.3 Direct and indirect paths from school leadership to student achievement and total effect sizes

0.13**

0.18**

0.22**

0.18**

0.22**

0.12

0.13

ES*

0.09**

0.09**

0.21

ES*

(continued)

Math Growth Rate

Math Growth Rate

Pupil Motivation & Responsibility for Learning

Label



Integrated leadership Proportion of School students leadership reaching or exceeding the state’s proficient level

2 year mean achievement score

2 year achievement gain

School leadershipc

School leadershipd


Added year Effect Math

Growth Rate Math

Transformational leadership Change in leadership

Collaborative leadership

Growth Rate Reading


Added year Effect Read

Initial Math scores (year 2)



Achievement measure

Leadership measure



Author and Year

Table A.3 (continued) Direct effect ES*a

School conditions


Instructional Practices

Change in school capacity

Change in school capacity

School capacity

0.66***

0.26**

0.49**

0.49**

0.12**

Proportion students exceeding proficient level



Growth Rate Math

Growth Rate Read

Initial Math level (year 2)

Label

Label

ES*



0.40***

0.49**

0.57**

0.20**

0.20**

0.15**

ES*

Label

ES*

(continued)




Leithwood et al. (2010) Model 1


Total

Collective leadershipe



Leadership measure

Author and Year


Percentage of students per school achieving level 3 or higher at math and literacy test


Percentage of students meeting or exceeding the proficiency level on language and math tests

Achievement measure

Direct effect ES*a

Family path

Rational path: Disciplinary climate

0.33**

-0.07

Organizational path

Rational path: Academic Press

0.15** 0.57**

Emotional path

0.56**

Rational path

ES*

0.22**

Percentage of 0.23** students achieving level 3 or higher at math and literacy test

0.26**

-0.08

0.21

Percentage of 0.26** students achieving level 3 or higher at math and literacy test

Label

Label

ES*


Indirect effect level 1 Label

ES*

(continued)



Author and Year

Distributed leadership (total)

Leadership measure


Achievement measure

Direct effect ES*a

-0.12

0.28**

0.30**

0.69**

Emotions path: Teacher trust in others Organizational path: Instructional time Organizational path: Professional learning community

Family path: Adult help at home

Family path: Computer at home

0.20

0.10**

Emotions path: Collective teacher efficacy

-0.16**

0.26**

0.34**

Label

ES*

Label

ES*



ES*

(continued)



Percentages of students at school (building) level meeting or exceeding the proficiency level 2005 math tests

Achievement measure

Total

Math

Integrated leadership Mathematics

Total

Integrated leadership Reading

Total

Shared leadership

Opdenakker and Van Participative Damme (2007) professionally oriented leadership

De Maeyer et al. (2007)


Louis et al. (2010)

Total

Leadership measure

Author and Year


-0.15

-0.27

Direct effect ES*a

0.01

0.01

Cooperation

0.59**

0.59**

Relations climate

Relations climate

Mathematics

Reading

0.381*** Focused instruction

Focused instruction

0.267*** Focused instruction

Cooperation

Academic climate

Academic climate

Professional community

Professional community

Label

Label

ES*



Mean math achievement

0.75***

0.75***

-0.02**

0.43**

Learning climate

Learning climate

0.395*** Mean math achievement

0.148

0.395*** Mean math achievement

ES*

1.00**

1.00**

0.205***

0.205***

0.205***

ES*

(continued)




Author and Year

Total


Total

Leadership measure


Composite school score: residuals from regression 2001 scores over 2000 scores (averaged across grades en subjects)

Achievement measure

Direct effect ES*a

Teacher commitment 0.12** to community partnerships


0.33***

Teacher commitment 0.78*** to community partnerships


0.48***


Teacher commitment 0.49*** to professional community

-0.14

Teacher commitment 0.27*** to professional community

0.48***


0.17

Teacher commitment 0.20*** to school mission


0.48***

ES*

Teacher commitment 0.75*** to school mission


Label

Label

ES*




0.33****

-0.14

0.17

ES*

(continued)

Teacher Commitment to Professional Community


Composite school score (achievement)

Label






Leadership style Human relations (teacher perceptions)

Total

Leadership style Internal process (teacher perceptions)

Total

Leadership style Rational goals (teacher perceptions)

Total


Total

Leadership measure

Author and Year


Average exam mark

Mathematics

English Language Arts

Achievement measure

-0.17**

Direct effect ES*a

Development orientation


0.35**

0.23**

0.57**

0.14***

Change in instruction


0.30***

0.18***

Change in instruction Peer influence

0.38***

Peer influence

Teacher’s work

Teacher’s work

Teacher’s work

Mathematics




Label

Label

ES*



0.24**

0.24**

0.24**

-0.04

0.26***

0.11**

0.21***

ES*

Promotion rate

Promotion rate

Promotion rate

Mathematics


Label

0.18**

0.18**

0.18**

-0.04

0.11***

ES*

(continued)





Author and Year

Time spent on administrative duties

Time spent on instructional leadership

Total

Leadership style Open systems (principal perceptions)

Leadership style Rational goals (principal perceptions)

Total

Leadership style Open systems (teacher perceptions)

Total

Leadership measure


Math achievement

Achievement measure

-0.04

-0.32**

-0.18**

Direct effect ES*a

Valuing math

Valuing math




0.08

-0.12

0.33*

0.28**

0.56**

Math achievement

Math achievement

Teacher’s work

Teacher’s work

Teacher’s work

Label

Label

ES*



-0.21

-0.21

0.24**

0.24**

0.24**

ES*

Promotion rate

Promotion rate

Promotion rate

Label

0.17**

0.17**

0.18**

ES*

(continued)



Author and Year

Time spent on public relations

Total

Time spent on supervising teachers

Total

Leadership measure


Achievement measure

0.04

Direct effect ES*a

Topic coverage

0.15

-0.06 0.12

Valuing math

-0.13

Topic coverage

Topic coverage

Label

ES*

Label

Math achievement

Math achievement

Math achievement

Math achievement



0.24

0.24

-0.21

0.24

ES*

Label

ES*

(Continued)



Integrated leadership: (secondary level)

Day et al. (2009)b



Heck and Hallinger (2010) Leadership quarterly

Math Growth Rate


Math Growth Rate


Integrated leadership School leadership

Proportion of students reaching or exceeding the state’s proficient level


Growth Rate Math



Growth Rate Reading



Initial Math scores (year 2)



Initial Reading scores (year 2)


Total

Math Growth Rate


Total

Math Growth Rate



Achievement measure


Total

Integrated leadership: (primary level)

Day et al. (2010)


Leadership measure

Author and Year


0.18

ES*

Label

ES*

Label Change in pupil outcomes



(continued)

0.24***

0.14**

0.16**

0.13**

0.10**

0.02**

0.02**

0.092

0.28

0.04

0.001

ES*

Total effect


De Maeyer et al. (2007)

Louis et al. (2010)




Total


Total


Total

Shared leadership

Total


Distributed leadership (total)


Total

Percentage of students meeting or exceeding the proficiency level on language and math tests

Collective leadershipe

Mathematics

Reading

Percentages of students at school (building) level meeting or exceeding the proficiency level 2005 math tests



2 year achievement gain

School leadershipd

2 year mean achievement score

School leadershipc



Achievement measure

Leadership measure

Author and Year


ES*

Label

Label

ES*



(continued)

-0.16

-0.02

0.031

0.052

0.15**

0.11**

0.24***

-0.6

0.11

ES*

Total effect





Participative professionally oriented leadership


Total

Leadership style Human relations (teacher perceptions)

Total

Leadership style Internal process (teacher perceptions)

Total

Leadership style Rational goals (teacher perceptions)

Total


Total


Total


Total

Leadership measure

Author and Year


Mathematics


Composite school score: residuals from regression 2001 scores over 2000 scores (averaged across grades en subjects)

Math

Achievement measure

Average exam mark

Average exam mark

0.27**

0.27**

0.27**

0.75***

Effort

Average exam mark

0.08 Math

Math 0.38*

0.14*

ES*

Label

ES*

Label Opportunity to learn



(continued)

0.00

0.00

-0.16**

-0.01

0.03**

0.220

0.006

ES*

Total effect


0.09 0.04

-0.09

0.02

-0.31**

0.00

-0.18**

ES*

Time spent on public relations

0.20*

ES*

Total effect

Total

Time spent on supervising teachers

Total

Time spent on administrative duties

Time spent on instructional leadership

Total

Average exam mark

Leadership style Open systems (principal perceptions)

0.20*

0.27**

Label

ES*

Label Average exam mark



Average exam mark

Math achievement

Achievement measure

Leadership style rational goals (principal perceptions)

Total

Leadership style Open systems (teacher perceptions)

Leadership measure

b

(* p \ 0.10, ** p \ 0.05, *** p \ 0.01) The model is too complex to describe in full. Only the most important path and the contribution of all other paths together are included c Only the standardized total effects for independent and mediating variables on mean student achievement are presented, so it is unknown if the effects from leadership on achievement are direct or indirect d Only the standardized total effects for independent and mediating variables on mean student achievement are presented, so it is unknown if the effects from leadership on achievement are direct or indirect e In the article just the total indirect effect is presented

a



Author and Year




127

References Day, C., Sammons, P., Hopkins, D., Harris, A., Leithwood, K., Gu, Q., et al. (2009). The impact of school leadership on pupil outcomes. Nottingham: The National College for School Leadership. De Maeyer, S., Rymenans, R., van Petegem, P., van den Bergh, H., & Rijlaarsdam, G. (2007). Instructional leadership and pupil achievement: The choice of a valid conceptual model to test effects in school effectiveness research. School effectiveness and School Improvement, 18(2), 125–145. Hallinger, P. (1983). Assessing the instructional management behavior of principals (Unpublished Doctoral Dissertation, Stanford University, 1983). Hallinger, P., & Heck, R. H. (1996a). Reassessing the principals role in school effectiveness: A review of empirical research, 1980–1995. Educational Administration Quarterly, 32, 5–44. Hallinger, P., & Heck, R. H. (1996b). The principal’ s role in school effectiveness an assessment of methodological progress 1980–1995. In K. Leithwood, et al. (Eds.), IInternational handbook of instructional leadership and administration. Dordrecht: Kluwer Academic Publishers. Hallinger, P., & Heck, R. H. (1998). Exploring the principal’s contribution to school effectiveness: 1980–1995. School Effectiveness and School Improvement, 9(2), 157–191. Heck, R. H., & Hallinger, P. (2009). Testing a dynamic model of organizational leadership and school improvement effects on growth in learning. Paper presented at the annual meeting of American Educational Research Association, disciplined inquiry: Education research in the circle of knowledge, San Diego Heck, R. H., & Hallinger, P. (2010). Testing a longitudinal model of distributed leadership effects on school improvement. The Leadership Quarterly, 21, 867–885. Heck, R. H., & Moriyama, K. (2010). Examining relationships among elementary schools’ contexts, leadership, instructional practices, and added-year outcomes: A regression discontinuity approach. School Effectiveness and School Improvement, 21(4), 377–408. Horng, E. L., Klasik, D., & Loeb, S. (2010). Principal’s time use and school effectiveness. American Journal of Education, 116(4), 491–523. Kythreotis, A., Pashiardis, P., & Kyriakides, L. (2010). The influence of school leadership styles and culture on students’ achievement in Cyprus primary schools. Journal of Educational Administration, 48(2), 218–240. Leithwood, K., & Jantzi, D. (2008). Linking leadership to student learning: The contributions of leader efficacy. Educational Administration Quarterly, 44, 496–528. Leithwood, K., & Mascall, B. (2008). Collective leadership effects on student achievement. Educational Administration Quarterly, 44, 529–561. Leithwood, K., Patten, S., & Jantzi, D. (2010). Testing a conception of how school leadership influences student learning. Educational Administration Quarterly, 46, 671–706. Louis, K. S., Dretzke, B., & Wahlstrom, K. (2010). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21(3), 315–336. Louis, K. S., & Kruse, S. D. (1995). Professionalism and community: Perspectives on reforming urban schools. Thousand Oaks: Corwin Press. Martin, E., Martinez-Arias, R., Marchesi, A., & Perez, E. M. (2008). Variables that predict academic achievement in the Spanish compulsory secondary educational system: A longitudinal, multi-level analysis. Spanish Journal of Psychology, 11(2), 400–413. O’Donnell, R. J., & White, G. P. (2005). Within the account era: Principals’ instructional leadership behaviors and student achievement. NASSP Bulletin, 89(645), 56–71. Opdenakker, M.-C., & Van Damme, J. (2007). Do school context, student composition and school leadership affect school practice and outcomes in secondary education? British Educational Research Journal, 33(2), 179–206.

128


Pitner, N. J. (1986). Substitutes for principal leader behavior: An exploratory study. Educational Administration Quarterly, 22(2), 23–42. Ross, J. A., & Gray, P. (2006). School leadership and student achievement. The mediating effects of teacher beliefs. Canadian Journal of Education, 29(3), 798–822. Scheerens, J., Luyten, H., Steen, R., & Luyten-de Thouars, Y. (2007). Review and meta-analyses of school and teaching effectiveness. Enschede: University of Twente, Department of Educational Organisation and Management. Supovitz, J. A. (2008). Instructional leadership in American high schools. In M. M. Mangin & S. R. Stoelinga (Eds.), Effective teacher leadership: Using research to inform and reform (pp. 144–162). New York: Teachers College Press. Ten Bruggencate, G. (2009). Maken schoolleiders het verschil? [Do school leaders make a difference?]. Enschede: University of Twente. Ten Bruggencate, G., Luyten, H., & Scheerens, J. (2010). Quantitative analysis of international data, exploring indrect effect models of school leadership. Enschede: University of Twente.

Annex: References Used in the Meta-Analyses: Direct Effect Models Anderson, J. B. (2008). Principals’ role and public primary schools’ effectiveness in four Latin American cities The Elementary School Journal, 109(1), 36–60. Borden, A. M. (2010). Relationships between Paraguayan principals’ characteristics, teachers’ perceptions of instructional leadership and school outcomes’, International Journal of Leadership in Education, First published on: 29 October 2010 (iFirst). Horng, E. L., Klasik, D., & Loeb, S. (2010). Principal’s time use and school effectiveness. American Journal of Education, 116(4), 491–523. Kythreotis, A., Pashiardis, P., & Kyriakides, L. (2010). The influence of school leadership styles and culture on students’ achievement in Cyprus primary schools. Journal of Educational Administration, 48(2), 218–240. Leithwood, K., & Jantzi, D. (2006). Transformational school leadership for large-scale reform: Effects on students, teachers, and their classroom practices. School Effectiveness and School Improvement, 17(2), 201–227. Louis, K. S., Dretzke, B., & Wahlstrom, K. (2010). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21(3), 315–336. Martin, E., Martinez-Arias, R., Marchesi, A., & Perez, E. M. (2008). Variables that predict academic achievement in the spanish compulsory secondary educational system: A longitudinal, multi-level analysis. Spanish Journal of Psychology, 11(2), 400–413. Miller, R. J., & Rowan, B. (2006). Effects of organic management on student achievement. American Educational Research Journal, 43(2), 219–253. O’Donnell, R. J., & White, G. P. (2005). Within the account era: Principals’ instructional leadership behaviors and student achievement. NASSP Bulletin, 89(645), 56–71. Opdenakker, M., & Van Damme, J. (2006). Differences between secondary schools: A study about school context, group composition, school practice and school effects with special attention to public and catholic schools and types of schools. School Effectiveness and School Improvement, 17(1), 87–117. Shin, S-H., & Slater, Ch. L. (2010). Principal leadership and mathematics achievement: An international comparative study. School Leadership & Management, 30(4), 317–334.


129

Indirect Effect Models Day, C., Sammons, P., Hopkins, D., Harris, A., Leithwood, K., Gu, Q., et al. (2009). The Impcat of school leadership on pupil outcomes. Nottingham: The National College for School Leadership. De Maeyer, S., Rymenans, R., van Petegem, P., van den Bergh, H., & Rijlaarsdam, G. (2007). Instructional leadership and pupil achievement: The choice of a valid conceptual model to test effects in school effectiveness research. School effectiveness and School Improvement, 18(2), 125–145. Heck, R. H., & Hallinger, Ph. (2009). Assessing the contribution of distributed leadership to school improvement and growth in math achievement. American Educational Research Journal, 46(3), 659–689. Heck, R. H., & Hallinger, Ph. (2010). Testing a longitudinal model of distributed leadership effects on school improvement. The Leadership Quarterly, 21, 867–885. Heck, R. H., & Moriyama, K. (2010). Examining relationships among elementary schools’ contexts, leadership, instructional practices, and added-year outcomes: A regression discontinuity approach. School Effectiveness and School Improvement, 21(4), 377–408. Leithwood, K., Jantzi, D., & McElheron-Hopkins, Ch. (2006). The development and testing of a school improvement model. School Effectiveness and School Improvement, 17(4), 441–464. Leithwood, K., & Jantzi, D. (2008). Linking leadership to student learning: The contributions of leader efficacy. Educational Administration Quarterly, 44, 496–528. Leithwood, K., & Mascall, B. (2008). Collective leadership effects on student achievement. Educational Administration Quarterly, 44, 529–561. Leithwood, K., Patten, S., & Jantzi, D. (2010). Testing a conception of how school leadership influences student learning. Educational Administration Quarterly, 46, 671–706. Louis, K. S., Dretzke, B., & Wahlstrom, K. (2010c). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21(3), 315–336. Opdenakker, M.-C., & Van Damme, J. (2007). Do school context, student composition and school leadership affect school practice and outcomes in secondary education? British Educational Research Journal, 33(2), 179–206. Ross John, A., & Gray, P. (2006). School leadership and student achievement: The mediating effects of teacher beliefs. Canadian Journal of Education, 29(3), 798–822. Supovitz, J., Sirinides, Ph., & May, H. (2010). How principals and peers influence teaching and learning. Educational Administration Quarterly, 46, 31–56. Ten Bruggencate, G. C. (2009). Maken schoolleiders het verschil? [Do school leaders make a difference?]. Enschede: University of Twente. Ten Bruggencate, G., Luyten, H., & Scheerens, J. (2010). Quantitative analysis of international data exploring indrect effect models of school leadership. Enschede: University of Twente.

Chapter 5

Summary and Conclusion: Instructional Leadership in Schools as Loosely Coupled Organizations Jaap Scheerens

In this final chapter a summary of the main outcomes is given. The study has looked at the definition and concept formation of school leadership, analyzed modeling and theory foundation and presented results of meta-analyses of leadership effects. In the last section some implications for educational practice and policy are drawn.

The Most Frequently Used Operational Definitions of School Leadership Empirical studies that have investigated the effects of instructional leadership on student achievement tend to focus on broad categories of leadership behavior. Leadership traits and leadership styles, as more personality tied leadership dispositions, are hardly addressed in the research literature in question. The most commonly used behavioral categories go back to Hallinger’s definition of what we have termed ‘‘extended’’ instructional leadership: defining the school’s mission, managing the curriculum and creating a task-oriented learning climate. Two of these dimensions are quite similar to the influential distinction made by Leithwood et al. (2006) in four main school leadership behaviors: setting directions, managing the instructional program, developing people and redesigning the organization. Where instructional leadership is focused on optimizing teaching and learning, transformational leadership is more oriented on secondary processes aimed at improving organizational structures, organizational culture and organizational processes. J. Scheerens (&) Department of Educational Organization and Management, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands e-mail: [email protected]


131

132

J. Scheerens

A D

B E

C

Fig. 5.1 Decomposition of school leadership concepts

Given the considerable overlap between the two constructs, the term integrated leadership has been coined. A more precise indication of overlapping and unique dimensions was given in Fig. 1.8, reproduced again in Fig. 5.1. A stands for transformational leadership, D stands for the broader perception of instructional leadership encompassing direction setting, developing a task-oriented climate and managing the teaching and learning program; B represents the intersection of transformational leadership and instructional leadership, encompassing direction setting, and developing a task-oriented culture; C is instructional leadership in the narrow sense (managing the teaching and learning program); E stands for distributed leadership as a sub-set of transformational leadership. The union of A and D (transformational leadership and instructional leadership) is integrated leadership.

Modeling Leadership Effects Direct and Indirect Effect Models Initially direct effects of ‘‘strong’’, or ‘‘instructional’’ leadership on student performance were studied. Strong leadership was one of the five key determinants of effective schools as identified by one of the pioneers of the field (Edmonds 1979). In their review and meta-analyses of educational effectiveness Scheerens and Bosker (1997) noted an enormous difference between the strong expectations about practically all of the school variables that were identified as being associated with high performance in qualitative studies, including leadership, as compared to syntheses of quantitative evidence which showed very modest effects; they reported an effect size of 0.04 (correlation) for instructional leadership. This difference in tone, between qualitative and quantitative leadership studies, exists until today; compare the description of the study by Day et al. in Chap. 3. During the last decade empirical studies have shifted to addressing indirect effect models of leadership. This means that leadership effects are seen as (partly) mediated by other school, and sometimes also classroom level variables. Indirect effect models have intuitive appeal, it is clear that a school principal operates at a certain

5 Summary and Conclusion

133

distance from teaching and learning, and that he or she could be expected to facilitate these core processes indirectly via a range of school conditions. Indirect effect models may have several layers or stages of intermediary variables, may simultaneously investigate direct and indirect effects, include antecedent conditions or covariables and even consider reciprocal effects. Path analytic techniques and structural equation modeling are the favored analysis tools for investigating indirect leadership effect models. A key question that indirect effect models address is the question whether a stable set of intermediary school conditions can be identified. Ideally such conditions would show clear impacts from leadership behavior and in their turn are expected to have a sizeable impact on student achievement.

The Full Circle of Leadership Distribution The classical model of the school as an organization is the professional bureaucracy. In professional bureaucracies training of teachers is the most important coordination mechanism. Based on their professional skills teachers can operate quite autonomously and there is little need for direct supervision. According to Weick (2001) professionals operating in loosely coupled structures are well equipped to react adaptively to small changes in the environment (e.g. changes in the composition of the student population). A professional bureaucracy can exist without pronounced leadership. The school effectiveness and school improvement movements expect teachers to become involved in cooperative activities, to stimulate common orientations on the ends and perhaps also the means of schooling. Strong instructional leadership was the boldest jump forward in breaking the traditional structures with goal coordination (an achievement oriented school culture and emphasis on basic skills) as the main orientation, next to creating an orderly atmosphere and stressing the monitoring of students’ progress. Transformational leadership, more of an invention of school improvement experts than of school effectiveness researchers, retreated somewhat from the bold intrusion in the direction of the primary process of teaching and learning and concentrated on organizational development and organizational learning with a looser or in any case much more indirect relationship to enhancing student outcomes. One could read the move toward distributive or collective leadership, sometimes also described as ‘‘teacher leadership’’, as a kind of reinstallation of the professional autonomy of teachers. In some studies leadership (defined as distributed leadership) and the organizational development that it is supposed to stimulate (more collaboration) appear almost one and the same. When Hallinger and Heck (2010) in a recent contribution speak of organizational leadership; the idea of individualized or focused leadership has fused with ‘‘structural coordination’’, in the sense of structural and sustained procedural arrangements, such as cooperative structures, task divisions and common standards. This concept seems to come close to the much older idea of ‘‘substitutes for leadership’’, coined by Kerr and Jermier (1978). The full circle of leadership distribution is depicted in Fig. 1.7 reproduced in Fig. 5.2.

134

J. Scheerens

3) transformational

2) instructional leadership

1) professional bureaucracy

4)

collective leadership

5) organizational leadership

6) substitutes for leadership

Fig. 5.2 The full circle of concept development on school leadership

This circular movement, however, should not be read as a return to square one. Under the influence of more turbulent environmental conditions a modernization process has undoubtedly occurred in which schools have become more integrated structures and pay more attention to secondary processes (coordinating, evaluating, maintaining external contacts). This means that alignment might gradually replace loose coupling, and that collegial support, data driven teaching and task-related cooperation between teachers are considered as important levers of instructional improvement [see the seminal review by Elmore (2000)]. At the same time the development in the conceptualization of leadership can be read as a gradual return to the recognition of the importance of teachers as resourceful practitioners. The question might be raised to what extent this process has led to an accompanying retreat of personal leadership ambitions. The leadership functions are still in place, and have sometimes even been elaborated. Personal leadership still has a role in school organizations that are on the one hand more integrated, but on the other still maintain important characteristics of ‘‘loose coupling’’. This role might be well captured with the concept of ‘‘meta-control’’, which literary means control of control. School leadership as meta-control would seek to fully exploit the potential of distributed leadership, organizational leadership and substitutes for leadership while maintaining an overarching outlook on the well functioning of the whole.

Theoretical Foundations The Leadership Construct The behavioral emphasis in the operational concepts of leadership in the empirical research literature reviewed is loosely embedded in school effectiveness and school improvement thinking, but not rooted in more established and more widely


135

applicable leadership models. More generic leadership models such as those by Quinn and Rohrbaugh (1983) and Bolman and Deal (1997) take a broader outlook at organizational effectiveness; including multiple success criteria. For example, in the case of the Quinn and Rohrbaugh ‘‘competing values’’ framework, responsiveness to the environment, formalization of internal procedures and the well being of ‘‘human resources’’ are recognized as effectiveness criteria next to the productivity of primary outputs of the organization. Only a limited set of empirical leadership studies in education have used the Quinn and Rohrbaugh framework; examples are Griffith (2003); Ten Bruggencate (2009), and Maslowski (1997). The competing values framework is build on major organizational theories, open systems thinking (responsiveness), the human relations approach, the theory of the bureaucracy (the organizational process models that emphasizes formalized procedure) and the rational goal model (productivity, effectiveness). The use of theory-based leadership models has the advantage of a clearer conceptual basis for instrument development, a more comprehensive outlook on organizational effectiveness and offers the possibility of comparability with empirical studies carried out in other societal sectors.

The Conceptual Foundation of Indirect Effect Models The introduction of indirect effect models invites reflection on hypothetical causal associations between leadership variables, intermediary school factors and educational outcomes. Here a connection could be made with integrative or comprehensive, multi-level educational effectiveness models (Scheerens 1992; Creemers 1994; Stringfield and Slavin 1992; Creemers and Kyriakides 2008; Huber and Muijs 2010). Generally speaking these models try to define nested structures of facilitating conditions, teachers facilitating student learning and school organizations and school leaders facilitating effective instruction. In actual fact the connection to comprehensive educational effectiveness models has not been made in the major empirical leadership studies, until very recently [see the description of the studies by Ten Bruggencate et al. (2010) and particularly the one by Heck and Moriyama (2010)]. What is notoriously lacking in most of the earlier indirect effect studies is the inclusion of well defined and objectively measured instructional conditions. A better integration between leadership effect studies and model driven educational effectiveness research appears to be an important development that is about to take place. A key issue is to better understand how certain leadership behaviors indirectly influence student outcomes. On this issue both conceptual work and qualitative study could further illuminate basic expectations and make quantitative studies more targeted. Critical questions should be raised about how exactly school leaders can, most effectively support teaching effectiveness, how much to be expected from ‘‘people strategies’’, like consensus building and how much from direct support and coordination of curriculum and instructional conditions and optimization of internal evaluation and monitoring practices.

136

J. Scheerens

A Conceptual Map In Fig. 5.3 an overview is given of leadership traits, leadership styles, leadership behaviors and potential intermediary variables. The figure shows a global division between person and task-related strategies. So far leadership studies have particularly focused at person-oriented approaches, building consensus and cooperation. A new emphasis on facilitating teaching and learning, not only by people strategies but also by means of ‘‘technology’’ in the broadest sense, might become an additional emphasis in future work.

Relevant personality traits and competencies

Leadership style

Extraversion social appraisal skills

Leadership behaviour

Effectiveness enhancing factors

External contacts Buffering

Enhanced teaching time

intelligence motivation internal locus of control domain specific knowledge conscientiousness

Task-related

Direction setting (goals, standards Monitors curriculum and instruction (managing the instructional program)

Clear goals and standards Opportunity to learn Student monitoring & feedback Structured teaching Active teaching Active learning

Extraversion Social appraisal skills Self confidence

Person-related

HRM & HRD Coaches teachers Recruits teachers Builds consensus

Cohesion among teachers Professionalization Teacher competency Teachers’ sense of self efficacy

Sets values Creates climate

Shared sense of purpose among teachers High expectations Disciplinary climate Supportive climate

Basic human values General moral beliefs Role responsibility

Fig. 5.3 Intermediary causal structure of leadership at school

Methodological Issues Study Design Most of the empirical studies that were reviewed are cross sectional, based on survey data. Several more recent studies (e.g. Day et al. 2009; Heck and Moriyama 2010) have succeeded in addressing achievement gain over time. Given the


137

inherent weaknesses of cross-sectional studies to support causal inference, among other things due to the threat of reversed causality, realizing longitudinal designs means an important step forward. Direct effect models were usually tested by means of multi-level analyses, while indirect effect models make use of structural equation modeling, where data were usually defined or aggregated to the school level. In future studies, particularly when instructional conditions at classroom level would be added, use of multi-level structural equation modeling could be considered. Taking a next step forward in designing quantitative effect studies by means of true or quasi experiments, unfortunately, appears rather unlikely. It does not seem feasible to randomly assign schools to school leaders who would agree on following a particular leadership strategy. In cases where randomized field trials might be realizable specific leadership approaches would most likely be ‘‘packaged’’ with other school conditions (as in the case of Comprehensive School Reform projects) and not allow for an assessment of leadership effects per se. This would imply that the best that could be done in terms of improving the design of empirical leadership studies would be to try and do justice to the complexity of indirect effect models by using longitudinal data, taking in consideration the multilevel structure of the data, while also possibly addressing reciprocal effects and non linear relationships. Despite these important sophistications in structural equation modeling, the approach in question should still be seen as one lens to look at a very complex reality. The advantage of quantitative analyses being that research syntheses and accumulation of results is a feasible endeavor, which is hardly the case for some of the qualitative alternatives to be discussed below.

Instruments and Data Collection Measures Leadership characteristics and intermediary variables are usually measured by means of questionnaires to principals, teachers and sometimes students. Student outcomes are mostly achievement in basic subjects such as mathematics and reading. In the best case leadership is measured by collecting data from teachers, but many studies are based on self reports. It is the impression that the questionnaires that are used are often of a high inference and judgmental rather than a factual nature (for example measuring school size by asking whether it is considered to be excessive). Particularly in the case of self reports from school leaders social desirable answering patterns are hard to rule out. In some cases doubts might be raised about the independence of measures of leadership dimensions and intermediary variables. It may be the case that variables show a positive correlation because they measure merely the same underlying dimension. This occurs frequently when variables are composed from Likert scales. When the correlation between such variables is high then regression models may suffer from multi co linearity, which means that in regression analyses it is hard to distinguish the contribution of each of the highly correlated variables. Distributed leadership and staff collaboration, for example, are conceptually very close, and when, in addition, they are rated by the same respondents, e.g. teachers,

138

J. Scheerens

one could wonder whether correlations reflect a common underlying dimension rather than an association between to separate variables. As is the unfortunate custom in educational effectiveness research, each researcher tends to develop his or her own measurements, with the results that there are few well researched and validated instruments around, to measure the central school variables. The one positive exception as far as leadership is concerned is the Principal Instructional Management Rating Scale, developed by Hallinger (1984). It is an as yet unfulfilled ambition of the educational effectiveness research program to develop a core set of standardized instruments to measure key process variables. Improvement in this direction is helped by the proliferation of recent studies making use of the Dynamic Educational Effectiveness Model, by Creemers and Kyriakides (2008).

Alternative Designs It is quite tempting to move to qualitative case studies to avoid pressing leadership in the somewhat narrow mall of surveys and structural equation modeling. ‘‘I would say the effect of the HT (head teacher, J.S.) is like the so-called butterfly effect in climate changes or chaos theory. That is, a small trigger can mobilize as well as immobilize the whole school mechanism to work’’ (James Ko, personal communication). Indeed the way leadership affects schooling might differ from a kind of continuously ongoing force that the linear models would assume. However, it is questionable whether qualitative research could resolve this problem. Without any doubt, richer descriptions of leadership at work could be obtained, but relating such descriptions to factual data on student achievement is a different matter. One interesting approach, followed in the study by Day et al. (2009) was to select schools that had improved their achievement over a period of 3 years, and next collect qualitative data on leadership behavior and other aspects of school functioning. Generally the importance of most of the school variables addressed was confirmed. This kind of work might increase understanding of how school functioning is perceived by the actors in the field, but cannot be seen as convincingly supporting causal claims. Another interesting alternative way of looking at school leadership effects is focused on the role of the school leader in ‘‘turning around failing schools’’. There is a general understanding that school leaders play a crucial role in this process. The American What Works Clearing House (2008) has published a review of ten case studies that have addressed the issue. The Clearing House authors note that they consider the strength of evidence of these ten case studies as low. Nevertheless their recommendations about how to go about changing a failing school are interesting: • • • •

Signal the need for dramatic change with strong leadership Maintain a consistent focus on improving instruction Make visible improvements early in the school turnaround process (quick wins) Build a committed staff


139

A final alternative methodology, which is again quantitative, would be to use system dynamics and simulation techniques, to play management games (e.g. Clauset and Gaynor 1982; De Vos and Bosker 1999).

Summary of Results Effect Sizes from Meta-Analyses In this report different sources were used to quantitatively assess leadership effects: our own meta-analyses carried out over the period from 1985 to 2005, a summary of 8 other meta-analyses and a recently conducted meta-analysis, based on 14 studies and 28 replications that were conducted during the last 6 years and which used indirect effect models. Over this last period we also examined 8 direct effect studies, but had to conclude that the majority of studies had not documented analysis results in a sufficiently explicit way to allow for a quantitative synthesis. Instead a so-called ‘‘vote count’’ on significant replications was carried out to provide an impression of the results of these direct effect leadership studies. The results of our current metaanalysis, presented in Chap. 4, are presented once more in Table 5.1. Table 5.1 Summary of indirect effects Author and year Leadership measure Day et al. (2010)

Heck and Hallinger (2009) Heck and Hallinger (2010)

Integrated leadership (primary level) Integrated leadership (secondary level) Initial distributed leadership Change in leadership Distributed leadership idem


Change in leadership idem Collaborative leadership idem Integrated leadership: school leadership

Achievement measure

Total effect

Change in pupil outcomes over 3 years idem

0.001

0.04

Growth rate math

0.03

idem Initial reading scores (year 2) Initial math scores (year 2) Growth rate reading Growth rate math Added year effect reading Added year effect math Proportion of students reaching or exceeding the state’s proficient level

0.09 0.02 0.02 0.10 0.10 0.16 0.14 0.24

(continued)

140

J. Scheerens

Table 5.1 (continued) Author and year Leadership measure Leithwood et al. (2006)

School leadership idem


Collective leadership



Louis et al. (2010)

De Maeyer et al. (2007) Opdenakker and Van Damme (2007) Ross and Gray (2006) Supovitz et al. (2010) Ten Bruggencate (2009)

idem Instructional leadership

Shared leadership Integrated leadership

Achievement measure

Total effect

2-year mean achievement score 2-year mean achievement gain Percentage of students meeting or exceeding the proficiency level on language and math tests Percentage of students per school achieving level 3 or higher at math and literacy test idem Percentage of students at school level meeting or exceeding the proficiency level 2005 math tests idem Reading

0.11 -0.06 0.24

0.11

0.15 0.05

0.03 -0.02

idem Participative professionally oriented leadership

Math Math



0.22


English language and arts Math Average exam mark

0.03

idem Leadership style: rational goals (teacher perceptions) Leadership style: internal process (teacher perceptions) Leadership style: human relations (teacher perceptions) Leadership style: open systems (teacher perceptions) Leadership style: rational goals (principal perceptions) Leadership style: open systems (principal perceptions)

-0.16 0.006

-0.009 -0.16 0.003 0.004 -0.18 0.002 -0.31 (continued)


141

Table 5.1 (continued) Author and year Leadership measure Ten Bruggencate et al. (2010)

Achievement measure

Time spent on instructional leadership Time spent on administrative duties Time spent on supervising teachers Time spent on public relations 15 publications; 34 effect measures

Mean SE mean Without Ten Bruggencate (2009) Mean 14 publications; 28 effect measures SE mean

Total effect 0.02 -0.09 0.09 0.04 0.031 (0.020) 0.060 (0.018)

Combined with the results of our earlier meta-analysis and the summary of seven other meta-analyses the quantitative evidence from this study can be compiled as shown in Table 5.2. Table 5.2 Summary of mean effect sizes from meta-analyses discussed in this report Source Average effect size (correlation coefficients) Meta-analysis Scheerens et al. (2007) Summary of seven meta-analyses Studies 2005–2010, this report

0.05 0.18 0.06

In our own work we consistently find effect sizes in the order of 0.05, while some other meta-analyses have found much higher effect sizes. We have no clear explanation for this fact. When we would venture to average the three coefficients in Table 5.2 we would arrive at an effect size of 0.09. The vote count procedure on the 12 direct effect studies pointed out that of a total of 94 replications, i.e. associations between leadership and student achievement, 70 were none significant, 4 significant negative and 20 significant positive.

Interpretation of Effect Sizes According to Cohen’s standards for interpreting effect sizes,1 our results on leadership effects should be interpreted as negligible to small. It should be noted however that several authors argue that Cohen’s standards are to be considered as too conservative, and do not match the practical significance of malleable school

1

According to Cohen (1988), small effects are in the order of r = 0.10, medium effects r = 0.30 and large effects r = 0.50 or higher.

142

J. Scheerens

variables. Richard et al. (2003, cited by Baumert et al. 2006) found a mean correlation of r = 0.21 in their meta-analysis of meta-analyses in social psychology, and proposed a modification of Cohen’s classification, considering a correlation of 0.30 to indicate a large effect (p. 339). Baumert et al. (2006) propose the learning gain during one school year as a realistic standard to express effects of schooling. They cite several studies indicating that this learning gain has the magnitude of about d = 0.30, which would be comparable to a correlation of 0.15. These authors also discuss a method to compute effect sizes developed by Tymms et al. (1997), which, when applied to a practical example, suggests that effect sizes of about r = 0.15–0.20 (small to medium, according to Cohen’s standards) would equal the learning gain in one school year, which they consider an effect of huge practical relevance. Seen in this light the effect size of 0.09 that we arrive at when we average the results summarized in Table 5.2, might perhaps be upgraded in its rating for practical significance. Yet, the literature on estimating year effects of schooling shows huge differences between subjects, grade levels and national contexts; with coefficients as high as 0.45 (Luyten 2007), so that this yardstick against which to compare leadership effects is not a very stable one. Given the theoretical models of the school that we discussed and the long causal chain between leadership actions and student achievement results, small effect sizes should not really come as a surprise with the kind of research designs that were used in the majority of studies analyzed. In fact it is rather the effect sizes in the order of magnitude of r = 0.40 that should be seen as remarkable.

What are the Most Promising Intermediary Variables from Indirect Effect Studies? In earlier reviews we noted that the number of studies in which indirect effects of school leadership were assessed was still too small to allow for any conclusions about promising mediating variables, i.e. variables that mediate the influence of school leadership behavior on student outcomes. In this study we have quantitatively analyzed 14 indirect effect studies, and summarized the most important indirect effects in Table 4.7. This table is presented once more in Table 5.3. The intermediary variables that ‘‘worked’’ can be broadly categorized into ‘‘organizational capacity’’ (improvement focus, standard setting, quality of student support, professional capacity of the staff, systematic evaluation), ‘‘teachers’ commitment and cooperation’’, ‘‘academic climate’’ and ‘‘instructional conditions’’. It is interesting to note than in recent studies, those conducted in 2009 and 2010, ‘‘instructional conditions’’ are introduced as a relevant type of intermediary variable, while, in the earlier studies, this category was rather treated as a black box. In Chap. 3, when referring to the study by Heck and Moriyama (2010), this new focus was seen against the background of a better integration of leadership effect studies in integrative modeling of school and instructional effectiveness.


143

Table 5.3 Most important indirect effects and intermediary variables Author and Leadership measure Achievement Intermediate Year measure variables

Day et al. (2010) Heck and Hallinger (2009) Heck and Hallinger (2010)





Integrated leadership Change in pupil (secondary level) outcomes over 3 years Initial distributed Growth rate math leadership Change in leadership Growth rate math Initial reading scores Distributed leadership (year 2) Distributed Initial math scores leadership (year 2) Change in leadership Growth rate reading Change in leadership Growth rate math Collaborative Added year effect leadership reading Collaborative Added year effect leadership math Integrated Proportion of leadership: students reaching school leadership or exceeding the state’s proficient level School leadership 2-year mean achievement score idem 2-year mean achievement gain Collective Percentage of leadership students meeting or exceeding the proficiency level on language and math tests Distributed Percentage of leadership students per school achieving level 3 or higher at math and literacy test Distributed idem leadership


Leadership distribution in the school Change in capacity

0.04

0.03

Change in capacity School capacity

0.08 0.02

School capacity

0.02

Change in capacity Change in capacity Instructional practices Instructional practices School conditions

0.10 0.10 0.16

(Not reported)

0.11

(Not reported)

-0.06

0.14 0.24

None

0.24


0.11


0.15

(continued)

144

J. Scheerens

Table 5.3 (continued) Author and Year Leadership measure

Achievement measure


Focused instruction Percentage of students at school level meeting or exceeding the proficiency level 2005 math tests Instructional idem Professional leadership community and focused instruction Shared leadership idem Professional community and focused instruction De Maeyer et al. Integrated leadership Reading None (2007) Integrated leadership Reading Academic climate Integrated leadership Math None Transformational Composite school Teacher Ross and Gray (2006) leadership score commitment to School mission Transformational Composite school Collective teacher leadership score efficacy and Teacher commitment to community partnerships Transformational Composite school Teacher leadership score commitment to professional community Supovitz et al. Principal leadership English, language Change in (2010) and arts instruction Average exam mark None Leadership style: Ten rational goals Bruggencate (teacher (2009) perceptions) Average exam mark None Leadership style: open systems (teacher perceptions) Leadership style: Average exam mark None open systems (principal perceptions) Louis et al. (2010)


Combined effect via this path 0.03

0.02

0.03

-0.27 0.25 -0.15 0.13

0.12

-0.07

0.02 -0.16

-0.18

-0.32

(continued)

5 Summary and Conclusion Table 5.3 (continued) Author and Year Leadership measure

Ten Bruggencate Time spent on et al. (2010) instructional leadership Time spent on administrative duties Time spent on administrative duties Time spent on administrative duties Time spent on supervising teachers Time spent on supervising teachers Time spent on supervising teachers Time spent on public relations

145

Achievement measure



Math achievement

Valuing math

Math achievement

None

-0.04

Math achievement

Valuing math

-0.02

Math achievement


-0.03

Math achievement

None

0.04

Math achievement

Valuing math

0.01

Math achievement


0.03

Math achievement


0.04

0.02

The most relevant paths found in indirect effect models with the combined effects

These four main categories of intermediary variables are well-matched to the leadership emphases distinguished by Leithwood (e.g. Leithwood and Jantzi 2008), setting directions, developing people, redesigning the organization and managing the teaching and learning program. This connection is systematically summarized in Table 5.4. Table 5.4 Connection between leadership emphases and intermediary conditions Leadership emphasis (Leithwood) Main categories of intermediary conditions Setting directions Developing people Redesigning the organization Managing the teaching and learning program

Academic climate Professional capacity of the staff, cooperation and commitment of staff Organizational capacity Instructional conditions

The data, summarized in Table 5.4, are still too limited to draw conclusions about the relative importance among the criteria. Further quantitative and qualitative work would be needed to strengthen the knowledge base on indirect

146

J. Scheerens

leadership effect models. A semi structured qualitative approach, for instance, might take the available research results and operational definition of the key intermediary variables as a starting point for qualitative reflection of acting school leaders, in order to better understand the way they perceive indirect causation in their work.

Implications for Educational Policy and Practice Schools Need School Leaders When examining the results of school leadership effect studies over almost three decades we find rather small direct and indirect leadership effects. Theoretical work on the school as an organization clarifies why we should not have expected high leadership effects in the first place. We have noted that in the development of school leadership concepts over time the notion that schools have many ‘‘substitutes’’ for leadership has been re-discovered, and in some recent studies of distributed and organizational leadership, focused action of one central leader has practically disappeared from the scene. Qualifying the potential of school leadership in this way does not mean that we can do away with school leaders. Schools, like any kind of organization, need leadership. The theoretical work and results of empirical studies highlighted in this report suggest that in ‘‘normal’’ situations of average schools a ‘‘lean’’ kind of management might be sufficient, which would make maximum use of the available substitutes and self organization offered by the school staff and other provisions. We have compared this kind of management to ‘‘meta control’’, which could be interpreted as orchestrating the control by the other actors on the school scene.

The Toolkit for School Leaders as Meta-Controllers The concept of meta-control originates from control theory (De Leeuw 1990). According to his ‘‘control paradigm’’, four major types of direct-control can be distinguished: routine control, adaptive control, goal control and environmental control. Routine control is about the day-to-day monitoring of an organization’s primary process. In the case of schools, under normal circumstances, very little monitoring of teachers and teaching is required. The narrow interpretation of instructional leadership is close to this kind of routine control. Meta-control directed at this kind of routine control could be seen as creating favorable conditions for teachers to do their work independently. It could mean that, on the one hand, the school leader protects teachers against disturbing external influences (buffering) and, on the other hand provides facilitation in the sense of opportunities


147

for professional development, alignment among staff, feedback and provision of the necessary teaching resources. Taking care of and overseeing administrative and clerical tasks of the school could be seen as part of the buffering function. Adaptive control refers to the supervision and change of the organization’s structure and core processes. Creation of new structures for teacher cooperation and the school wide adoption of specific ICT applications are examples of adaptive control. Meta-control directed at organizational structures and key processes is close to the management of change and transformational leadership. A school leader as a meta-controller would also need to oversee the pro’s and cons of structural school reform, as compared to ‘‘simply’’ optimizing normal functioning (routine control). Similarly adaptively oriented meta-control would have to strike a balance between supportive, routine and innovative aspects of the functioning of the organization. Goal control has to do with upholding performance standards and soliciting agreement on the core objectives of the organization. Goal control as meta-control recognizes that within the controlled system, in our case the school’s sub-units (i.e. teachers) have their own goals. In this case the meta-controller has a task in coordinating the individual goals and uniting them under a common school mission. This kind of goal control is close to the extended view of instructional leadership, described in the first chapter and to transformational leadership. In environmental control leaders influence the functioning of the organization by means of putting into play stimulants from the environment. This role of leadership becomes more important as schools are increasingly operating in networks or as part of higher level organizations, such as school districts. In short school leaders as meta-controllers need to have a broad overview of key areas of organizational functioning, a keen eye for self steering and self organization and a detached attitude of taking matters in their own hand (diverting from meta-, to direct-control).

The Special Case of Leadership in Turning Around Failing Schools The message for educational practice and policy so far was: despite the low effect sizes found in the empirical studies reviewed there is no doubt that schools need leadership, but, in normal conditions of average schools, such leadership can be ‘‘lean’’ and indirectly facilitating (the concept of meta-control). At this point it is worthwhile to introduce further nuance by looking at different kinds of schools. One of the interesting findings from the qualitative part of the study by Day et al. (2009) was that they noted that school leaders in schools who had progressed from low to average performance had other leadership priorities than schools that departed from an already higher level of functioning. In the early phase of improvement leadership was particularly directed at improvement of the physical environment of the school, standard setting and installing performance

148

J. Scheerens

management systems, in later phases the focus shifted to use of data, distributed leadership and providing an enriched curriculum (ibid 10). Looking somewhat deeper in the issue of school differences, the usual picture of the distribution of school performance, in industrialized countries, is as indicated in Fig. 5.4.

Fig. 5.4 Distribution of schools means in mathematics, Dutch (HAVO) secondary schools

This typical pattern shows that performance differences are only statistically significant at the extremes. This means that performance among schools in a country is usually characterized by a broad range of schools performing close to the average, and relatively few low achieving and high achieving schools (at about 5–10% at the upper and lower ends of the distribution). Slavin (1996, 1998) has based a typology of school functioning on these distributional properties of school performance. He makes a distinction between ‘‘seed’’, ‘‘brick’’ and ‘‘sand’’ schools. Of ‘‘seed’’ schools he says that. ‘‘Such schools are ones in which staff is cohesive, excited about teaching, led by a visionary leader willing to involve the entire staff in decisions, and broadly aware of research trends and ideas being implemented elsewhere.’’ (p. 1303). The second category of schools, ‘‘brick schools’’, Slavin describes as schools that would like to do a better job, but do not perceive the need of the capability to develop new curricula. According to his categorization these are schools with good relations among staff and leadership, a positive orientation toward change, and some degree of stability in the school and its district. Finally, as a third category, he refers to schools ‘‘in which even the most heroic attempts at reform are doomed to failure. Trying to implement change


149

in such schools is like trying to build a structure out of sand’’ (ibid 1303). Accordingly he refers to these schools as ‘‘sand’’ schools. The way Slavin connects this categorization of schools to school improvement approaches has clear implications for leadership. School improvement of the organizational development kind (in which school staffs are engaged in an extended process of formulating a vision, identifying resources—such as external assistance, professional development and instructional materials—to help the school toward its vision) is considered only feasible in ‘‘seed schools’’, which he estimates at 5% of all schools in the USA. Sand schools, also about 5% of all schools would require fundamental changes before they can support any type of school change. The overall majority of schools, according to Slavin, are the brick-schools and they could most efficiently benefit from what he calls comprehensive reform models. His ‘‘Success for All’’ program is an example. Comprehensive reform models provide schools with specific student materials, teachers’ manuals, focused professional development, and relatively prescribed patterns of staffing, school governance, internal and external assessment and other features of the school organization. For the low functioning school at the extreme of the distribution, Slavin’s ‘‘sand’’ schools, more leadership effort is required than for the average and low performing schools. Moreover this leadership should be more directive and oriented at basic issues like resourcing, standard setting and profiling leadership. The model of school leadership as meta-control would be more appropriate for the broad band of average schools. In line with Slavin’s perspective on school improvement, this role would include an important element of liaison with external support organizations. Within the context of ‘‘evidence based’’ school improvement a new brokerage role for school leadership might be required. In the literature on ‘‘failing schools’’ weak leadership is diagnosed as one of the facets of low performance. Stringfield (1998) mentions the following characteristics of failing schools: • • • • • • • • • •

Lack of academic focus Teachers working in isolation Academic periods starting late and ending early Lack of coordination between teachers in use of textbooks Bureaucratic leadership, not curriculum or instruction oriented Head teachers passive in teacher recruitment Lack of teacher assessment No public rewards for students’ academic excellence Difficulties in maintaining funding Underutilization of library

In summary then, failing schools need more leadership than average schools, and differently directed leadership. In failing schools leadership should be looking at basic aspects of school functioning, like resourcing (human and material), staff and leadership development, textbook choice and curriculum standards.

150

J. Scheerens

Efficiency in School Management If, as perhaps this report underlines, it is quite difficult to properly assess school leadership effects in research studies, it would seem to be even more difficult to evaluate leadership in practical contexts. This would mean that in the absence of performance-oriented practical control, overproduction of management would be a real danger. The school leadership literature in its zeal to underline the crucial role of leadership may have underscored the potential for efficiency in schools as professional bureaucracies. The view of a lean kind of leadership, defined in terms of meta-control, might be a perspective that could be helpful in the avoidance of ‘‘too much management’’. Conditions like increasing scale in schooling, and placing several schools under common governance are likely to create a further upward pressure on total management provision. If schools are operating in networks and districts, the issue of task division between systemic governance and site leadership and management is an additional challenge to keep ‘‘managerial overhead’’, within limits. In this context it would be interesting to develop quantitative indicators on management/teacher or management/pupil ratio’s, and benchmark these on international comparative data.

References Baumert, J., Lüdtke, O., & Trautwein, U. (2006). Interpreting effect sizes in large-scale educational assessments. Berlin: Max Planck Institute for Human Development. Bolman, L. G., & Deal, T. E. (1997). Reframing organizations: Artistry, choice, and leadership (2nd ed.). San Francisco, CA: Jossey-Bass. Clauset, K. H., & Gaynor, A. K. (1982). A systems perspective on effective schools. Educational Leadership, 40(3), 54–59. Cohen, D. K. (1988). Teaching practice. Plus ça change. In P. Jackson (Ed.), Contributing to educational change: Perspectives on research and practice. Berkeley, CA: McCutchan. Creemers, B. P. M. (1994). The effective classroom. London: Cassell. Creemers, B., & Kyriakides, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. New York: Routledge. Day, C., Sammons, P., Hopkins, D., Harris, A., Leithwood, K., Gu, Q., et al. (2009). The impact of school leadership on pupil outcomes. Nottingham, UK: The National College for School Leadership. De Leeuw, A. C. J. (1990). Organisaties: Management, analyse, ontwerp en verandering: Een systeem visie. Assen/Maastricht: Van Gorcum. De Maeyer, S., Rymenans, R., van Petegem, P., van den Bergh, H., & Rijlaarsdam, G. (2007). Instructional leadership and pupil achievement: The choice of a valid conceptual model to test effects in school effectiveness research. School effectiveness and School Improvement, 18(2), 125–145. De Vos, H., & Bosker, R. J. (1999). Simulation of educational effects. In C. van Dijkum, D. DeTombe & E. van Kuijk (Eds.), Validation of simulation models (pp. 88–111). Amsterdam: SISWO Publication 403 Edmonds, R. (1979). Effective schools for the urban poor. Educational Leadership, 37, 15–24.


151

Elmore, R. (2000). Building a new structure for school leadership. Washington, DC: The Albert Shanker Institute. Griffith, J. (2003). Schools as organizational models: Implications for examining school effectiveness. Elementary School Journal, 104(1), 29–47. Hallinger, P. (1984). Principal instructional management rating scale. New York, NY: Leading Development Associates. Hallinger, P., & Heck, R. H. (2010). Leadership for learning: Does collaborative leadership make a difference? Educational Management Administration and Leadership, 38(6), 654–678. Heck, R. H. & Hallinger, P. (2009). Testing a dynamic model of organizational leadership and school improvement effects on growth in learning. Paper presented at the Annual Meeting of American Educational Research Association: Disciplined Inquiry: Education Research in the Circle of Knowledge, San Diego, CA. Heck, R. H., & Hallinger, P. (2010). Testing a longitudinal model of distributed leadership effects on school improvement. The Leadership Quarterly, 21, 867–885. Heck, R. H., & Moriyama, K. (2010). Examining relationships among elementary schools’ contexts, leadership, instructional practices, and added-year outcomes: A regression discontinuity approach. School Effectiveness and School Improvement, 21(4), 377–408. Huber, S. G., & Muijs, D. (2010). School leadership effectiveness. Studies in Educational Leadership, 10, 57–77. Kerr, S., & Jermier, J. (1978). Substitutes for leadership: Their meaning and measurement. Organizational Behaviour and Human Performance, 22, 375–403. Leithwood, K., Jantzi, D., & McElheron-Hopkins, C. (2006). The development and testing of a school improvement model. School Effectiveness and School Improvement, 17(4), 441–464. Leithwood, K., & Jantzi, D. (2008). Linking leadership to student learning: The contributions of leader efficacy. Educational Administration Quarterly, 44, 496–528. Leithwood, K., & Mascall, B. (2008). Collective leadership effects on student achievement. Educational Administration Quarterly, 44, 529–561. Leithwood, K., Patten, S., & Jantzi, D. (2010). Testing a conception of how school leadership influences student learning. Educational Administration Quarterly, 46, 671–706. Louis, K. S., Dretzke, B., & Wahlstrom, K. (2010). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21(3), 315–336. Luyten, H. (2007). Opportunities to utilize already existing quantitative data for PISA 2009. Discussion paper, University of Twente, Enschede Maslowski, R. (1997). Schoolcultuur: kenmerken en veranderingsmogelijkheden [School culture: characteristics and levers for change]. In B.P.M. Creemers et al. (Eds.), Handboek Schoolorganisatie en Onderwijsmanagement (pp. B1400/1–B1400/25). Alphen aan den Rijn: Samsom Tjeenk Willink. Opdenakker, M.-C., & Van Damme, J. (2007). Do school context, student composition and school leadership affect school practice and outcomes in secondary education? British Educational Research Journal, 33(2), 179–206. Quinn, R. E., & Rohrbaugh, J. (1983). A spatial model of effectiveness criteria: Towards a competing values approach to organizational analysis. Management Science, 29(3), 363–377. Richard, F. D., Bond, C. F., Jr., & Stokes-Zoota, J. J. (2003). One hundred years of social psychology quantitatively described. Review of General Psychology, 7, 331–363. Ross John, A., & Gray, P. (2006). School leadership and student achievement: The mediating effects of teacher beliefs. Canadian Journal of Education, 29(3), 798–822. Scheerens, J. (1992). Effective schooling, research, theory and practice. London: Cassell. Scheerens, J., & Bosker, R. (1997). The foundations of educational effectiveness. Oxford: Pergamon. Scheerens, J., Luyten, H., Steen, R., & Luyten-de Thouars, Y. (2007). Review and meta-analyses of school and teaching effectiveness. Enschede: Department of Educational Organisation and Management, University of Twente. Slavin, R. E. (1996). Success for all. Lisse: Swets & Zeitlinger.

152

J. Scheerens

Slavin, R. E. (1998). Every child, every school: Success for all. Thousand Oaks, CA: Corwin Press. Stringfield, S. (1998). Choosing success. American Educator, 22(3), 1–5. Stringfield, S. C., & Slavin, R. E. (1992). A hierarchic al longitudinal model for elementary school effects. In B. P. M. Creemers & G. J. Reezigt (Eds.), Evaluation of effectiveness (ICO Publication 2). Groningen: ICO. Supovitz, J., Sirinides, Ph., & May, H. (2010). How principals and peers influence teaching and learning. Educational Administration Quarterly, 46, 31–56. Ten Bruggencate, G., Luyten, H., & Scheerens, J. (2010). Quantitative analysis of international data, exploring indirect effect models of school leadership. Enschede: University of Twente. Ten Bruggencate, G. (2009). Maken schoolleiders het verschil? (Do school leaders make a 438 difference?). Enschede: University of Twente. Tymms, P., Merrell, C., & Henderson, B. (1997). The first year at school. A quantitative investigation of the attainment and progress of pupils. Educational Research and Evaluation, 3(2), 101–118. Weick, K. E. (2001). Making sense of the organization. Chichester: Wiley.

School Leadership Effects Revisited: Review and Meta-Analysis of Empirical Studies (SpringerBriefs in Education)

The Ethical Dimensions of School Leadership (Studies in Educational Leadership)

Participatory Learning: Religious Education in a Globalizing Society (Empirical Studies in Theology) (Empirical Studies in Theology)

Leaders and Leadership in Education

Leaders and leadership in education

Leadership and Discovery (Jepson Studies in Leadership)

Normativity And Empirical Research In Theology (Empirical Studies in Theology)

Leadership and Discovery (Jepson Studies in Leadership)

The Limits to Growth Revisited (SpringerBriefs in Energy Energy Analysis)

Current issues in school leadership

The Art of School Leadership

Studies in Church Leadership

Teacher Supply: The Key Issues (Continuum Empirical Studies in Education)

Leadership and the Liberal Arts: Achieving the Promise of a Liberal Education (Jepson Studies in Leadership)

Leadership Gender and Culture in Education

Leadership and management development in education

Theory of Education (Continuum Studies in Education)

Empirical Methods and Studies in Software Engineering

Youth Leadership in Sport and Physical Education

Leadership in International Business Education and Research

School District Leadership Matters

Gender Leadership Review

School Leadership for the 21st Century (School Leadership)

SpringerBriefs in Child Development)

Developing Successful Leadership (Studies in Educational Leadership)

Social Constructionism and Theology (Empirical Studies in Theology) (Empirical Studies in Theology)

Leadership and Diversity: Challenging Theory and Practice in Education (Education Leadership for Social Justice)

Lincoln's Legacy of Leadership (Jepson Studies in Leadership)

Women and School Leadership: International Perspectives (Suny Series in Women in Education)

School Leadership Effects Revisited: Review and Meta-Analysis of Empirical Studies (SpringerBriefs in Education)

The Ethical Dimensions of School Leadership (Studies in Educational Leadership)

Participatory Learning: Religious Education in a Globalizing Society (Empirical Studies in Theology) (Empirical Studies in Theology)

Leaders and Leadership in Education

Leaders and leadership in education

Leadership and Discovery (Jepson Studies in Leadership)

Normativity And Empirical Research In Theology (Empirical Studies in Theology)

Leadership and Discovery (Jepson Studies in Leadership)

The Limits to Growth Revisited (SpringerBriefs in Energy Energy Analysis)

Current issues in school leadership

The Art of School Leadership

Studies in Church Leadership

Teacher Supply: The Key Issues (Continuum Empirical Studies in Education)

Leadership and the Liberal Arts: Achieving the Promise of a Liberal Education (Jepson Studies in Leadership)

Leadership Gender and Culture in Education

Leadership and management development in education

Theory of Education (Continuum Studies in Education)

Empirical Methods and Studies in Software Engineering

Youth Leadership in Sport and Physical Education

Leadership in International Business Education and Research

School District Leadership Matters

Gender Leadership Review

School Leadership for the 21st Century (School Leadership)

SpringerBriefs in Child Development)

Developing Successful Leadership (Studies in Educational Leadership)

Social Constructionism and Theology (Empirical Studies in Theology) (Empirical Studies in Theology)

Leadership and Diversity: Challenging Theory and Practice in Education (Education Leadership for Social Justice)

Lincoln's Legacy of Leadership (Jepson Studies in Leadership)

Women and School Leadership: International Perspectives (Suny Series in Women in Education)

Recommend Documents