What PD leaders and learners need to know about assessment reporting in high pressure accountability contexts

High-stakes accountability pressures can conflict with faculty implementation of best disciplinary and pedagogical practices (Meuwissen, 2017). In both public postsecondary and public K-12 education in the United States, the tension between, on the one hand, funding-  and policy-driven expectations at the state and institution levels and the high-stakes faculty evaluations based on those expectations, and, on the other, subject-specific learning and teaching, raises the question of who and what should define and control what happens in the classroom as well as the question of what should define student success.

Professional development (PD), when defined by a federally regulated institution rather than a teaching profession, runs the risk of over-emphasizing high-stakes assessments at the expense of developing students’ thinking and learning dispositions and
capacities (Meuwissen, 2017).

When technology-related PD is implemented primarily at the institutional rather than disciplinary level, this risk may be amplified. Yet, the reality is that technological, pedagogical, and teaching content knowledge all constrain one another in ways that relate to disciplinarily and developmentally appropriate teaching (Koehler & Mishra,
2009).

This post explores the origins of the mandated student and teacher assessment data that drive current state and institutional accountability, and hence that increasingly control faculty decisions about what happens in the classroom. I also consider how “research data” can be defined and used in ways that objectify and constrain, rather than support, teaching decisions and practices that may be truly “best” in terms of
supporting students’ learning.

 

Tracing the history of legislative influence on K-12 teacher assessment

Ingersoll and Collins (2019) define professional work such as teaching as work that “involves highly complex sets of skills, intellectual functioning, and knowledge that are neither easily acquired nor widely held. For this reason, professions are often referred to as ‘knowledge-based’ occupations. In the professional model, practitioners are,
ideally, first provided with the training, resources, conditions, and autonomy
to do the job, and then held accountable for doing the job well” (p. 176).

Before discussing the assumptions about, uses of, and cultural/ideological contexts surrounding student and teacher assessment data, it is important to understand some of the cultural and legislative history that has resulted in current models of instructional supervision and in the way the accountability movement is manifesting itself at both K-12 and postsecondary levels.

Although the U.S. Constitution makes no provision for federal control of education and the Tenth Amendment stipulates that powers not delegated to the federal government by the Constitution are reserved for the states, federal control entered the American educational system through the indirect means of requirements attached to funding (Lunenburg, 2019).

However, “the impetus for public education came from the federal government” through the Northwest Ordinances of 1785 and 1787, which designated a sixteenth of each township for public schools. A century later, The Hatch, Morrill, and Adams Acts established grants as a form of federal support for “land grant” colleges (Lunenburg, 2019). Federal intervention through funding requirements increased in the mid twentieth century, in particular through the Elementary and Secondary Education Act (ESEA) of 1965 under President Lyndon Johnson (Lunenburg, 2019). ESEA must be reauthorized every 5 or 6 years and includes sections such as Title I, which includes a formula for extra funding for schools in high poverty areas.

The 1988 reauthorization of ESEA called for adoption of measurable standards for student academic performance. A movement to establish standards in core subjects called America 2000 was enacted under President George W. Bush the following year (Lunenburg, 2019). The ESEA reauthorization in 1994 under President Bill Clinton called for the creation of assessments aligned with these standards; the establishment of benchmarks for “adequate yearly progress;” disaggregat[ion of] data by race, gender, disability, socioeconomic status, and limited English proficiency;” and corrective action after two years without improvement (Lunenburg, 2019).

Lunenburg (2019) identifies this as the point at which the focus shifted from minimum competency to proficiency and accountability became a primary focus. This shift continued in the No Child Left Behind Act (NCLB), which encompassed and amended ESEA in 2002 and added the requirement “that states administer tests to all public school students” and “narrow the test score gap between economically advantaged students” and every other subgroup as well as calling for sanctions for schools who did not meet state-developed proficiency levels for any subgroup (Lunenburg, 2019). States developed assessments of their content standards and defined achievement or “performance” standards to measure levels of proficiency students achieve.

Lunenburg (2019) notes two important NCLB stipulations: that teachers meet designated requirements as “highly qualified” and that programs be based on “scientific research.” Both are important for understanding the current culture of education and educational supervision: “The scientifically based research provision in the law [was] notable for its implications for instructional methods, curricular materials, and instructional supervision, areas of education that [had] been historically outside the realm of federal involvement” (Lunenburg, 2019). Arnold (2019), discussed below, stresses the cultural and ideological ways that “scientific research” can be used in policy making that can control what happens in the classroom.

In 2009, the administration of President Barack Obama used a competitive grant program, Race to the Top (RTTT or RT3), to incentivize development of teacher evaluation policies. Two years later, that administration authorized waivers for some provisions of NCLB for states that achieved 100% student proficiency (Lunenburg, 2019).

The final piece of legislation important for understanding current education, teacher evaluation, and professional development policies in American K-12 schools is the Every Student Succeeds Act (ESSA) of 2015, which became law under the USDOE leadership of Arne Duncan. A key feature of ESSA is the continued focus of NCLB on high standards for all students (Lunenburg, 2019). The goal was to ensure equal opportunity for all students to succeed, regardless of their background or mailing address.

McMunn Dooley, Owens, and Conley (2019) note that the TPA (teacher performance
assessment) systems currently in place were fully developed and implemented in
an average of only 1.3 years from the time that the flexibility waivers were
first made available in 2011, that their development was funded by the Gates
and Wallace foundation, and that there are many critiques of their validity and
reliability.

As studies of the validity, reliability, and effectiveness of TPAs (notably the
Measures of Effective Teachers (MET) study itself carried out by the Gates
Foundation) begin, McMunn Dooley, Owens and Conley carried out a large content
analysis of all state teacher evaluation policies and protocols, finding that
“TPAs rarely evaluated what resources were made available to teachers, what
opportunities were afforded them, or whether teachers actually learn from
professional development sessions that they attend” and that “professional
development seems to be the outcome of evaluation, not the focus of evaluation
itself” (p. 426).

 

A research-based case for shifting the dial toward teacher agency

Ingersoll and Collins (2019) synthesized current data with those from research projects conducted over 20 years; sources of data included the Schools and Staffing Survey (SASS), related Teacher Follow-up Survey (TFS), the largest data source available on K-12 teachers in the U.S., as well as data collected by the Organization for Economic Cooperation and Development (OECD), Programme for International Student Assessment (PISA), and Teaching, Empowering Leading and Learning (TELL) survey. By synthesizing data on the levels of distribution and effects of accountability and control in American schools, Ingersoll and Collins (2019) showed that while public K-12 education in the U.S. is relatively less federally controlled compared to European countries, it is relatively more principal-controlled; that the imbalance between teacher responsibilities and teacher control is increasing; and that while the current accountability climate identifies important issues and problems in American schools, attempts to improve student achievement through professional development and evaluation based on a teacher deficit model that does not give adequate attention to “the character of the teaching occupation, and the character of the organizations in
which teachers work,” including the distribution, mechanism and effects of control
and power, is inadequate.

In a series of multiple regression analyses of TELL data, the authors showed that eight separate measures of teacher control were related to student achievement at a statistically significant level. The two areas that showed the strongest relationship with student achievement were discipline and improvement planning. Data also showed that lower performing schools with sanctions had less teacher turnover, and in fact had turnover comparable to high performing schools, when teachers were given more autonomy.

The authors recommend that improving teacher quality through accountability reforms offers only a one-sided perspective and that “solving the problem of teacher quality” involves also providing teachers with the autonomy and flexibility they need to do their jobs effectively, and assessing school management in light of a balanced approach to control and accountability.

While Ingersoll and Collins (2019) use extensive data sets and analyses to make the case that PD leaders and education managers should balance pragmatic emphasis on high-stakes accountability measures with restoring teacher agency, considering unique school contexts, and evaluating institutional factors, Arnold (2019) uses critical pedagogy to theorize about the reasons for the imbalance between the two emphases. I’ll return to her more theoretical case for shifting the dial toward teacher agency.

 

Data reporting at the postsecondary level

Most colleges and universities are also required to report data to the federal government in order to receive federal student financial aid funding. Rice and Russell (2012) outline the legislative history beginning with the Student Right-to-Know Act of 1990 (SRTK) that mandates graduation reporting for subgroups in order to participate in Title IV programs. Graduation rates, along with persistence and completion rates, are
key institutional measures at the postsecondary level.

The USDOE’s Institute of Education Sciences (IES) makes reported data available and tools for analyzing that data available to constituencies including the states, researchers and policymakers at the institutions themselves, and the general public, as well as using these data to guide or justify federal education policy through several centers, including the National Center for Education Statistics (NCES) (Fink & Muntz, 2012). NCES also collects data through a survey research system. The Postsecondary, Adult, and Career Education Division (PACE) collects information on postsecondary institutions. The main postsecondary dataset made available by NCES is the Integrated Postsecondary Education Data System (IPEDS). The three main tools that IPEDS provides to constituents are College Navigator , the Executive Peer Tool (ExPT), and the Data Center (Fink & Muntz, 2012).

While postsecondary faculty often have considerably more control than K-12 teachers “over the content of their teaching, over the hiring of new colleagues, over the evaluation and promotion of members through peer review, and, hence, over the ongoing content and character of their profession” (Ingersoll & Collins, 2019, p. 166), assessment measures, policies, and college curricula are increasingly influenced by both federal funding guidelines and by the network of grantmaking organizations such as the Gates and Lumina Foundations that have been influential in funding K-12 student and teacher assessment (Addison, 2015).

Rice and Russell (2012) call for ways to “expand and refocus” the conventional use of graduation rate in postsecondary reporting:

Given the paucity of comparable, widely accepted outcome measures for postsecondary education, state and federal policy makers have latched on to student
persistence and graduation measures as key accountability indicators. At the
same time, critics have denounced simple graduation rates as inadequate and
misleading metrics. Despite the many limitations of these measures, graduation
rates have gained attention and influence in the media, in college ranking
systems, and among consumers. As a result, many higher education administrators
are concerned that this narrowly focused concept of student success does not
adequately reflect all that they do, and that it deflects attention and resources from broader institutional concerns. (237)

While postsecondary reporting measures may be critiqued for imbalanced or narrow focus on reporting measures, the growing imbalance between teacher responsibility and teacher control at the K-12 level identified through the synthesized study of Ingersoll and Collins (2019) forms a “datafied” counterpart to Arnold’s (2019) theory-based suggestions for correcting such imbalance.

Theorizing a more balanced model of PD in a high-stakes accountability context

At the K-12 level, Arnold (2019) calls for a counter-balancing of the “cultural shifts that have institutionalized and ideologized education” (p. 583). She argues that NCLB and the Common Core State Standards, and the supervisory models grounded in the standards based accountability movement, emphasize homogeneity over the unique context of each school and student population. Thus the measures that were meant to result in opportunity for all are in reality reducing opportunity, as learning models emphasize efficiency over depth of learning, for example through competency-based “badging” (p. 578). Arnold argues that accountability has rendered not only teaching but leadership less creative, and that “if adults are uncreative and constricted in their thinking, then children will be as well” (p. 587).

Arnold makes a case that assessment reporting can become a matter of “datafication”, as “teachers and leaders…feel pressured to produce the ‘right’ data” and as “self professed evidence-based policy making is premised on the idea that research results can justify political, economic, and social decision making” which becomes a problem if research is “flawed in design, analytics, or reporting” (pp. 578-79).

In response to this diagnosis, Arnold calls for leadership development based in critical pedagogy in order to help leaders such as principals focus on both pragmatics and “contextual diversity.” Critical pedagogy is the interpretive stance that questions the assumptions that lead to inequalities. The practical shifts she calls for include “wise feedback” in teacher evaluation and professional development; “wise feedback,” separate from formal feedback, is similar to the formative feedback teachers provide to
students that includes the assurance they will be successful. Arnold also calls on school leaders to model “good” use of research, by which she means refraining from sending the “underlying message…that university research has more significance than the experiences of those who contribute directly to the school day” and engaging teachers in identifying problems of practice and speaking of “evidence-informed policy rather than evidence-based policy” (pp. 584-585). Her strengths-based, context-based, collaborative suggestions encourage a more humanistic approach to teacher evaluation/professional development and thus to building the teaching and learning culture of an institution.

While professionals—teachers and administrators alike—should
understand the role that standards and standardization play in creating quality
and equality in education, institutional cultures are cultures of people that
by definition cannot be based on abstract and standardized values alone. When it comes to professional development, the studies cited by Meuwissen (2017)
demonstrate how “high stakes assessment climates complicate PD efforts to shift teachers’ practices toward in-depth inquiry and analysis” while also steering “PD toward covering curriculum and measuring and management of student achievement
outcomes instead of strengthening in-depth subject matter learning and instruction” (p. 251). Balanced leadership should support teachers in achieving mandates
while also building their professional agency.

References:

Addison, J. (2015). Shifting the locus of control: Why
the common core state standards and emerging standardized tests may reshape college writing classrooms. The Journal of Writing Assessment8(1).
Retrieved from: http://journalofwritingassessment.org/article.php?article=82

Adler-Kassner, L. (2012). The companies we keep or the companies we would like to try to keep: Strategies and tactics in challenging times. Writing Program Administrator, 36(1), 119-140.

Arnold, N. (2019). Supervisory identity: Cultural shift, critical pedagogy, and the
crisis of supervision. In Zepeda, S.J. & Ponticell J.A. (Eds.), The Wiley handbook of educational supervision (575-600). (Wiley handbooks in education). Hoboken, NJ: John Wiley & Sons. Retrieved from https://onlinelibrary-wiley-com.ezproxy.spu.edu/doi/pdf/10.1002/9781119128304

Fink, G.M., & Muntz, G. (2012). Federal higher education reporting databases and tools. In Howard, R.D., McLaughlin, G.W., & Knight, E.D. (Eds.), The handbook of institutional research (354-370). (The Jossey-Bass higher and adult education series). San Francisco, CA: Jossey-Bass.

Ingersoll, R.M., & Collins, G.J. (2019). Accountability, control, and teachers’ work in
American schools. In Zepeda, S.J. & Ponticell J.A. (Eds.), The Wiley handbook of educational supervision (159-182). (Wiley handbooks in education). Hoboken, NJ: John Wiley & Sons. Retrieved from https://onlinelibrary-wiley-com.ezproxy.spu.edu/doi/pdf/10.1002/9781119128304

Jensen, B., Sonnemann, J., Roberts-Hull, K., & Hunter, A. (2016). Beyond PD:
Teacher professional learning in high-performing systems. Washington, D.C.:
National Center on Education and the Economy. Retrieved from http://www.necc.org/wp-content/uploads/2015/08/BeyondPDWeb.pdf

Koehler, M.J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary issues in technology and teacher education, 9(1), pp. 60-70. Retrieved from: http://www.citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge/

Lunenburg, F.C. (2019). National policy/standards: Changes in instructional supervision since the implementation of recent federal legislation. In Zepeda, S.J. & Ponticell
J.A. (Eds.), The Wiley handbook of educational supervision (381-406)(Wiley handbooks in education). Hoboken, NJ: John Wiley & Sons. Retrieved from https://onlinelibrary-wiley-com.ezproxy.spu.edu/doi/pdf/10.1002/9781119128304

McMunn Dooley, C., Owens, S.J., & Conley, M. (2019). Teacher performance
assessments mandated during the Duncan era. In Zepeda, S.J. & Ponticell
J.A. (Eds.), The Wiley handbook of educational supervision (407-432.).
(Wiley handbooks in education). Hoboken, NJ: John Wiley & Sons. Retrieved from https://onlinelibrary-wiley-com.ezproxy.spu.edu/doi/pdf/10.1002/9781119128304

Meuwissen, K.W. (2017). “Happy professional development at an unhappy time”: Learning to teach for historical thinking in a high-pressure accountability context. Theory & research in social education, 45(2), 248-285. doi:10.1080/00933104.2016.1232208

Rice, G.A., & Russell, A.B. (2012). Refocusing student success: Toward a comprehensive
model.In Howard, R.D., McLaughlin, G.W., & Knight, E.D. (Eds.), The
handbook of institutional research
 (237-255). (The Jossey-Bass higher and adult education series). San Francisco, CA: Jossey-Bass.

Leave a Reply

Your email address will not be published. Required fields are marked *