(
White Papers
Next-Generation Assessments (438 KB, PDF)
by Malbert Smith, III, Ph.D., President, MetaMetrics, May 2009
Today, it is "in vogue" to write, talk and think about the measurement of 21st century skills. Generally, these discussions focus on what should be measured (e.g., critical thinking, cultural awareness, digital literacy), but not necessarily how these constructs should be measured.
More than 30 years ago, legendary assessment guru Oscar K. Buros reflected on the last 50 years of testing (Buros, 1977). His concern about the lack of progress made in the testing field was punctuated in the following: "If you would examine these books and the best of the achievement and intelligence tests then available, you might be surprised that so little progress has been made in the past fifty years-in fact, in some areas we are not doing as well. Except for the tremendous advances in electronic scoring, analysis, and reporting of test results, we don't have a great deal to show for fifty years of work. Essentially, achievement tests are being constructed today in the same way they were fifty years ago-the major changes being the use of more sophisticated statistical procedures for doing what we did then-mistakes and all" [p. 10].
Stop Summer Academic Loss: An Education Policy Priority (1.03 MB, PDF)
by Malbert Smith, III, Ph.D. and Dee Brewer, M.A., M.Ed., MetaMetrics, March 2007
Every year, U.S. students go to school for an average of 180 days. During that time, most progress along a learning trajectory and grow in terms of knowledge and skills. However, when summer break comes along, the formal learning process often ends, and many students, particularly those from low-income families, begin to show learning losses. In fact, research shows that all students experience learning loss when they do not engage in educational activities during the summer.
...Scientific research over decades has confirmed that, without intervention, children who start school behind likely will stay behind and that children who cannot read at grade level by the fourth grade will likely face an ongoing struggle to learn...
Aligning the Journey With a Destination (400 KB, PDF)
by Gary L. Williamson, Ph.D., October 2006
A great deal has been written in the last few decades about the condition of public education in the United States. Many authors have argued that students are unprepared after high school for the variety of experiences they seek in life, not only those experiences related to higher education but also those related to the military, the workplace and the day-today responsibilities of citizenship.
...Two threads of recent research provide one possible way of developing and aligning (or at least informing the discussion about) student achievement standards for K-16...
Managing Multiple Measures (120 KB, PDF)
by Gary L. Williamson, Ph.D., September 2006
When the No Child Left Behind (NCLB) Act of 2001 was signed into law in January 2002, it established sweeping requirements related to annual achievement testing for state accountability purposes. Since that time, states have re-evaluated their testing programs, and in some cases expanded them, to ensure they meet the requirements of the law. However, states and school districts use achievement tests for purposes other than accountability (e.g., for instructional or programmatic monitoring) and often supplement the annual accountability assessments required by NCLB with interim tests during the school year to gauge whether they are on track to meet the annual achievement targets required by the law. Consequently, students are more likely than ever before to be assessed multiple times during a school year.
...With increasing variety and frequency of assessments, combined with the inclination to employ a common metric for all assessments of a given construct, it is increasingly routine to have multiple, comparable assessment measures for reading or mathematics available for each student...
What is Expected Growth? (204 KB, PDF)
by Gary L. Williamson, Ph.D., July 2006
We are all familiar with children, either through knowing our own or through acquaintance with those of other people. Perhaps no other thing in life is as obvious as the dramatic way that human beings develop and grow. Our key social and political institutions devote a significant part of their resources to ensuring that children grow and learn to function as productive citizens. Growth and learning are central to the mission of our country's public schools...
Because there are a number of alternative ways to conceptualize student growth and to measure it, states face a challenge to design and implement accountability systems that address a variety of information needs and still comply with state and federal laws. In this context, there are naturally many viewpoints about how best to conceptualize and measure student growth and to set appropriate goals for growth. This makes it especially important for students, parents and educators to better understand student growth, how it is measured, and how growth expectations may be set in different contexts for different purposes...
Student Readiness for Postsecondary Options (506 KB, PDF)
by Gary L. Williamson, Ph.D., July 2004
Students who leave high school successfully may nevertheless be unprepared for the array of possibilities that face them in the postsecondary world. Whether the goal is further education, a job, or enlistment in the military, there are those who say that high school graduates are unprepared because of a lack of basic skills attained in the public schools. Some even question whether young adults have developed enough basic literacy skills to function effectively as citizens...
The Need for Objective Measurement Under the No Child Left Behind Act (552 KB, PDF)
by Malbert Smith III, Ph.D., President, MetaMetrics, July 2004
With the passage of the No Child Left Behind Act, Congress reauthorized the Elementary and Secondary Education Act (ESEA) - the principal federal law affecting education from kindergarten through high school. In amending ESEA (commonly referred to as No Child Left Behind, or NCLB), the new law represents a sweeping overhaul of federal efforts to support elementary and secondary education in the United States...
One of the major weaknesses of reading education today is the lack of meaningful measurement systems. The key in the "hard" sciences is unification of measurement. In the case of the measurement of temperature in the 1600s, there were literally dozens of instrument makers with their own scale. However, once a theory of temperature had been developed and accepted, measurement unification was possible. Today, it is inconsequential whether a temperature is taken with a thermometer purchased at CVS or K-Mart - the scale is independent of the manufacturer of the instrument...
Why do Scores Change? (453 KB, PDF)
by Gary L. Williamson, Ph.D., April 2004
Since 1990, educational accountability systems have been widely implemented in the United States. The focus on accountability recently gained new emphasis with the reauthorization of the Elementary and Secondary Education Act (ESEA) signed into law by President Bush on January 8, 2002. The law, usually called the No Child Left Behind Act of 2001 (NCLB), put in place sweeping requirements for increased accountability in the public schools of the United States. A central feature of the new law is the requirement for annual assessments of students in reading and mathematics.
Because of the new federal requirements as well as state testing programs that were already in place in many states, the academic performance of students in the United States is perhaps more widely measured now than at any previous time in history. With more frequent measurement, parents and teachers have access to more information about their students' performance than at any previous time. With the increased availability of information, parents and teachers are better informed than in the past. Ironically, they may also find themselves having more questions about the results than at any time in the past...
The Lexile® Framework as an Approach for Reading Measurement and Success (580 KB, PDF)
by Colleen Lennon and Hal Burdick, MetaMetrics, April 2004
The Lexile Framework for Reading is an innovative approach to reading comprehension that can be implemented by educators, parents and readers of all ages. Lexile measures, as components of the Lexile scale, are the result of more than 20 years of ongoing research based on two well-established predictors of how difficult a text is to comprehend. By measuring both text difficulty and reader ability on the same scale, readers can be appropriately matched with books that will be both engaging and challenging.
Lexile measures are the most widely adopted reading measures in use today. Tens of thousands of books and tens of millions of newspaper and magazine articles have Lexile measures - more than 450 publishers Lexile their titles. In addition, all major standardized reading tests and many popular instructional reading programs can report student reading scores in Lexile measures. Implementation of the Lexile Framework has led to reading success and improved reading enjoyment at all levels of proficiency...
An Investigation of Dimensionality Across Grade Levels and Effects on Vertical Linking for Elementary Grade Mathematics Achievement Tests (904KB, PDF)
by Samantha S. Burg, Ph.D., Research Associate, MetaMetrics, April 2004
The psychometric models used in the context of many achievement tests assume a unidimensional construct is being measured. That is, in the context of measuring student achievement, most tests are considered to measure one latent trait, construct or ability (i.e., unidimensional). Other tests are designed to measure a combination of abilities (in which it is referred to as multidimensional). In either context, the dimensional structure of a test is intricately tied into the purpose and definition of the construct to be measured. However, it is sometimes the case that a test that is intended to be unidimensional may unintentionally be measuring more than one latent variable.