A Closer Look at State Assessments

I recently had the occasion to look at Washington’s achievement data on two key performance indicators: 4th grade reading, and 8th grade math.

I chose these data points because they represent key transition points in each student’s educational career.  Research tells us students who are not reading at grade level by the end of 3rd grade have difficulty catching up thereafter.  Indeed, 3rd grade reading proficiency is a strong statistical predictor of high school graduation.   Similarly, middle school math represents a key transition point– students not achieving math proficiency in middle school are less likely to be college and or career-ready by the end of high school, and qualify for the high-skill, living wage jobs (many of them STEM-related) Washington is having difficulty filling.

Which Story Do We Believe?  Juxtaposing State vs. National Data on Math and Reading

In my review, one finding sticks out.  On state assessments, the Washington Report Card has more students achieving proficiency reading in 4th grade (67.3%) than math in 8th grade (50.4%).  However, when comparing scores on the National Assessment of Educational Progress (NAEP), the inverse is true – Washington actually has a greater percentage of students proficient in 8th grade math than 4th grade reading: 40% compared to 34%.  WA also outpaces the nation by a wider margin in 8th grade math as well (see chart below).

So, the state and national data seem to tell conflicting stories, both in terms of the relative ranking of 8th grade math versus 4th grade reading, and in terms of how many kids are actually achieving ‘proficiency’.  And the disparity is not trivial.  For example, national NAEP Reading data suggests about half as many kids are achieving “proficiency” (MSP: 67.3% vs. NAEP: 34%).

So what is going on here?  What is our biggest ‘transition point’ problem – middle school math or early reading?  How do the state and national definitions of ‘proficiency’ differ, and which is to be trusted?

As it turns out, there are a few thing going on.  And Washington is not unique among states in this situation.

For one, Washington’s definition of ‘proficient’ is less rigorous than that utilized by NAEP, although our relative standing among the states is much different in 8th grade math than 4th grade reading.

Figure 3 below shows a ranking of states produced by the National Center for Educational Statistics (NCES), comparing WA’s standards to equivalent scale scores on NAEP.  I have put stars next to Washington in each case, and you will see WA ranks 16th in Reading, with a proficiency standard that is below NAEP ‘basic’, and far below NAEP ‘proficient’.

But in 8th grade math, it’s a much different story (See Figure 3).   Washington’s 8th grade math standard is considerably more rigorous than NAEP ‘basic’ and comes closer to NAEP ‘proficient’.   It’s also noteworthy that we ranked 2nd in the nation on this comparison.

From this, It would appear that our 8th grade math standards are simply more rigorous than our 4th grade reading standards.  This would seem to explain why our relative math/reading proficiency rates flip-flop in the state-to national comparisons.  So maybe our performance in Math, in objective terms, isn’t really all that far behind our performance in Reading.  Indeed, maybe it isn’t behind at all.

CHART 1 – WASHINGTON COMPARED TO THE NATION ON 4TH GRADE READING AND 8TH GRADE MATH – NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS (NAEP) 

Additionally, aside from WA’s ranking, it is striking how widely varying the definitions of ‘proficiency’ are state-to-state.   A student could be deemed ‘proficient’ in Reading if they live in Portland, Oregon, but considerably below proficiency if he or she lived just across the border in Vancouver, WA (note that Oregon ranks next-to-last among the states on the Reading standard).   That is, in essence, the strongest argument there is for Common Core.

CHART 2 – WASHINGTON COMPARED TO OTHER STATES – STATE STANDARDS EQUIVALENCE TO NAEP SCALE SCORE – 2 GRAPHS: 8TH GRADE MATH AND 4TH GRADE READING 2009 (SOURCE, NCES)

I’m left with a few lasting impressions.  First, I have a renewed appreciation of the need for Common Core – such widely divergent definitions of ‘proficiency’ across the states present real equity issues for students.

CHART 3 – NAEP SCALE EQUIVALENTS OF STATE GRADE 8 MATHEMATICS STANDARDS FOR PROFICIENT PERFORMANCE, BY STATE: 2009

But I also have a real curiosity as to whether our first test results from the Common Core assessments  reveal the same relative dearth of proficiency in 8th math vs. 4th reading.  The low math scores, relative to reading, have garnered a lot of attention, and driven a lot of policy decisions in this state over the past decade.  But could it be that the new test results, scheduled for 2014-15, turn the conventional wisdom about math performance in this state on its head?

 

This entry was posted in Assessments. Bookmark the permalink.

2 Responses to A Closer Look at State Assessments

  1. David says:

    It should be obvious that we all want to see growth in the number of Washington students who are proficient on these performance measures. What’s frequently missed is that proficiency is not enough. Proficiency should be the minimum level we expect our students to achieve, not a ceiling on their performance. Our assessments need to have enough headroom to measure those who are capable of exceeding standards, and our schools need to deliver an appropriate education that allows those students to continue to grow.

  2. sbewordpressblog says:

    We agree that proficiency should not be the only measure. We will be adding a new indicator in 2013, student growth, to be included in the revised Achievement Index. We hope to have this Index available for public consumption by the close of 2013. More information about our work with the Index can be found at: sbe.wa.gov/aaw.php

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s