We use Scholastic Reading Counts in our district. Kids read books, from Dr. Seuss through the Odyssey, and then they take a quiz on the computers to see how well they understood the book. The questions are low level thinking/recall questions, but are pretty tricky. They earn points that can be tracked by the teacher or librarian. This chapter reinforced for me how important it is not to overemphasize the extrinsic rewards that some teachers and librarians have attached to this program. What I see in students is a sense of pride that they read the book and understood it well enough to pass the test. We have a super ethnically and socioeconomically diverse district, so for many kids they just get a kick out of having one more reason to use the computers.

Students like my kid had a requirement to earn a set number of points in the school year (9th grader) this forced him to knuckle down and get some "free" reading done, but what I saw was that because he started the habit, he grew to like it much more. So for him, he discovered the intrinsic rewards because of the extrinsic. The program could backfire for a student who already loves to read.

One cool idea for this kind of program is for a teacher to set a class goal to read a million words. The program will keep track of this and so everyone in the class can feel like they are contributing, whether they read 10 books or 2, chapter books or picture books.

Sure, feel free to use this example. I'm so glad I escaped that district when I did. Large districts probably feel the need to standardize practices for all their schools. Unfortunately this can sometimes be done in ways that are a detriment to real progress.

Ahhh... gotcha. Yep, the district assessments sound pretty poor. =(

Thanks for the additional info. I might use this (anonymized, of course) as an example in my DDDM workshop for Arkansas teachers/principals in September!

Question 1) Are the tests valid? (Influencer Strategy - Personal Motivation - Is it worth doing?)

The district tests measure student mastery of a standard based on 1-3 questions. Every six weeks a different test, based on the recent readings is given. Students can be proficient in one assessment period and basic in the next. It seems questionable to me whether this small number of questions gives an accurate measure of how well a student has mastered a standard.

These second graders are asked to do word associations for multiple meaning words such as reservation, the meaning of reservation the test seeks is a place where Indian's live; opposites for words such as courageous. Students in the school are English Language Learners and At Risk students some of whom are not yet reading at grade level. It seems that students need to be reading at a higher level for these assessments to be useful.

Question 2 - Is the data useful to teachers to guide instruction? (Influencer Strategy: Structural ability - Is it easy? does it make sense? and Personal Ability, do the teacher really understand how to analyze the data in a useful way?)

Teachers receive a static document with the students names on one side of the spreadsheet and student performance on each standard on the other axis.
This static document seems limited to me in that teachers cannot easily manipulate the data to group students and turn the information into knowledge. In my district, our assessments were created by teachers and are scanned in to a web based computer program that allows teachers to easily see the groups of students who did not master a given standard. They can even drill down to see the question missed to see if there is a problem with the question.

Question 3 - If teachers know their students best, shouldn't they use assessments they find helpful to get them to the next level? (Influencer Strategy - Personal Ability, Social motivation)

Examples of the assessments my friend finds useful are the San Diego Quick, in which students read a list of words and based on their performance, a teacher can quickly gauge the student's reading level.

She also gives reading fluency tests to each child and asks her own comprehension questions to judge whether or not the child is comprehending what they read.

If the district does believe the tests are valid, they need to allow more training for teachers in how to use the data. (My friend noted that even the vice principal couldn't clearly explain to them how to use the data.)

My understanding of data analysis efforts that have improved student achievement is as follows:
1) Teachers take ownership of establishing student learning goals
2) Teachers design or find assessment tools that measure student progress
3) Teachers collaborate, sharing skills and strategies that help students improve

I guess the biggest problems with these district assessments are the lack of teacher buy in, and lack of teacher understanding about how to use the results. These two combined with the lack of support for teachers to try other strategies as a team.

Scott,

I have a question about data use practices. I took a walk with a former colleague recently and we were discussing the use of data in her school and school district, my former district. The school district has students take theme tests from the Open Court series. The students are judged to be proficient in a standard based on as few as 1-3 questions that "measure" the standard. She and her grade level team members are expected to sit and plan instruction based on the test data which are provided to them as a spreadsheet that includes the list of students and standards. She sees these assessments as useless and yet she and her colleagues are required to use them and discuss them at their meetings.

She has been teaching for at least 14 years, has a master's degree, did a Beginning Teacher support project in which she assembled assessments for first graders. She worked with me when we used reading assessments some of which were actually quite helpful. Yet, she is required to use useless assessments from the school district. She uses her own assessments so that she'll have information on her own students, but this is not used by the grade level team since they are required to use the other assessment data.

Why don't we use the expertise available among our teachers? It's no wonder some become bitter and resentful and distrusting of "improvement" programs.

The title of this post is from page 222 of the book. I don't think we pay enough attention to the physical world of schools...

1. I’ve done a great deal of work with schools, helping them to ‘uncover the invisible’ through effective and creative uses of data. I appreciated the authors' emphasis on changing people's cognitive maps using data. We don't do this enough in schools. We're still so focused on data for NCLB compliance that we miss a lot of opportunities to use data analysis for more meaningful school improvement. Innovative data collection and presentation can go a long way…

Example story: I worked with a high school assistant principal who was concerned about one teacher, Teacher C, who sent students to the office nearly every class period. The school had smaller learning communities (like middle school teams); a group of teachers shared the same pool of students. She and I made a chart that displayed the discipline referrals for every teacher on the team (remember, they all had the same students) for the past three months. The height of Teacher C’s bar was four times that of any other teacher on the team. No names on the chart except for Teacher C. The assistant principal slipped the chart into the teacher’s box with a note explaining the chart and offering to help. Surprise! The number of disciplinary referrals dropped dramatically and immediately! This is the kind of creative use of data we need to do more of…

2. The physical environment of schools gets in the way of meaningful collaboration and interaction. The teacher’s lounge is often a poisonous place but may be the only place where teachers bump elbows. What can schools do to increase the likelihood of positive, productive elbow-bumping?

3. I liked the emphasis on MAKING THE CORRECT BEHAVIOR EASIER (and, eventually, UNAVOIDABLE). Our paperwork and other bureacratic structures / regulations / procedures usually make the correct behavior more difficult, not easier. I think this is an area that more principals and superintendents could (should) pay attention to…

Changing Environments
As I read chapter 9 several questions kept recurring.
Is there anything about the physical environment of our school that might be changed in order to better allow some of the changes we have discussed? In thinking about the answer to this I have gone and ahead and decided to treat the school schedule as a part of the physical environment since it determines how teachers and students are placed throughout the day. I work in an elementary school where I see students for technology class in 30 minute blocks of time, once per week. Teachers have planning while I teach, thus we cannot plan together unless it is outside of the school day which many teachers (more and more each year) are less willing to do. This relatively short amount of teaching time restricts the types of projects I can do with the students on their computers. Ideally I would have longer blocks of time that would allow for both instruction and then application/creation on the student’s part. I have tried to shift some of the instruction to outside the school day by using online avenues (Blackboard). Unfortunately for the age students I work with school is still a physical building that closes at 4 pm. The only traffic I get from students hitting my course sites are those that are truly enamored by technology. Ideally I would be able to meld what I do to support the curriculum of the classroom teachers but they feel some of the some physical restrictions in that they must share technology resources (computers).

Affordable 1-1 technology will be a huge boon to the way schools do business with their students. Unfortunately even when the technology does become affordable our schools will not be ready for the sweeping change. I wonder if classroom teachers feel the same limitation of the schedule and the way we chop up the subjects into predetermined times. Ie All math learning will occur between 10-11 am. All science learning will occur between 11-11:45 am. Although there are so many tie-ins between the two subjects (as there are between reading and social studies) we continue to physical separate the subjects. Do teachers see this or is it something we have been doing for so long that we fail to notice it?

The authors continue to discuss the importance of reinforcing vital behaviors in chapter 9. While I’m not sure our school has agree to what the most vital behaviors are (we really should) these behaviors need to be reinforced in order to change the existing concept map of our teachers. My goal is to get teachers to try new, more engaging ways of teaching and to give children more opportunities to CREATE in the classroom as opposed to REGURGITATE. If our school/district leadership decides this is a vital behavior we must find ways to reinforce it. One such way suggested by the authors to change the concept map (try something new or different) is to surround teachers with a focused data stream that reminds them of the importance of these behaviors. With the proximity (or lack of) teachers have from each other (close the door and teach) how do we create such a stream and what information becomes part of that stream. I enjoyed the HP story but don’t see a school stopping business to do that. After all, we have 500 students to supervise. Contractually we can only require teachers to be together a certain number of minutes beyond the school day.
Interpersonal relationships are important to collaborative efforts, but in our building over the past few years less and less teachers even make their way to the staff room to eat lunch and visit. This subtle change over the years has lessened the cohesiveness of our staff.
I’ll be curious to read other takes on chapter 9.