Dr. Gary Thompson is the Utah child psychologist that has been on Glenn Beck’s show and written about the dangers of the behavioral testing under the new Utah assessment system as part of Common Core. He recently met with State Education Superintendent Menlove, and then sent him this letter (and sent me a copy with permission to post as well). In addition to Dr. Thompson’s letter, is a letter from Attorney Edward Flint, a special education attorney.
Early Life Child Psychology & Education Center, Inc.
Dr. Gary T. Thompson
Director of Clinical Training & Community Advocacy
Edward D. Flint, Esq.
Special Education Attorney at Law
www.earlylifepsych.com
Adaptive Testing & Utah Common Core
Introduction
The intricacies surrounding the psychometric qualities involved in adaptive testing, specifically the adaptive tests currently being designed by AIR (American Institutes for Research) via grants from the Federal Government and a $32,000,000 contract from Utah, are complex indeed. This is a layman’s memo regarding minor, but important elements of CAT (Computerized Adaptive Testing) that I feel the community should have a basic understanding of, as well as specific populations of school aged children who may be adversely affected by the implementation of Common Core if significant “adjustments” are not made prior to implementation.
Basics of CAT (Computer Adaptive Testing)
Simply put, all CAT is nothing but the following: The measurement of predetermined variables that result in a data set that can be interpreted. That basic definition applies to any academic, cognitive, or psychological test. More simply put, CAT measures “something.” In terms of our involvement as parents, all we wish to know is what exactly are the tests being designed by AIR going to measure. USOE states, in simplicity, that the AIR CAT tests are only going to measure academic related subjects, thus providing educators with data which shows the current progress of a individual student in those subjects.
I love adaptive testing. While in graduate school, the study of the future implementations of adaptive testing in the fields of education and psychology were in its infancy in terms of having a finished “product” ready for dissemination for public and private agencies. Professors estimated that within 5 years, that time would come. It is now 5 years later, and yes indeed, the future of adaptive testing has arrived. For certain populations of school aged children, this is indeed a great time in history, whereas these tests will assist them in their academic quests.
Capabilities of Adaptive Testing:
In a nutshell, as preached by representatives of USOE, adaptive testing is a remarkable tool designed to meet the academic measurement needs of a certain subset of Utah’s public schools students. It is fast, and they are for the most part, highly statistically accurate. With the exceptions of a student population that will be discussed later, I have little issues as local clinical community scientist with the push for implementing the latest and greatest technologies to assist with the accurate measurement of student progress in academics. Mead and Drasgarb (1993) stated, “research on cognitive ability tests shows that adaptive tests, and paper-and-pencil tests lead to equivalent scores. I believe the pilot testing results from AIR will show similar statistical accuracies regarding academic related tests with certain populations of school aged children.
However, along with the advancements of CAT in education, the psychology side of CAT has advanced even quicker along the lines of measuring psychological traits of people. Hol (2008), stated, “In conclusion, this study indicates that there is potential for the development of adaptive test for personality assessment.” Having a doctorate in clinical psychology with an emphasis in Forensics, this area of adaptive testing has always intrigued me, yet at the same time has brought a certain level of professional trepidation. Test results of properly administered valid tests of any kind can assist people greatly. Psychological test results in the hands of lay education officials will most likely result in errors, confusion, and possibly tragedy.
Conspiracy Theory or Valid Concern?
I’m not a psychometrician by any means. I was taught by several of them in graduate school, some of them who had prior experiences working for researchers along the likes of AIR. While learning the basis of this art in graduate school, we were often treated to stories of developments and research from within this rarified air of psychometric laboratories. We also learned the basics of test design. Suffice to say, I have zero doubt that the group of psychometricians and researchers both at my graduate school, and AIR have the potential and training to devise a test that measures any variables that they wish to measure. Nevertheless, I also have zero doubt that these same skilled researchers are capable of designing “test within tests.”
Anyone who states that AIR does not have the capacity to input selected variables that measure “behavioral characteristics”, along with variables that measure language arts, science or math is sorely misguided. It would be relatively “easy” to design a language adaptive test that has behavioral characteristics embedded into the design of the test. Formulas could be designed to produce two sets of results (language and behavior), and then forward the language test results to its intended target (Utah Schools), and the behavioral results to another intended target (Federal Government, Private Agencies). I will put my reputation within the community on the line by stating that the psychometric capacities of doctoral level psychometricians can devise these “dual” test designs.
When you add into this equation that AIR stated mission is to “to conduct and apply the best behavioral and social science research and evaluation towards improving peoples’ lives, with a special emphasis on the disadvantaged,” any reasonable minded person, as well as a State Superintendent of Public Schools, should at least reasonably conclude that this billion dollar research corporation (AIR) with some of the brightest minds on the planet can design tests any way that they please, unless per contractual agreement and other applied constraints, they are expressly forbidden to do so.
Utah’s parents have been told in multiple town hall meeting by USOE that they will never be able to have access to testing questions devised by AIR in order to ensure “test integrity.” Although I am impressed with USOE and various politicians who stated that 15 parents and a few politicians will be allowed to view the tests being designed by AIR, I question their qualifications to perform anything more than a cursory review of the questions being designed. Speaker of the House Becky Lockhart may be able to balance a complex state budget, but I doubt that she has the necessary background in psychometrics to perform a critical analysis of the issues that need to be examined.
The average person that Ms. Lockhart and other education officials that have the honor of selecting the lucky 15 parents more than likely will not have the expertise to perform a critical analysis of the adaptive test designs being constructed by AIR on behalf of Utah’s students. In fact, I would have to ethically decline such an invitation, since despite my doctorate in clinical psychology, I am not qualified to examine complex psychometrics that will be in place with the adaptive testing. 99.7% of you reading this letter in the State of Utah are not qualified either.
So the question remains of, “who is watching the Hen House?” This very basic question was not answerable during my meeting with the office of the Superintendent.
All of this leads to a very reasonable conclusion in this area of discussion. Someone, independent of AIR, MUST have access to every single item on the tests being designed in order to insure and that absolutely ZERO behavioral indicators are being measured on tests that parents in Utah believe are only measuring “reading, writing and arithmetic”. Undue paranoia? Nope, just plain old American accountability to the parents of Utah’s children who not only deserve this, but have paid $32,000,000.00 of their tax dollars to have AIR design the test. More importantly, the population of children discussed below, based upon current peer reviewed research, may be seriously adversely affected for the remainder of their academic careers if time is not taken to insure the viability of adaptive testing on their respective groups.
Children With Disabilities & Other Culture Related Issues:
AIR has provided absolutely zero evidence that the specific adaptive testing measures being utilized will be valid for the following populations whom, statistically and historically, have not performed well on standardized, or adaptive assessments:
- Children with ADHD
- African American Children
- Latino Children
- Asperger’s Children
- Autistic Children
- Children with Specific Learning Disorders
- Gifted Children who possess any of the above characteristics ( Commonly known as“2 L’s, or “Duel Exceptional Children)
This list is not exclusive. Nevertheless, the numbers of children in our state who can be categorized on the above list is staggeringly high. Just recently, it was reported by researchers at the University of Utah that Utah has one of the highest numbers of children with Autism in the entire country. That is 1 in every 5 children. Do the math.
Suffice to say, this is common knowledge in the fields of psychological testing and assessment, and such core basic understanding of this is taught in every accredited graduate school in the country. Implying, as was done in the USOE Alpine Town Hall Meeting, that any disability group in the country has approved a test (that has not yet to be designed) for the valid use with these populations, is disingenuous at the very least, or a flat out, deliberate misrepresentation to the parents of Utah (and the rest of the country) at worst.
Parents of these children need to see and review validity data PRIOR to subjecting their children to testing environments that may exasperate existing psychological conditions. In the case of Latino and African American students, subjecting them to tests that have been properly validated for that culture may not only subject them to increased frustrations and decreased self esteem, but may ultimately not be the best type of tool to utilize to measure academic achievement. Below is a brief snippet from my doctoral project/dissertation regarding African American School Aged Children:
“Do to time and emphasis spent on controlling alleged behavioral problems amongst African American children, many students of color are simply not taught the knowledge and skills necessary to ensure academic success (Webb-Johnson, 2002). Award winning African American researcher, Patton (2001) refers to such phenomenon as “ABT” (Ain’t Been Taught). This skill deficit is present because teachers not possess the cultural skill sets necessary to implement proper motivational practices, while simultaneously demanding high academic and behavioral standards (Webb-Johnson, 2002).”
More…….
“It has been documented that many African American students challenge the white middle class status quo of classroom decorum in a effort to express their unique cultural individuality to avoid tedious academic presentation (Webb-Johnson, 2002). This is often expressed through game playing, and alleged deviant behaviors (Webb-Johnson, 2002). Development of culturally sound contextual or assessment model that does not view these problems as residing in the child is essential because teacher’s perceptions on students have long lasting effects on school achievement (Meisels, Steele, & Quinn-Leering, 1993). “
Finally…..
“It has been shown that generally, Caucasian students are more object oriented and have developed a learning style that centers on performing task oriented work independently with individual achievement and competition as a conceptual framework (Hosp, 2001). “
Other assessment conceptual frameworks can be found in my (and other peer reviewed research across the spectrum) doctoral project that applies to Caucasion, ADHD, gifted, learning disordered, Autistic children, Latino and Duel Exceptional children of all races, that need to be considered when devising high stakes testing instruments.
None of this has been considered or articulated to the parents in Utah in writing or via the USOE Town Hall meetings that I have attended in person.
In other words, the voices of approximately 200,000 children in the State of Utah have not been heard. Dr. Menlove, you as well as the elected members of the Utah State Board of Education, are the voice of these children, not the U.S. Department of Education, AIR, Utah politicians, or Bill & Melinda Gates. You have an ethical, moral and professional obligation to examine the possible academic, emotional and psychological affects of a radical change in the way public school knowledge is taught and disseminated to these vulnerable, and oft misunderstood populations of local Utah children.
Conclusion:
Dr. Martell Menlove, State Superintendent of Schools, asked me what it would take to get my daughter Zoey enrolled in a Utah Public School Kindergarten. Here is what it will take, in terms of the assessment/testing aspects associated with Common Core:
- A truly independent review by three independent, Board Certified, joint Ph.D. level psychometricians and licensed clinical psychologist, of all of the test items developed by AIR to ensure that there are no line item variables that could be reasonable utilized to measure “behavioral characteristics” as such may be defined by the American Psychological Association, or Journals published by this group. (This list is pretty long).
- A “opt out option” for children with disabilities until data of validity and efficacy is published and disseminated to the public, which ensures fair and accurate measurement of academic achievement.
- An joint endorsement from the clinical AND educational psychology department heads of the University of Utah, Utah State University, Brigham Young University, Westminster College, and the clinical director of a private independent, APA approved school of ps