All Assessments Are NOT Created Equal

Guest post by Lyndsey Zurawski, SLP.D, CCC-SLP

Eval after eval, after eval. For most Speech-Language Pathologists (SLPs), completing an evaluation is a smaller part of the job. According to the 2020 ASHA Schools Survey, SLPs across all facility settings reported an average of 4 hours per week on diagnostic activities. What are diagnostic activities? This could include formal evaluations, informal evaluations, scoring evaluations, analysis of evaluation, screening, observations, and report writing. Does this sound like more or less than the average that you spend on diagnostic activities? For SLPs with high caseloads and limited amount of workload time allotted in their schedules, completing an evaluation can seem tedious. When required to conduct an evaluation, whether as an initial or a reevaluation, sometimes convenience and/or speed can override best practices. 

We’ve all been there… An evaluation needs to get done, we grab the quickest evaluation tool on our shelf and administer it. We score it, write up the report, and along with observations, parent input, and classroom/intervention data we go to eligibility or ineligibility. But, what about best practice? Administering that one quick test is NOT best practice. Even still, we have all been there, and when I say “we”, I mean me! 

What are best practices? 

First let’s discuss what should be included in a comprehensive evaluation. 

• Case History/File Review
• Hearing and Vision Screenings
• Observations in more than 1 setting
• Assessments in all areas of suspected disability 
• Culturally and linguistically sensitive assessments
• Recommendations and Summary

Jump to assessments in all areas of suspected disability. Do you have a favorite assessment tool? I don’t just have a favorite tool, I have a favorite battery of assessments I use. Now, to clarify, part of my job is as a diagnostician conducting in-depth language and literacy evaluations. So, to be fair, my assessment battery has to be more comprehensive than what I would typically utilize for my caseload evaluations and reevaluations. However, that should not stop me (or you) from utilizing best practices when we conduct an evaluation. 

It is important for us, as SLPs, to consider diagnostic accuracy which includes sensitivity and specificity. Now, I’m not going to get all super nerdy and discuss the ins and outs of psychometric properties, insead I’ll refer you over to a page on the Informed SLP that is easy to read and digest. The important thing to know about sensitivity and specificity, simply speaking, is that while no test will be at 100%, a test is judged to be “good” if it is 90% to 100% accurate; “fair” if it is accurate 80 to 89 percent of the time, while anything less than 80% is considered unacceptable (Plante & Vance, 1994) Do you know what tests fit this bill? I don’t off the top of my head. Would you like a resource that does tell you this information at the tip of your fingertips? Our friends over at the Virginia Department of Education created an SLP Test Comparison Card that includes this information. In 2018, they added a few additional tests as a supplemental card. One of the most common assessment tools used, the Oral and Written Language Scales-II (OWLS-II) does not report sensitivity and specificity. Information for other common assessments such as the Test of Integrated Language and Literacy Skills (TILLS), Comprehensive Assessment of Spoken Language- 2nd Edition (CASL-2), and the Clinical Evaluation of Language Fundamentals-5th Edition (CELF-5) are included from the Virgina DOE. 

Another factor is cultural and linguistic sensitivity/bias in our assessments. One way we can ensure that we don’t under or over identify students is to utilize dynamic assessment, especially with students from diverse backgrounds. Ireland (2019) reported that with dynamic assessment sensitivity and specificity has been documented up to 100%. 

---

As you’re reading, I’m sure you’re thinking— hurry up and get to the part where you share which tests to use and not use. While it is not as simple as that, I am going to share what I use and why. This is based on my own clinical experience, along with information related to the factors listed above. It is best practice to administer more than one assessment when conducting an evaluation. All assessments are not created equal. As diagnosticians, which each and every one of us is, it is our job to pick and choose the assessment tools that can reliably and validly identify a speech, language, literacy, and/or social communication disorder. How do we do that? There’s not one simple answer, but it comes down to using best practices including evidence-based practice, clinical judgment and experience, as well as considering cultural and linguistic diversity and sensitivity in all of our evaluations. 

Most frequently I am utilizing the TILLS (for students 6 thru 18) because it is a comprehensive language and literacy assessment, has solid sensitivity and specificity, and provides detailed information on interpreting the results and includes relation/impact to curriculum-based content. As an added bonus, there’s a report writing template that is available to use for your own reports. While it is lengthy to administer, I could essentially only use this test and feel comfortable with my results. However, I typically supplement with other assessment tools that are not considered a global language assessment. 

Another assessment tool that I use, that is often little known or underrated, is the Oral Passage Understanding Scale (OPUS) for students 5-0 thru 21-0. The publisher, Western Psychological Services (WPS) provides the following information in the overview, “…is a new measure of listening (auditory) comprehension. It evaluates a person’s ability to listen to passages that are read aloud and recall information about them. This ability is key to success in the classroom, as well as in social and occupational settings. The OPUS also measures memory skills, which are integral to listening comprehension.” What I like most about this assessment is the length and complexity of the passages, along with the types of questions. This assessment is most closely aligned to the types of questions students may be asked in a classroom environment. 

If I’m looking to delve into where a student’s breakdown is within the language hierarchy, I like to administer the Language Processing Test-3rd Edition (LPT-3) for students 5-0 thru 11-11. This test is fairly quick to administer and can provide insight into where a child’s strengths and weakness are along the language hierarchy. It is also a great tool to re-administer after a child has received therapy for some time to demonstrate the growth or progress in the weak areas. 

For another global language test (that is lesser known), I like to administer the Illinois Test of Psycholinguistic Abilities-3rd Edition (ITPA-3) for students 5-0 thru 12-11. I got my hands on this assessment when my boss was changing positions. She told me, “this is an oldie but a goodie.” I did not understand how true of a statement that was until I administered the test, then I was hooked–line and sinker! Why am I hooked? This test includes the assessment of spoken analogies, spoken vocabulary, morphology, syntax, reading comprehension, written language, decoding and encoding. It also does not take a significant amount of time to administer this test in its entirety. 

For older students, similar to the ITPA-3, I like to administer the Test of Adolescent and Adult Language-4th Edition (TOAL-4) for students 12-0 thru 24-11. This test measures spoken and written language including word opposites, word derivations, spoken analogies, word similarities, sentence combining, and orthographic usage. Again, this test provides a solid overview of students strengths and weaknesses without taking too long to administer. 

If I am looking to assess social language, I am enjoying the new Clinical Assessment of Pragmatics (CAPs) which is for students 7 thru 18. Why do I like this assessment? 1) It is hard to find a solid standardized pragmatic assessment 2) It has digital video scenes that depict real life scenarios (as much is feasibly possible) 3) It looks at a variety of aspects of social language skills including Pragmatic judgment (comprehension), Performance of pragmatic language (expression), Understanding context and emotions (intent of the speaker through inference, sarcasm, and indirect requests), and Nonverbal cues (facial expressions, prosody, and gestures) and 4) helps determine a student’s strengths and weaknesses. 

Some additional tests that I’m not going to go in-depth about, but that I love having in my assessment toolkit are: the School Motivation and Learning Strategies Inventory (SMALSI), the Gray Oral Reading Test- 5th Edition (GORT-5), and the Phonological Awareness Test-2nd Edition: NU (PAT-2:NU). 

A lesser known tool, that is an informal assessment measure, is the School-Age Language Measures (SLAM) from the LEADERS Project that I use in almost every language assessment I give. Why do I like the SLAM cards? Straight from the LEADERS Project, SLAM cards are “meant to elicit a language sample that can be analyzed in the context of typical language development as well as the child’s background (e.g., educational experiences, family, linguistic and cultural background, etc). For this reason, no scores are included..” Now, they have even updated the site to include guidelines for analysis. The best part, IT IS FREE! 

---

Now, you may have noticed I did not mention the OWLS-II, CELF-5, or CASL-2. I will and have administered all of them. But, I find there are weaknesses with all of these assessments. Let me mention a few– For the OWLS-II, often there is not enough information that can be gleaned simply from administering the Listening Comprehension and Oral Expression Subtests. In regard to the reading subtest, the rigor is just not there. The same goes for the written language subtest. That isn’t to say there isn’t good information that can be obtained from administering these subtests. However, if I had a choice there are better options. In regard to the CELF-5, the primary reason I do not administer the test often is that it relies heavily on auditory input and memory. Many of the students we work with or are assessing, have difficulty with auditory memory, short term memory, and attention which can negatively impact the scores obtained on the assessment. For the CASL-2, I do like the assessment, but it is lacking in some areas including diversity. It also can take quite a long time to administer in its entirety. While the subtests can stand alone, if I am going to administer the CASL-2, I prefer to give the entire assessment so I can look at the strengths and weaknesses across subtests. 

If you’ve read this far, I’d love to hear from you about what your favorite assessment tools are and why! Maureen and I will be doing a live chat on Instagram on … where we will be candidly discussing assessments and try and answer questions that you have! 

You can follow me at @speechtothecore on IG, FB, and Twitter. 

---

References: 

American Speech-Language-Hearing Association. (2020). 2020 Schools survey. Survey summary report: Numbers and types of responses, SLPs. www.asha.org.

Brydon, M. (2018, September). How do we judge the quality of a standardized test? The Informed SLP. 

https://www.theinformedslp.com/qa_standardized_tests.html

Ireland, M. (2019, July). Dynamic Assessment. Paper presented at FLASHA Annual Convention, At Sea, FL.

LEADERS Project. (2020). SLAM The Crayons Picture. LeadersProject.Org. 

SLAM The Crayons Picture

Plante, E. & Vance, R. (1994). Selection of preschool language tests: A data-based approach. Language, Speech, and Hearing Services in Schools, 25, 15-24.

Virginia Department of Education. (2015). SLP Test Comparison. 

Western Psychological Services. (2020). Clinical Assessment of Pragmatics. WPS. 

https://www.wpspublish.com/caps-clinical-assessment-of-pragmatics

Western Psychological Services. (2020). Oral Passage and Understanding Scale. WPS. 

https://www.wpspublish.com/opus-oral-passage-understanding-scale