Challenges in Assessing Bilingual Populations First in the series on Development of a Bilingual Test for Spanish-English Children, this presentation discusses why bilingual children are not just "two monolinguals in one". Presentation Video
Free
Presentation Video  |   November 01, 2013
Challenges in Assessing Bilingual Populations
Author Affiliations & Notes
  • Elizabeth D. Peña
    University of Texas at Austin
  • Originally presented at the ASHA Convention (November 2013) as part of the session Development of a Bilingual Test for Spanish-English Children: A Long and Winding Road. Co-Presenters: Elizabeth D. Peña, University of Texas at Austin; Aquiles Iglesias, Temple University; Vera F. Gutierrez-Clellen, San Diego State University; Brian A. Goldstein, La Salle University; and Lisa M. Bedore, University of Texas at Austin.
    Originally presented at the ASHA Convention (November 2013) as part of the session Development of a Bilingual Test for Spanish-English Children: A Long and Winding Road. Co-Presenters: Elizabeth D. Peña, University of Texas at Austin; Aquiles Iglesias, Temple University; Vera F. Gutierrez-Clellen, San Diego State University; Brian A. Goldstein, La Salle University; and Lisa M. Bedore, University of Texas at Austin.×
  • Disclosure: All of the above-listed authors/co-presenters benefit financially from royalty payments from the Bilingual English-Spanish Assessment (BESA).
    Disclosure: All of the above-listed authors/co-presenters benefit financially from royalty payments from the Bilingual English-Spanish Assessment (BESA).×
Article Information
Cultural & Linguistic Diversity / Research Issues, Methods & Evidence-Based Practice / Clinical Practice Research / Diagnostics, Screening and Assessment Research
Presentation Video   |   November 01, 2013
Challenges in Assessing Bilingual Populations
CREd Library, November 2013, doi:10.1044/cred-pvd-c13006
CREd Library, November 2013, doi:10.1044/cred-pvd-c13006

The following is a transcript of the presentation video, edited for clarity. Presentation slides are available for download via the PDF button in the toolbar.

Originally presented at the ASHA Convention (November 2013) as part of the session Development of a Bilingual Test for Spanish-English Children: A Long and Winding Road. Videos in this series are:

  1. Challenges in Assessing Bilingual Populations

    (Elizabeth D. Peña, University of Texas at Austin)

  2. Steps in Test Development

    (Elizabeth D. Peña, University of Texas at Austin)

  3. Bilingual Phonology Assessment Design

    (Brian A. Goldstein, La Salle University)

  4. Bilingual Pragmatics Assessment Design

    (Aquiles Iglesias, Temple University)

  5. Bilingual Semantics Assessment Design

    (Lisa M. Bedore, University of Texas at Austin)

  6. Bilingual Morphosyntax Assessment Design

    (Vera F. Gutierrez-Clellen)

What are the challenges of assessing bilingual kids? Grosjean has said, and this has been quoted very often, that bilinguals are not two monolinguals in one.
When we're testing bilingual kids, we probably need to do something different than what we would do for assessment of a monolingual. We can't necessarily give the same thing that you would do for a monolingual Spanish child or a monolingual English-speaking child in two languages.
Measuring L1 and L2
One of the challenges in looking at bilingualism is that it's highly variable at an individual level. The amount of contact that children have with each of their two languages is going to vary tremendously among the children you might be working with.
What are ways that we might think about measuring their first and their second language? There are a number of things that are recommended that we looked at in the literature.
Things like, the age of their second language exposure. At what point did they start learning English? We have some kids who start learning English -- and Spanish -- from birth. And some kids who have their first English exposure in preschool. And others when they're in kindergarten.
What is their current use and exposure? How much do they get English input and Spanish input in a typical day?
Then we can do direct testing. When we do direct testing, what do we test? Do we test vocabulary to make a determination of their language proficiency or their language dominance? Do we test grammar? Do we test their narratives? The kinds of measures that you test can also vary tremendously.
Language Dominance
One of the things we have been looking at -- because we think it's really important to disambiguate -- is the level of dominance in a language from language ability.
What we're trying to do as speech-language pathologists is identify whether this is characteristic of an impairment in language or an impairment in speech. So we want to find measures that might help us disambiguate that from the exposure they have to each of the two languages.
One of the things we've looked at is how much exposure they have. How much do they use Spanish? How much do they use English? And how much do they hear each? We've looked at this against, for example, semantics.
In this slide you can see that children who use English 60% of the time or more were also heavily English-dominant on a test of semantics.
Over here, children who used Spanish 60% or more of the time were also heavily Spanish-dominant of a test of semantics in Spanish.
But the bilinguals who use Spanish and English more or less the same, between 40% and 60% which clusters around 50/50, there's a small number of them who also showed balanced performance in semantics. So we have these differences here, on the ns. In terms of exposure, we have 122 kids who have balanced exposure, 80 children who show balance in semantics, but 39 who show balance on both of those measures.
For these bilingual kids, we get more dispersion around the middle. These bilingual kids are much more variable in what they hear, and how they perform on different measures.
We take the same kids, and we do the same thing on morphosyntax. Again, at the two ends of the continuum -- the kids who get exposed mainly to English do really well in English, the kids who get exposed mainly to Spanish do really well with Spanish. But those kids in the middle show some differences. In morphosyntax, we have an even smaller number of kids who show balanced exposure in Spanish and English and balanced performance in Spanish and English morphosyntax. We have these differences here represented by these boxes. So 122 who have equal exposure, and 33 who are balanced in their performance on morphosyntax. So you get a lot of dispersion around the middle for those bilingual kids.
Then last we look at the crosstabs between morphosyntax and semantics, and we get that same kind of pattern. This is again with that same group of kids.
The story I'm trying to emphasize here is that bilinguals are variable on a number of these different variables.
True Language Impairment vs. Outcomes of Divided Input
A challenge, then, is separating what's a true language impairment and what is simply an outcome of having divided input and divided use of each of your two languages.
Have you not had enough time to master the semantic system yet, and that's why the scores are low? Or is it really language impairment? Have you not mastered the grammatical system of your second language? Or are you still in the process of learning it? And how do we disambiguate those two?
Because of this mixed input, sometimes second language acquisition looks like primary language impairment. And also first language loss might also look like language impairment.
When you're going through that shift of acquiring a second language and maybe the first language is in attrition, or there's not as much exposure to the first language, you might start making errors in that first language that look very much like the errors that you see in monolingual children that have language impairment.
This is what we have to disambiguate.
We know also that in different kinds of domains, children have different kinds of experiences. So we might see sociocultural experiences also play a role in how kids perform on standardized tests, especially tests that are normed on a monolingual population.
Kids may have gaps. They may know some things in one language and have lexicalized things in another language. It's really common for preschool kids to have really good home vocabulary in Spanish, and really good school vocabulary in English, but they don't necessarily have translation equivalents of all those things.
And you're going to see this across the different domains. So depending on what the demands are -- and those linguistic demands are tied to a specific language -- you might see some variability across their two languages.
So what do we do, in terms of thinking about test development for this population that is highly variable, that has divided time in each of their two languages. Where age of acquisition of their first and second language is going to vary tremendously. And where familiarity of content is going to vary.
We need to do something more than what you would do for a monolingual test.
When you're developing a test for a monolingual population, these aren't variables that you necessarily take into account. Maybe familiarity of content -- but age of acquisition of L1 and L2 isn't an issue. And divided time between two languages isn't an issue.
Taking Bilingual Variables into Account
So how do you take these into account?
Usually, when you develop a standardized test, your goal is to classify language impairment, or phonological impairment, and you want to develop items that are challenging for your clinical group. So those items are sensitive items for phonological impairment. Or for language impairment.
At the same time, you don't want those items to be sensitive to different levels of language experiences. You want those items to be insensitive, as much as possible, to variation that might come from things that are not related to the clinical marker that you're looking for.
So, what we have been really focusing on is looking at items, and looking at the item-level data, in terms of development of a test.
We used an item-analysis approach where we compare item-level differences between children with typical development and children with language impairment or phonological impairment. Items that did a good job at differentiating those two groups are items that we want to keep.
At the same time, we also want to make sure that those items still work at these different levels of linguistic exposure. If you vary on this other dimension of L1 and L2 exposure or on the age at which children starting learning a second language, do these items still differentiate between children with typical development and children with language impairment or phonological impairment?
References for this Series
Allen, M. & Yen, W. (1979). Introduction to measurement theory. Belmont, CA: Wadsworth.
Allen, M. & Yen, W. (1979). Introduction to measurement theory. Belmont, CA: Wadsworth.×
Alt, M., Meyers, C. & Figueroa, C. (2013). Factors that influence fast mapping in children exposed to Spanish and English. Journal of Speech, Language, and Hearing Research, 56(4), 1237–1248 [Article]
Alt, M., Meyers, C. & Figueroa, C. (2013). Factors that influence fast mapping in children exposed to Spanish and English. Journal of Speech, Language, and Hearing Research, 56(4), 1237–1248 [Article] ×
Alt, M. & Suddarth, R. (2012). Learning novel words: Detail and vulnerability of initial representations for children with specific language impairment and typically developing peers. Journal of Communication Disorders, 45(2), 84–97 [Article] [PubMed]
Alt, M. & Suddarth, R. (2012). Learning novel words: Detail and vulnerability of initial representations for children with specific language impairment and typically developing peers. Journal of Communication Disorders, 45(2), 84–97 [Article] [PubMed]×
Anderson, R. T. (2001). Lexical morphology and verb use in child first language loss: A preliminary case study investigation. International Journal of Bilingualism, 5(4), 377–401 [Article]
Anderson, R. T. (2001). Lexical morphology and verb use in child first language loss: A preliminary case study investigation. International Journal of Bilingualism, 5(4), 377–401 [Article] ×
Bedore, L. M., Peña, E. D., Summers, C. L., Boerger, K. M., Resendiz, M. D., Greene, K., Bohman, T. M. & Gillam, R. B. (2012). The measure matters: Language dominance profiles across measures in Spanish/English bilingual children. Bilingualism: Language and Cognition, 15(3), 616–629 [Article]
Bedore, L. M., Peña, E. D., Summers, C. L., Boerger, K. M., Resendiz, M. D., Greene, K., Bohman, T. M. & Gillam, R. B. (2012). The measure matters: Language dominance profiles across measures in Spanish/English bilingual children. Bilingualism: Language and Cognition, 15(3), 616–629 [Article] ×
Bedore, L. M., Peña, E. D., Gillam, R. B. & Ho, T. (2010). Language sample measures and language ability in Spanish-English bilingual kindergarteners. Journal of Communication Disorders, 43(6), 498–510 [Article] [PubMed]
Bedore, L. M., Peña, E. D., Gillam, R. B. & Ho, T. (2010). Language sample measures and language ability in Spanish-English bilingual kindergarteners. Journal of Communication Disorders, 43(6), 498–510 [Article] [PubMed]×
Bishop, D. V. (1998). Development of the children's communication checklist (CCC): A method for assessing qualitative aspects of communicative impairment in children. Journal of Child Psychology and Psychiatry, 39(6), 879–891 [Article] [PubMed]
Bishop, D. V. (1998). Development of the children's communication checklist (CCC): A method for assessing qualitative aspects of communicative impairment in children. Journal of Child Psychology and Psychiatry, 39(6), 879–891 [Article] [PubMed]×
Bonifacio, S., Girolametto, L., Bulligan, M., Callegari, M., Vignola, S. & Zocconi, E. (2007). Assertive and responsive conversational skills of Italian-speaking late talkers. International Journal of Language & Communication Disorders, 42(5), 607–623 [Article]
Bonifacio, S., Girolametto, L., Bulligan, M., Callegari, M., Vignola, S. & Zocconi, E. (2007). Assertive and responsive conversational skills of Italian-speaking late talkers. International Journal of Language & Communication Disorders, 42(5), 607–623 [Article] ×
Brice, A. & Montgomery, J. (1996). Adolescent pragmatic skills: A comparison of Latino students in English as a second language and speech and language programs. Language, Speech, and Hearing Services in Schools, 27(1), 68–81 [Article]
Brice, A. & Montgomery, J. (1996). Adolescent pragmatic skills: A comparison of Latino students in English as a second language and speech and language programs. Language, Speech, and Hearing Services in Schools, 27(1), 68–81 [Article] ×
Cotton, E. & Sharp, J. (1988). Spanish in the Americas. Washington, DC: Georgetown University Press.
Cotton, E. & Sharp, J. (1988). Spanish in the Americas. Washington, DC: Georgetown University Press.×
Carrow, E. (1974). Austin Spanish articulation test. Austin, TX: Learning Concepts.
Carrow, E. (1974). Austin Spanish articulation test. Austin, TX: Learning Concepts.×
Hammond, R. (2001). The Sounds of Spanish: Analysis and application (with special reference to American English). Somerville, MA: Cascadilla Press.
Hammond, R. (2001). The Sounds of Spanish: Analysis and application (with special reference to American English). Somerville, MA: Cascadilla Press.×
Goldstein, B. (2007). Measuring phonological skills in bilingual children: Methodology and clinical applications. In Centeno J., Obler L. & Anderson R. (Eds.). Studying Communication Disorders In Spanish Speakers: Theoretical, Research, & Clinical Aspects. Clevedon, UK: Multilingual Matters.
Goldstein, B. (2007). Measuring phonological skills in bilingual children: Methodology and clinical applications. In Centeno J., Obler L. & Anderson R. (Eds.). Studying Communication Disorders In Spanish Speakers: Theoretical, Research, & Clinical Aspects. Clevedon, UK: Multilingual Matters.×
Goldstein, B. & McLeod, S. (2012). Typical and atypical multilingual speech acquisition. In McLeod S. & Goldstein B. (Eds.). Multilingual aspects of speech sound disorders in children. Clevedon, UK: Multilingual Matters.
Goldstein, B. & McLeod, S. (2012). Typical and atypical multilingual speech acquisition. In McLeod S. & Goldstein B. (Eds.). Multilingual aspects of speech sound disorders in children. Clevedon, UK: Multilingual Matters.×
Goldstein, B. & Gildersleeve-Neumann, C. (2012). Phonological development and disorders. In Goldstein B. (Ed.). Bilingual language development and disorders in Spanish-English speakers (2nd edition). Baltimore: Brookes.
Goldstein, B. & Gildersleeve-Neumann, C. (2012). Phonological development and disorders. In Goldstein B. (Ed.). Bilingual language development and disorders in Spanish-English speakers (2nd edition). Baltimore: Brookes.×
Gray, S. (2004). Word learning by preschoolers with specific language impairment: Predictors and poor learners. Journal of Speech, Language, and Hearing Research, 47(5), 1117–1132 [Article]
Gray, S. (2004). Word learning by preschoolers with specific language impairment: Predictors and poor learners. Journal of Speech, Language, and Hearing Research, 47(5), 1117–1132 [Article] ×
Gray, S. (2005). Word learning by preschoolers with specific language impairment: Effect of phonological or semantic cues. Journal of Speech, Language, and Hearing Research, 48(6), 1452–1467 [Article]
Gray, S. (2005). Word learning by preschoolers with specific language impairment: Effect of phonological or semantic cues. Journal of Speech, Language, and Hearing Research, 48(6), 1452–1467 [Article] ×
Gutiérrez-Clellen, V. F., Restrepo, M. A. & Simón-Cereijido, G. (2006). Evaluating the discriminant accuracy of a grammatical measure with Spanish-speaking children. Journal of Speech, Language, and Hearing Research, 49(6), 1209–1223 [Article]
Gutiérrez-Clellen, V. F., Restrepo, M. A. & Simón-Cereijido, G. (2006). Evaluating the discriminant accuracy of a grammatical measure with Spanish-speaking children. Journal of Speech, Language, and Hearing Research, 49(6), 1209–1223 [Article] ×
Gutiérrez-Clellen, V. F. & Simón-Cereijido, G. (2007). Evaluating the discriminant accuracy of a grammatical measure with Latino English-speaking children. Journal of Speech, Language, and Hearing Research, 50(4), 968–981 [Article]
Gutiérrez-Clellen, V. F. & Simón-Cereijido, G. (2007). Evaluating the discriminant accuracy of a grammatical measure with Latino English-speaking children. Journal of Speech, Language, and Hearing Research, 50(4), 968–981 [Article] ×
Hodson, B. (1986). Assessment of phonological processes-Spanish. San Diego: Los Amigos Research Associates.
Hodson, B. (1986). Assessment of phonological processes-Spanish. San Diego: Los Amigos Research Associates.×
Jacobson, P. F. & Schwartz, R. G. (2005). English past tense use in bilingual children with language impairment. American Journal of Speech-Language Pathology, 14(4), 313–323 [Article] [PubMed]
Jacobson, P. F. & Schwartz, R. G. (2005). English past tense use in bilingual children with language impairment. American Journal of Speech-Language Pathology, 14(4), 313–323 [Article] [PubMed]×
Kohnert, K. J. & Bates, E. (2002). Balancing bilinguals ii: Lexical comprehension and cognitive processing in children learning Spanish and English. Journal of Speech, Language, and Hearing Research, 45(2), 347–359 [Article]
Kohnert, K. J. & Bates, E. (2002). Balancing bilinguals ii: Lexical comprehension and cognitive processing in children learning Spanish and English. Journal of Speech, Language, and Hearing Research, 45(2), 347–359 [Article] ×
Leonard, L. B., Eyer, J. A., Bedore, L. M. & Grela, B. G. (1997). Three accounts of the grammatical morpheme difficulties of English-speaking children with specific language impairment. Journal of Speech, Language, and Hearing Research, 40(4), 741–753 [Article]
Leonard, L. B., Eyer, J. A., Bedore, L. M. & Grela, B. G. (1997). Three accounts of the grammatical morpheme difficulties of English-speaking children with specific language impairment. Journal of Speech, Language, and Hearing Research, 40(4), 741–753 [Article] ×
Mcgregor, K. K., Newman, R. M., Reilly, R. M. & Capone, N. C. (2002). Semantic representation and naming in children with specific language impairment. Journal of Speech, Language, and Hearing Research, 45(5), 998–1014 [Article]
Mcgregor, K. K., Newman, R. M., Reilly, R. M. & Capone, N. C. (2002). Semantic representation and naming in children with specific language impairment. Journal of Speech, Language, and Hearing Research, 45(5), 998–1014 [Article] ×
Mason, M., Smith, M. & Hinshaw, M. (1976). Medida Española de articulación (Measurement of Spanish Articulation). San Ysidro, CA: San Ysidro School District.
Mason, M., Smith, M. & Hinshaw, M. (1976). Medida Española de articulación (Measurement of Spanish Articulation). San Ysidro, CA: San Ysidro School District.×
Mattes, L. (1985). Spanish articulation measures. Oceanside, CA: Academic Communication Associates.
Mattes, L. (1985). Spanish articulation measures. Oceanside, CA: Academic Communication Associates.×
Melgar de Gonzalez, M. (1976). Como detectar al niño con problemas del habla [Identifying the child with speech problems]. Mexico City: Trillas.
Melgar de Gonzalez, M. (1976). Como detectar al niño con problemas del habla [Identifying the child with speech problems]. Mexico City: Trillas.×
Paul, R. & Norbury, C.F. (2012). Language disorders from Infancy through adolescence. St. Louis, MO: Elsevier.
Paul, R. & Norbury, C.F. (2012). Language disorders from Infancy through adolescence. St. Louis, MO: Elsevier.×
Peña, E.D., Gutíerrez-Clellen, V.F., Iglesias, A., Goldstein, B. & Bedore, L.M. (2014). Bilingual English Spanish Assessment (BESA). AR-Clinical Publications.
Peña, E.D., Gutíerrez-Clellen, V.F., Iglesias, A., Goldstein, B. & Bedore, L.M. (2014). Bilingual English Spanish Assessment (BESA). AR-Clinical Publications.×
Restrepo, M. A. & Kruth, K. (2000). Grammatical characteristics of a Spanish-English bilingual child with specific language impairment. Communication Disorders Quarterly, 21(2), 66–76 [Article]
Restrepo, M. A. & Kruth, K. (2000). Grammatical characteristics of a Spanish-English bilingual child with specific language impairment. Communication Disorders Quarterly, 21(2), 66–76 [Article] ×
Rice, M. L. & Wexler, K. (1996). Toward tense as a clinical marker of specific language impairment in English-speaking children. Journal of Speech, Language, and Hearing Research, 39(6), 1239–1257 [Article]
Rice, M. L. & Wexler, K. (1996). Toward tense as a clinical marker of specific language impairment in English-speaking children. Journal of Speech, Language, and Hearing Research, 39(6), 1239–1257 [Article] ×
Sheng, L., Peña, E. D., Bedore, L. M. & Fiestas, C. E. (2012). Semantic deficits in Spanish-English bilingual children with language impairment. Journal of Speech, Language, and Hearing Research, 55(1), 1–15 [Article]
Sheng, L., Peña, E. D., Bedore, L. M. & Fiestas, C. E. (2012). Semantic deficits in Spanish-English bilingual children with language impairment. Journal of Speech, Language, and Hearing Research, 55(1), 1–15 [Article] ×
Toronto, A. (1977). Southwest Spanish articulation test. Oceanside, CA: National Education Laboratory Publishers, Inc.
Toronto, A. (1977). Southwest Spanish articulation test. Oceanside, CA: National Education Laboratory Publishers, Inc.×
Vermeer, A. & Shohov, S.P. (2004). Exploring the lexicon: Quantitative and qualitative aspects of children's L1/L2 word knowledge. In Advances in Psychology Research. Hauppauge, NY: Nova Science Publishers.
Vermeer, A. & Shohov, S.P. (2004). Exploring the lexicon: Quantitative and qualitative aspects of children's L1/L2 word knowledge. In Advances in Psychology Research. Hauppauge, NY: Nova Science Publishers.×