How do multilingual “local” students differ from multilingual “immigrant” students on school-based language assessments?

Although a few countries in the world are English-dominant, most others are multilingual. There’s truth to the claim that multilingualism is the global norm. So, if it is the global norm, in what ways can school-based language assessments (even if multilingual) value “local” students’ multilingual language profiles over those of “non-local” or “immigrant” students? To answer this question, I try out an alternative reading of someone else’s study, like I’ve done before: without questioning their research methods or the quality of the data they collected, I provide an alternative interpretation of the same data. First, a disclaimer: the author, Prof. De Angelis, helped teachers in a school district in South Tyrol, Italy, to assess multilingual students’ language proficiencies multilingually in a complex setting. In summarizing and analyzing her study, which I do respect for its validity, rigour, and practicality, I ask: (1) How do multilingual “local” students differ from multilingual “immigrant” students on school-based language assessments? (2) How might these assessments, and the discourses surrounding them, frame the language profiles of the former group as the norm? and (3) How generalizable is this study to other contexts where both “local” and “immigrant” students are very multilingual?

De Angelis, G. (2021). Multilingual narratives: The South Tyrol study. In Multilingual testing and assessment (pp.96-124). Bristol, UK: Multilingual Matters.

In the province of Bolzano-Bozen, South Tyrol, Italy, there are three main languages: the regional language, Ladin (spoken by 4.53% of the population according to the 2011 census), German (69.41%), and Italian (26.06%). When South Tyrol (in the north of Italy, by Austria) became part of Italy after WWI, German speakers were the largest ethnic group. This is a historical case when a subordinate language in a society becomes the politically dominant one, but the language hierarchies in society remain unchanged; see Jaspers, 2011, for a discussion of this phenomenon).

In the 1970s, there was a shift in policy that gave all three language groups the right to education in their mother tongue, leading to the formation of three school boards that exists until today. In schools managed by the German board, German is used to teach all subjects but there is Italian language class; in schools managed by the Italian board, Italian is used to teach all subjects but there’s German language class. This methodologically innovative study—which attempts assessment in three languages—concerns the third school type: where Ladin L1 speakers are educated alongside a wide range of immigrant students in schools run by the Ladin school board.

While immigrants in Bolzano-Bozen came from Austria (3.2%), Ukraine (3.5%), Macedonia (4.2%), Slovakia (4.4%), Kosovo (5.1%), Romania (6.6%), Morocco (7.0%), Pakistan (7.2%), Germany (8.8%), Albania (11.4%), and other countries (38.4%), in this school board, teachers have to be fully trilingual in Ladin, German, and Italian. Thus, the 400 or so teachers are required to take a trilingual exam to get their licenses. According to De Angelis,

The project was implemented because the Ladin school board wanted to provide a good educational experience for the immigrant children entering the education system with limited knowledge of the three languages of instruction (Italian, German, and Ladin), while also maintaining high educational standards for local Ladin children already attending local schools and ensuring their learning needs continued to be fully met. (p. 98)

In such a statement, the linguistic needs of the multilingual “immigrant” children and the multilingual “local” children as presented as opposing, or being in tension—i.e., while one group would be served by making instruction in the three languages easier, the other group would be served by making it more challenging. This may be an oversimplification to help non-specialists understand the research, yet the linguistic challenges for children from immigrant families in the region cannot be underestimated, as they have to deal with the three additional languages immediately upon elementary school enrolment, with English introduced in the fourth year.

On the other hand, these children may not necessarily know nothing of the three languages Ladin, Italian, and German. While 60% of the immigrant population in South Tyrol is 40 or under, meaning it is composed mostly of first-generation adult immigrants and their children attending these schools, the children tend to be born in South Tyrol, or to immigrate at a very young age (p. 101). The study was innovative in that multilingual, school-based testing is rare. Most language testing in schools around the world is monolingual; rarely is it bilingual; one would be hard-pressed to find studies of assessment in at least three languages and are as rigorous as the work done by De Angelis, a professor at Trinity College Dublin—which she takes pains to make accessible to teachers. But as I take you through the chapter, I raise questions about the extent to which the uses and interpretations of the described assessment are any different from other monolingual or bilingual assessments used on language minoritized students.

Methods

The participants were two groups of students (“immigrant” and “Ladin L1”) attending schools in South Tyrol or the nearby Fassa/Fascia Valley of Trentino. The South Tyrol students were enrolled in schools managed by the Ladin school board, while the Fassa/Fascia students were in a bilingual Ladin/Italian region. Some immigrant children attending Italian- and German-medium schools in South Tyrol were also included in the study.

De Angelis explains that “most of the immigrants attending Ladin, Italian, and German schools in South Tyrol were either born in Italy or had reached Italy at a very young age,” while in the Fassa/Fascia valley, all were born outside of Italy but “most of them had reached the country at a very young age” (p. 103). All study participants ranged from 6 to 13/14 years old and had a wide variety of native languages: Ladin, Albanian, Arabic, Bengali (Bangla), Macedonian, Malagasy, Moldavian, Punjabi, Romanian, Serbian, Ukrainian, and Urdu.

One of the strengths of this study was that it was highly accessible to teachers, and therefore the assessment practice was likely sustained in schools after the study was complete. Students had to watch two silent Disney Pixar movies freely available on Youtube: La Luna (The Moon, ~7 minutes) and Il regalo (The Present, ~4.5 minutes). These movies did not contain any significant cultural bias. The students had to narrate what happened in the movies in each of the three languages, and were assessed on narrative coherence (i.e., how coherent their storytelling was) based on a 7-level framework developed by Hutson-Nechkash (1990). The point is that in a second or additional language, people will perform below their age in telling a coherent story.

Hutson-Nechkash’s framework could be assessed using a simple worksheet for teachers to follow (Hutson-Nechkash, 1990, p. 19, as cited in De Angelis, 2021, p. 108).

In the Ladin school board, there were 9 immigrant students and 13 Ladin L1 students who had to say what happened in each story: in Italian, in Ladin, and in German. There were also 7 immigrant students in Italian-medium schools, who were recorded telling both stories in Italian, and 7 immigrant students in German-medium schools, who were recorded telling both stories in German. In the other (smaller) sample site, the Fassa/Fascia Valley, there were 7 immigrant students recorded telling both stories in Ladin and Italian.

De Angelis then grouped all the immigrant students together (9 assessed trilingually + 7 assessed in Italian + 7 assessed in German + 7 assessed bilingually, so N=30, I presume), and all the Ladin L1 students together (N=13, assessed trilingually).

Findings

The first test was to assess whether the stories were most coherent in Ladin, German, or Italian. Thus, the first key finding was that the stories were equally coherent in all three languages—when all the Ladin samples were combined, when all the German samples were combined, and when all the Italian samples were combined, there was a range of performances but no language outperformed either of the others, suggesting that there were speakers of all three languages with higher, lower, and average competence in that language.

Next, De Angelis compared the two groups, “immigrant students” and “L1 Ladin speakers,” and found that for the most part, the Ladin L1 students did better than the immigrant students, at narrating both stories in Ladin and Italian, and one story in German:

Another finding was that Ladin L1 speakers had similar profiles and were more “balanced trilinguals,” while immigrant students varied more as individuals in their proficiencies in each language. Yet this is a general trend, as both groups varied widely at the individual level. Here are the scores for “La Luna” for the 9 immigrant students in South Tyrol, the 13 Ladin L1 speakers in South Tyrol, and the 7 immigrant students in Fassa/Fascia (I’ve rounded scores to the nearest whole point).

	Italian	German	Ladin
South Tyrol, Immigrant Students
I_ST_1	58	8	8
I_ST_2	67	17	67
I_ST_3	42	42	–
I_ST_4	42	25	33
I_ST_5	75	75	75
I_ST_6	67	67	67
I_ST_7	67	67	83
I_ST_8	–	58	50
I_ST_9	33	33	67
South Tyrol, Ladin L1 Students
L_ST_1	67	25	67
L_ST_2	50	50	58
L_ST_3	75	67	50
L_ST_4	58	58	50
L_ST_5	67	50	75
L_ST_6	58	67	75
L_ST_7	75	83	92
L_ST_8	66	58	67
L_ST_9	75	83	83
L_ST_10	75	75	83
L_ST_11	75	75	75
L_ST_12	67	67	75
L_ST_13	75	83	83
Fassa/Fascia, Immigrant Students
I_F_1	42	–	42
I_F_2	75	–	75
I_F_3	75	–	83
I_F_4	83	–	83
I_F_5	83	–	83
I_F_6	58	–	58
I_F_7	–	–	83

While De Angelis is right that “Ladin L1 participants… show remarkable consistency, with socres that are mostly ± 1 of the levels of story grammar development” (pp. 113-114), there is definitely individual variation, for example, South Tyrol immigrant students S5 and S6 are fairly balanced trilinguals, whereas South Tyrol Ladin L1 speakers S1, S3, S5 and S6 have substantially different performances across languages. Interestingly, in Fassa/Fascia, almost everyone seems to be a balanced bilingual of Ladin and Italian (even though that place was not the main focus of the study). How the Italian- and German-medium immigrant students in South Tyrol performed is not reported.

De Angelis states: “None of the Ladin L1 children fell below the 50% threshold in all languages, while several students with an immigrant background did so” (p. 115). BUT: for the “La Luna” story, with scores reproduced above, “several” would mean 5 out of 9 South Tyrol immigrant students and 1 out of 7 Fassa/Fascia immigrant students scored lower than 50 on at least one language. This also means that 4 out of 9 students in South Tyrol still met benchmarks for ALL 3 languages when assessed monolingually, and 6 out of 7 Fassa/Fascia immigrant students did so for BOTH Italian and Ladin when assessed monolingually (De Angelis did not assess the Fassa/Fascia students on German, and perhaps German is particular to South Tyrol). In other words, we need to give these immigrant multilingual students some credit!

De Angelis also calculated that A2 or “high elementary” would be the proficiency level (on the Common European Framework of Reference) that an immigrant student would need to reach to tell a reasonably coherent everyday story, and get a score of 50 (i.e., 50%) on the teacher-filled worksheet adapted from Hutson-Nechkash (1990). While I do not contest that claim, I do think that there are some interesting findings per language that point to common language acquisition thresholds across all students, whatever their home language background.

Note that for Italian, the national language, you see a bell curve, with the mean at 58.99. That is, everyone speaks Italian, more or less with average scores (50% to 60%). For German, we have a fair amount of students at every level rather than a bell curve, with 3-6 students at each level (if we combined B1 and B1+). This suggests that not everyone has equal access to German, either at home or in schooling. For Ladin, note that the highest level does not reach beyond A2, whereas it reaches B1 in the other two languages, and this may mean that, as students get older, Ladin will give way to the more regionally and nationally dominant languages and English. This does not mean that people will forget what they know in Ladin, but they may not develop it for academic purposes at the secondary or tertiary level, and this is something that the school district is rightly trying to change by offering early formal education in Ladin as a foundation. (Of course, that language development also depends on availability of reading materials in Ladin at those levels of education.)

When De Angelis surveyed parents about their language use outside of school, she found that “Ladin L1 children use the three languages of instruction in a fairly balanced manner” (p. 116). This is because they are exposed to Ladin at home and have to learn German and Italian in the community. On the other hand, immigrant children use (1) another language 26.39% of the time, which is their ethnic language, (2) Italian (the national language) the most, at 41.55% of the time; (3) German (a major regional language) in comparable amounts as their home language, at 24.29% of the time, and (4) Ladin, which is not their ethnic language, only 7.77% of the time (while Italian and German are probably used with Ladin L1 speakers). These are not surprising language acquisition patterns for different groups of students’ everyday needs, but the findings clearly explain why immigrant students would struggle in schools where a third of the instruction is in Ladin, because people are understandably trying to promote this language native to the area for academic purposes.

In terms of multilingualism in daily life, the two groups (Ladin L1 speakers and immigrant students) have equal multilingual attainments: the “typical” student in either group is trilingual in Italian, German, and a third language (Ladin or whatever their home language is).

Implications of the Findings

Although De Angelis mentions that “some flexibility was applied during scoring by accepting the occasional use of words in a language other than the language of narration” (p. 119), she also reminds us of the purpose of the study: to assess immigrant students’ proficiency in each language because they are being taught in each of the three official languages. A reference to translanguaging is made near the conclusion:

Had the students been given the opportunity to narrate the story by mixing languages at will—as a translanguaging approach would postulate—it would not have been possible to answer any of the questions asked and therefore identify potential threshold levels for narrative abilities in individual languages. (p. 119)

That threshold appears to be A2 (high elementary) in the CEFR benchmarks, or a 50% pass on Hutson-Nechkash’s (1990) framework, for a very simple story with no particular cultural content. In other words, if a child achieves A2 proficiency in one language and can tell an organized story in that language, these narrative skills can start to transfer to other languages. Moreover, the discovery that all languages were equal when it was languages, not students, that were compared, suggests that all students have age-appropriate cognitive and narrative skills in at least one language (which might also be an immigrant language, even if immigrant languages were not tested).

It is also worth trying to see immigrant students from an asset-oriented linguistic perspective, as the majority DID achieve age-appropriate proficiency in at least one L2 that was one of the 2 or 3 dominant languages, and a fair number met the benchmark in both/all languages. In Table 12 above (scores for one story):

• Of the 7 immigrant students in Fassa/Fascia, only 1 student had scores below 50 in both Ladin and Italian. Of the other 6, 1 student met the benchmark for Ladin but not Italian, and 5 met the benchmark for both languages.
• Of the 9 immigrant students in South Tyrol, only 2 scored below the benchmark in all 3 languages. Of the other 7 students, 1 met the benchmark for Italian only, 1 for Ladin only, 1 for Italian and Ladin, 1 for German and Ladin, and 3 for all 3 languages.

Since all the immigrant students were born in Italy or arrived at a very young age, the dominant language of most/all of them would be one of the three official languages. For the purposes of elementary schooling, the majority of immigrant children in Fassa/Fascia in this study (6 out of 7) could function in both Ladin and Italian, and the majority of immigrant students in South Tyrol who were schooled trilingually could function in at least one of the official languages, though not necessarily all three (i.e., only 3 out of 9 students achieved the score of 50 or more across all three languages). The educational challenge therefore seems to be more daunting for the latter group, because learning one or two additional languages is still not enough. Which leads me to wonder: if most/all of these immigrant children were born in Italy or arrived at a young age, and it would be inevitable that their dominant language would be one of the 3 dominant languages in South Tyrol rather than their ethnic language, surely there is a manageable schooling placement—but perhaps trilingual instruction isn’t the ideal one. I wonder about the wider social context here: why were they placed in the Ladin school catchment?

Moreover, while it is good that the teachers in the Ladin school catchment were trilingually fluent, able to adjust their language use spontaneously to scaffold students’ understanding during instruction, and to provide trilingual teacher-made learning materials… I wonder how they were trained to draw on other languages students knew as learning resources, even if they themselves did not speak these languages (e.g., Woodley & Brown, 2016).

Finally, I wonder to what extent this multilingual assessment is different from previous bilingual and monolingual assessments that test one language at a time. As De Angelis explained, the purpose of the study was to assess proficiency in individual languages, but this is not quite the same as asking what students know or can do if allowed to draw on their whole language repertoire, not only one part of it (García, 2009; Shohamy, 2011).

I wouldn’t fault De Angelis for trying to measure students’ proficiencies in distinct languages if that is what she clearly set out to do, and she also developed a teacher-friendly way of doing so. But all this appears to have been done in the Ladin school board in South Tyrol with a concern that the multilingual students who were non-ethnic Ladin speakers (positioned as “immigrants” despite being born in Italy or having arrived as an early age) were not becoming “balanced trilinguals” for the purposes of schooling, like the Ladin L1 students who also spoke Italian and German (yet might be negatively positioned for their dialectal ways of using Italian or German in another context). I also wonder to what extent this multilingual assessment was any different from monolingual, English-only assessments or bilingual, Spanish-only and English-only assessments for Latinx students in the United States (tests that are criticized by translanguaging scholar Ofelia García and her colleagues for the way they position bilingual students in that country).

In short, we should not assume that adding more languages to program design, instruction and assessment—but keeping the same deficit positioning or non-measurement of funds of knowledge of linguistically minoritized students in that context—makes things any better just because we are doing things multilingually and/or adding regional languages to the mix. In Africa, Europe, Latin America, and South/Southeast/East Asia, multilingualism is the norm, but the dominance of “majoritized bi/multilingualism” and the marginalization of “minoritized bi/multilingualism” in schooling is still pervasive in these more multilingual societies. The key to achieving linguistic equity in various settings is not simply to attack monolingualism or target-language-only policies in classrooms, but to always question how majority group language practices (whether mono-, bi-, or multilingual) are seen as the most normal in a place, or “When in Rome, do as the Romans do!” (as if there are only Romans in Rome). Such questioning has to start in school, for it to trickle up to the wider society (Hornberger, 2003; Seltzer & García, 2020).

References

García, O. (2009). Bilingual education in the 21st century: A global perspective. Malden, MA; Oxford, UK: Wiley-Blackwell.

Hornberger, N. H. (Ed.) (2003). Continua of biliteracy: An ecological framework for educational policy, research, and practice in multilingual settings. Clevedon, UK: Multilingual Matters.

Hutson-Nechkash, P. (1990). Storybuilding: A guide to structuring oral narratives. Eau Claire, WI: Thinking Publications.

Jaspers, J. (2011). Talking like a ‘zerolingual’: Ambiguous linguistic caricatures at an urban secondary school. Journal of Pragmatics, 43(5), 1264-1278. https://doi.org/10.1016/j.pragma.2010.05.012

Seltzer, K., & García, O. (2020). Broadening the view: Taking up a translanguaging pedagogy with all language-minoritized students. In Z. Tian, L. Aghai, P. Sayer & J. Schissel (Eds.), Envisioning TESOL through a translanguaging lens: Global perspectives (pp. 23-42). Cham, Switzerland: Springer.

Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95(3), 418-429. https://doi.org/10.1111/j.1540-4781.2011.01210.x

Woodley, H., & Brown, A. (2016). Balancing windows and mirrors: Translanguaging in a multilingual classroom. In O. García & T. Kleyn (Eds.), Translanguaging with multilingual students: Learning from classroom moments (pp. 83-99). Abingdon, UK; New York, NY: Routledge.