Students from low-income backgrounds may not have a personal computer or internet access outside of the school campus or school-issued devices, leaving students without the ability to engage online free from their school’s monitoring system. Students without personal devices may be especially uncomfortable using school devices to seek support, if they know that these devices are subject to monitoring. These disparate impacts may be especially pronounced during the COVID-19 pandemic: while learning remotely, students have limited opportunities to seek more information or professional assistance beyond the internet and school-issued devices because they may have limited access to in-person resources. A survey in 2020 found that 8 percent or 4.4 million households do not have a computer always available. In households where a computer was always available, 60 percent received devices from the child’s school or school district.⁷⁵ Similarly, students experiencing homelessness are unlikely to have access to personal devices and may heavily rely on school-issued devices, while especially needing to use them to search for non-academic resources or supports. These contextual factors suggest that self-harm monitoring programs require clear and transparent boundaries, protocols, and appropriate privacy protections. Otherwise, such programs risk harming the students they intend to protect. 

Almost 5 million students in schools across the country are English Language Learners, comprising 9% of all public school students.⁷⁶ Students who are English Language Learners or multilingual, as well as students with disabilities, may be at especially high risk of false, inequitable flagging and of experiencing harm⁷⁷ from being flagged by a monitoring system. Students who are English Language Learners may often use or interact with content in languages that school officials or a monitoring company do not understand or may interpret negatively.⁷⁸ Deeply ingrained biases against students who are English Language Learners can especially influence suspicious and negative interpretations of their writing and activities.⁷⁹ Similarly, students who are English Language Learners may sometimes lack the proficiency or cultural nuance to express themselves as non-English Learners would and may mistakenly use words or phrases that a monitoring program may flag or school officials may misinterpret as a threat to self. As a result, there is a high risk that intent and meaning may get lost in translation, and these students will end up flagged or penalized for innocuous language that school staff fail to accurately decipher. Language barriers or miscommunications and misunderstandings based on differential language use can also surface when monitoring technology scans the content of some students with disabilities. 

In addition, monitoring systems may utilize automatic, computerized translations when scanning student content in non-English languages. These computerized translations are frequently inaccurate and fail to account for idiomatic language use or cultural nuance.⁸⁰ For example, direct translation of a phrase meaning, “You’re annoying me,” from Korean to English resulted in widespread use of the phrase, “Do you wanna die?” in Korean-American communities.⁸¹ These inherent shortcomings of monitoring systems risk disproportionately targeting students who are English Language Learners. For more information on legal protections for students who are English Language Learners, see Legal Implications on page __.

In addition to disproportionate risks of stigmatization and criminalization, students with disabilities may be especially harmed by the ways self-harm monitoring systems analyze student content and writing. Some students with disabilities may interact with online content or use speech differently than their non-disabled peers and may consequently face risks of disproportionate flagging because of the limitations of these systems in interpreting context. Speech that is a manifestation of a disability may be misinterpreted as a threat to self-harm by the monitoring software or by untrained school staff who are unfamiliar with the intersection of disability and mental health. This misinterpretation often occurs with students who have developmental or learning disabilities.⁸² School district leaders should be aware that disparate treatment of students with disabilities, including disproportionately and needlessly flagging them due to typical manifestations of their disabilities, can constitute discrimination and invite potential legal challenges under the Americans with Disabilities Act (ADA). For more information on legal anti-discrimination protections for students with disabilities, please see Legal Implications on page ___.

In addition to harms stemming from sharing student information with law enforcement and referring mental health-related issues to law enforcement, students of color may disproportionately experience other harms from self-harm monitoring.

For example, natural language processing algorithms, which are used by monitoring systems, have been shown to analyze and interpret Black dialects of English used online less accurately than writing by white individuals online.⁸³ Likewise, research at MIT shows many common automated tools that scan online content using natural language processing disproportionately flag writing from Black users.⁸⁴ These examples demonstrate the technological shortcomings, and inequities, inherent in accurately monitoring online content. Such technological inaccuracies lead to racial disparities in students mistakenly flagged by monitoring systems and can cause students of color to disproportionately experience the harms related to mismanaged and privacy-violative monitoring.

Additionally, low-income youth of color and other vulnerable young people may have a very different relationship with school-based and medical-based systems of formal mental healthcare. These student populations may often look to community-based resources and peer social networks as their preferred sources of care and wellness.⁸⁵ While many monitoring technologies proceed from the assumption that school-based systems of care are best positioned to support young people, that may not be the case for many youth. For many students, state-based systems of mental health screenings and services can trigger harmful episodes where they, or their caregivers, have had to deal with the child welfare system, criminal legal system, juvenile justice system, etc.⁸⁶ School districts should keep the different needs and preferences of various student groups in mind and recognize that a one-size-fits-all approach to responding to student self harm will not equally benefit all students.

Another important consideration is the effect of high-surveillance schools⁸⁷ on the academic outcomes and well-being⁸⁸ of Black students and other structurally disadvantaged racial groups. These students may experience monitoring more as a form of surveillance and control of student behavior than as a mental health support tool, due to the greater prevalence of schools with harsh security and zero-tolerance policies in communities of color.⁸⁹ In these cases, implementing a monitoring system can add to an atmosphere of surveillance and criminalization, thereby compromising students’ sense of comfort and support in their school environment. Research from John Hopkins University and Washington University shows that high surveillance schools can lead to lower test scores and graduation rates for Black students, as well as greater disciplinary disparities.⁹⁰

Besides facing the risks of discipline and criminalization described above, LGBTQ students face unique additional harms from having their digital activities monitored. These unique harms can be exacerbated depending on students’ school and home environments. 

Research shows that LGBTQ students who experience victimization or bullying in school face detrimental psychological outcomes, such as higher instances of depression, low self-esteem, increased isolation, and increased suicidal ideation, compared to non-LGBTQ peers.⁹¹ The American Psychological Association has reported⁹² that 64 percent of LGBTQ students feel unsafe in schools because of prejudice and harassment. Sixty percent of these students did not report these incidents to school officials due to fear the situation would be made worse or that the school would take no action to help them. Self-harm monitoring technologies that flag incidents of harassment and prejudice may result in these very fears for LGBTQ students, particularly in unsupportive school environments or without thoughtful protocols for handling flags. 

LGBTQ students have unique interests in who their personal information is shared with, including not just incidents of harassment, but also their sexual orientation and gender identity, again, particularly in situations of unsafe home or school environments. Nonprofit suicide-prevention organization The Trevor Project reports that about 50 percent of LGBTQ youth selectively and carefully decide which family members and teachers and in which contexts they disclose their sexual orientation or gender identity.⁹³ In a national survey conducted by The Trevor Project, less than half of LGBTQ youth had disclosed their identity to an adult at school.⁹⁴ LGBTQ youth are more likely than their peers to seek identity-related resources and help online.⁹⁵ Monitoring systems may discourage youth from seeking LGBTQ-affirming resources online if they fear surveillance, repercussions, or reporting/being outed to school staff, other students, or even their parents through the monitoring program. 

This ability to decide when and how to come out is a critical right that supports mental well-being, particularly when students are in situations where they may feel unsafe or unsupported. This includes school environments where students do not feel confident that their school leaders would support their identities if they were to report bullying or harassment. Consequently, exposing LGBTQ students as a result of monitoring, even with the good intention to help them, can in fact undermine their mental health and safety by damaging this important protective strategy.

School leaders concerned about the mental health of LGBTQ youth should work to create actively affirming and supportive school climates that respect students’ boundaries and privacy, and to provide resources and information in school related to sexual orientation and gender identity, rather than engage in monitoring that would invasively and forcefully expose these students.