Could school computers be a tool in preventing ISIS from recruiting teens?

A software company is piloting a service that alerts educators when students use phrases on school computers that could indicate radicalization or sympathy with violent extremism.

Teenagers in states ranging from North Carolina to Minnesota have been arrested this year on suspicion of planning to carry out violent attacks in the name of the Islamic State. Can schools do anything to detect and prevent the recruitment of young people into terrorist groups?

A software company is piloting a service that alerts educators when students use phrases on school computers that could indicate radicalization or sympathy with violent extremism.

While still nascent, the effort raises a host of questions: Would such technology provide a useful tool or pose a burdensome new responsibility on educators? Would it impinge upon student privacy and free speech? Could it, however unintentionally, stigmatize some students, such as Muslim immigrants?

Impero Software, based primarily in Britain, already works with schools for classroom teaching and to detect other potential harms such as bullying, suicide, and grooming for sexual exploitation in dozens of countries and 40 percent of high schools in Britain. It also serves 500 districts in the United States. 

Impero’s anti-extremism pilot project began after the British government in February started requiring schools to do more to prevent young people’s involvement in terrorism. But it’s also being tested in five unnamed schools in Texas, New York, and Pennsylvania.

A number of school technology, security, and cybersafety experts are skeptical about whether such a monitoring system would be a good fit for US schools.

“It’s not unreasonable to say school resource officers and principals may … use [digital] tools to see what might pop up.… But if you start crossing the line into individual monitoring, there are some legal and practicality issues,” says Kenneth Trump, president of National School Safety and Security Services.

Mr. Trump also wonders whether the monitoring system could unfairly force busy teachers or principals into the role of cyberpolice.

The software offers a frequently updated list of key words often found in online interactions intended to radicalize or recruit young people. Schools can customize what to flag and who will be notified. Officials can then look at a screen grab or short video to see the context in which students are using the words on school computers.

Teachers “can start building up a picture over time, so one week it might be they’re noticing a student looking up news article about extremism or ISIS, and the next week they’re looking up beheading videos,” says Sally-Ann Griffiths, E-safety and product marketing manager for Impero and a former teacher. “It can really empower a teacher to say we might need to do some intervention with this child and support them.”

Sixteen- to 25-year-olds are the most at risk of being radicalized, Ms. Griffiths says. “If you are exposed to these ideologies in your teenage years, and maybe you’ve been the victim of some sort of racism or bullying and you have some perceived grievances, then you can be very easily manipulated if you are not given some sort of counternarrative,” she says. 

The key word list, developed in conjunction with Quilliam Foundation, a British anti-extremist think tank, has terms primarily related to Islamic extremism, but also includes indicators of far-right extremism. Here are a few examples (edited for clarity and brevity):

• YODO – Acronym for “You Only Die Once,” a jihadist parody of YOLO.
• Kuffs – Arabic term for infidel, hate speech used by Islamists to denote non-Muslims.
• al Farouq Institute for Cubs – Terrorist training camp in Raqqah, Syria, for young members of ISIS.
• Don Black – American white nationalist and white supremacist, editor of the Stormfront website, and formerly a leader of the Ku Klux Klan and member of the American Nazi party.

Teachers should look not just at student computer use but also behavior patterns, such as sudden changes in dress or academic habits, or becoming vocal about viewpoints that echo extremism, Griffiths says.

Students may be savvy enough to do certain types of searches only at home, but there are plenty of potentially harmful situations that they wouldn’t be hiding, Griffiths says. For instance, they may be unaware that they are being groomed for recruitment, a process that can be similar to sexual predators manipulating children online.

But privacy and cybersecurity experts say they believe the software’s efficacy would be limited – and may be outweighed by the concerns.

While the tool may have potential in certain areas if large numbers of youths are being targeted for recruitment, generally speaking, “radicalization is too subtle to find using digital technology that schools can handle,” says online safety expert Parry Aftab.

The US context is different from Britain because of its size and diversity, the relatively small number of cases so far, and the lack of a mandate here for schools to counter terrorism recruitment. In England, The New York Times reported in February, 600 people had tried or succeeded in traveling to Syria to fight, compared with 150 from the US. 

Impero is not the only company offering to monitor students’ digital behavior, although it is the first to say it will include radicalization as a category. A variety of companies say they have been able to prevent student harm, such as suicide and bullying, through such monitoring.

Florida’s Orange County schools recently signed up with Snaptrends, a company that helps districts monitor publicly posted information and detect elements that pose safety concerns. Snaptrends declined to comment for this article.

Geo Listening has worked with districts such as Glendale, Calif., to monitor students’ public online postings. The percentage of postings that relate to radical extremism is “minuscule,” though “we have certainly reported matched criteria for this type of concern to several of our clients over the recent school year,” Geo Listening founder Chris Frydrych writes in an e-mail to the Monitor.

Outside of large urban areas, school districts may lack the budget to either buy such software or have staff available to respond to what it finds, says Mike Ribble, the IT director in the Manhattan-Ogden district in Kansas and the man behind the Digital Citizenship website.

Many teachers still need more basic training on cybersafety, and a number of organizations are promoting the teaching of better digital citizenship. Teachers are interested in the question, “How do we mitigate the negative pieces but still provide all the resources and tools that technology can provide?” Mr. Ribble says.

Privacy and antidiscrimination advocates also worry that computer monitoring may lead to unfair consequences for students.

Publicly posted material isn’t off limits for school monitoring, says Brenda Leong, senior counsel for the Future of Privacy Forum in Washington. But schools “should be cautious about monitoring that could stigmatize minority students by focusing on their activity,” and they should “be sure students get a fair hearing” before taking action that would penalize a student, she writes in an e-mail to the Monitor.

That also could be a concern for discipline-reform advocates, who have long been highlighting what they see as an overcriminalization of student behavior and a school-to-prison pipeline. 

Somali students in Minnesota have already been stereotyped because of the arrests in recent years of a number of Somali youths accused of terrorist ties, says Jaylani Hussein, executive director of CAIR-Minnesota (the Council on American-Islamic Relations).

He worries that if a software program like Impero’s were put in place, teachers wouldn’t be well enough informed to avoid overreacting to things they see online. A Somali adult-education student had his life turned upside down because of law enforcement investigations after a teacher reported him for reading maps, which he was using to train to become a cab driver, Mr. Hussein says.

“You can see how …  when teachers don’t know what they are really looking for, and they start playing this role, how horrible it can turn,” he says.