‘Woeful’ Evidence for Efficacy of Anonymous Reporting Tools

Not long after a gunman killed 10 people at Santa Fe High School in Texas in 2018, the Texas Department of Public Safety created iWatchTexas, an anonymous reporting system in which community members may offer tips to authorities about suspicious activities. Subsequently, institutions around the state, including Texas State University and Lee College, added the tool to their emergency preparedness plans. In the years following Santa Fe, iWatchTexas’ website, app and hotline together fielded hundreds of tips, which may have saved lives. But the system did not prevent a gunman from walking into Robb Elementary in Uvalde, Tex., earlier this year and murdering 21 children and teachers.

Might Uvalde have thwarted its dark moment in the American gun-violence spotlight if the community had access to a better anonymous reporting system? Or is iWatchTexas the best system available?

States, colleges and schools can choose from an array of tools tipsters can use to report concerning behaviors that may foretell violence. Some, like iWatchTexas, were created by state governments. Others, such as Sandy Hook Promise’s Say Something, developed after 20 children and six adults were killed at Sandy Hook Elementary School in Newtown, Conn., hail from nonprofit organizations. Still others, such as STOPit, emerge from the corporate world. Also, some institutions develop their own online reporting tools, such as one the University of Houston has integrated into its UH Go app.

But research into the best practices and efficacy of anonymous reporting systems is a “woeful state of affairs,” according to a Health Education and Behavior study published earlier this year. That leaves colleges and schools in Texas and beyond facing an array of options, often without clear guidance, on life-or-death matters.

“We’re relying a lot on either anecdotal reports or higher-level summaries of how these systems are being used,” said Justin Heinze, assistant professor of health behavior and health education at the University of Michigan and study co-author. “It’s not necessarily that the systems aren’t in place and not getting used. We want more systematic evidence about how effective they are … [so that] we implement them well.”

Anonymous Reporting Systems Are Not Alike

A recent Dallas Morning News investigation found that iWatchTexas, which fielded 747 school-related tips in the three and a half years after Santa Fe, appeared to trail other reporting apps such as STOPit, which fielded 40,000 tips in Texas in the past five years.

The investigation also highlighted that the state’s system does not offer two-way messaging between the tipster and the individual vetting the report. Such a feature can be important, since tipsters may not know exactly what information law enforcement needs to follow up. (Both STOPit and Say Something have two-way messaging.) Also, the state does not publicly report the number of training sessions provided for its system or how many attended each event.

“It’s really important that you don’t just put a system out there and say, ‘Hey, here’s an app,’ ‘Here’s a telephone number,’ ‘Here’s an email address,’” Heinze said of the need to pair systems with training that increases students’ willingness to report and their ability to recognize concerning behaviors. “You want students to have the self-efficacy and self-confidence to actually make a tip rather than debate, ‘Well, I don’t know. I don’t want to get anybody in trouble.’”

But the Texas system is not all bad. The Texas Department of Public Safety website mentions “numerous instances” in which iWatchTexas helped thwart potential attacks, including one in September 2021 in which a teenager in San Patricio County was planning to attack his school. The department was unable to accommodate an interview before this story’s deadline.

“One advantage of iWatchTexas is that it reports directly to the fusion center,” said Bret Collier, field operations captain at the University of Houston Police Department. Texas’ eight fusion centers are so-called because they “fuse” together various police agencies. The University of Houston does not use iWatchTexas, Collier said.

A reporting system’s tip-management protocol may affect how—or whether—students use it. Some, such as Safe2Tell, which all accredited public and private schools in Colorado are required to use, involve law enforcement throughout the reporting process. For others, such as the TIPS (threat assessment, incident management and prevention services) system and STOPit, school administrators who manage the tips use their judgment about whether to involve law enforcement.

“Who’s responding to these calls, and are they best equipped to respond to them?” Heinze asked. “Many of these reporting systems were designed in the wake of school shootings, and we thought that they were going to be used for students to report just those types of things.” But students use the systems to report a range of concerns, including bullying, mental health concerns or friends in crisis, Heinze said.

Sparse Guidance on Quality and Quantity

Some institutions, lacking clarity about which reporting system is best, offer a variety. For example, Texas State University’s Report It webpage aggregates 19 different reporting mechanisms. If one student witnesses another student’s concerning behavior, the reporting student must choose from options such as a crisis support phone number for the counseling center, a “Here to Help” webform to “report concerning student behavior that may adversely impact a student or our university community,” iWatchTexas, a university police department complaint form or a university police department’s anonymous tip line. On the same page, students will also find reporting options for less critical concerns, such as a link to a form for reporting, for example, a cracked sidewalk that could cause someone to trip.

“We’ve put students, faculty or staff in positions where they have to know what category of report they’re reporting, and they typically don’t,” said Ken Pierce, vice president for information technology and chief information officer at Texas State. “We’re looking at this now and trying to figure out how we simplify this … so if you want to report something, you don’t have to know. We’ll figure out on the back end where it needs to go.”

Lee College in Texas does not have a dedicated staff for emergency preparedness. Instead, the college relies on a safety committee with representatives from across campus to make decisions about anonymous reporting tools. Committee members learned about the iWatchTexas system from a higher education coordinating board memo that promoted it as free, easy to use and a means for colleges to connect with local law enforcement, according to Amanda Summers, the community college’s executive director of human resources and lead of the safety committee.

“We started learning a little bit about it, and the committee decided, ‘Yeah this could be a good tool for us,’” Summers said. The college already had an emergency-preparedness website with its own emergency reporting form, and iWatchTexas was added to that page. Another Lee webpage provides a different form for reporting “concerning, disruptive, or threatening” behavior to the institution’s CARES team, which stands for “concern, access, refer, and educate for success.” Since the two webpages are not connected, students who find one of the pages may be unaware of the other option. Even those who find both may be unclear about the differences between the reporting options.

Summers learned of the benefits of a two-way messaging feature for anonymous reporting systems in the Dallas Morning News investigative story. Now, she may bring other options to the safety committee for consideration.

“I don’t know enough to say that one of them would replace another,” Summers said, speculating that students could be overwhelmed if they are offered too many options, though she does not know how many is too many.

In fact, all the college officials contacted for this story displayed humility regarding their understanding of what quality means for anonymous reporting systems and what quantity is desirable. Many inquired what this reporter had learned from speaking with researchers and representatives at other colleges.

A Research Gap on Best Practices

Heinze and his co-authors’ comprehensive literature search on anonymous reporting systems for school-based violence prevention captured the period from a few years before the violent attack in Columbine, Colo., until recently (from 1995 to 2020). The team found only four studies on the efficacy of these systems, only one of which was peer reviewed.

Nonetheless, colleges and schools widely implement anonymous reporting systems. Institutions vary widely in the type of systems they use, how they implement them and how they educate their communities about their availability and use.

Without evidence-based research, colleges and schools may be unclear about best practices, implementation issues, how to get buy-in among students, how to determine whether their systems are working to reduce violence and how to select from between anonymous or confidential reporting systems, according to the study.

Fundamental questions on technology-enhanced reporting systems need evidence-based answers, according to the researchers. These questions include: Do some features work better or worse in rural versus urban settings, in large versus small settings or among high school versus college populations? Do different demographic populations have different needs? Does training increase use and effectiveness? When tips are received, how should they be managed? How does anonymity or confidentiality affect reporting behavior and tip follow-up?

Confidential reporting systems collect tipsters’ identifying information but restrict the release of that information. Anonymous reporting systems do not collect and do not reveal tipsters’ identifying information, if such information is provided. Researchers hypothesize but do not know for sure that anonymous systems may encourage tipsters to report but hinder the abilities of those who follow up on tips. Similarly, confidential systems may discourage some tipsters from reporting but facilitate follow-up efforts.

“Any feature or any part of these systems that you bring up, there’s usually a line of two or three questions that are waiting to be answered,” Heinze said.

As another example, middle school, high school and college students have developmental differences.

“When we talk to our elementary or middle school audiences, we use different language even from that we would use with older adolescents,” Heinze asked. “You’re probably going to have to think about a very different sort of packaging and language that you would use for what college students deem normative and appropriate.”

To be sure, anonymous reporting tools are not the only technologies that alert authorities to behavior that may foretell violence.

“In lots of cases, we find out that something’s happening in social media before we would ever see a report,” Pierce said.

Also, old-school reporting mechanisms, such as calling 911, are still important.

“You can offer other tools, but people may not make use of those tools,” Collier said, speculating that not everyone likes to populate their phones with an abundance of apps they may never use or wade through a variety of sometimes-confusing online options. “The more basic the better, and the easier to access the better.”