As cities around the U.S. struggle to respond to gun violence in neighborhoods where residents may not trust law enforcement, dozens of police departments are turning to a gunshot-detection system called ShotSpotter. Yet, the true measures of its effectiveness at reducing crime may lie in research that no one has conducted and data few people can access.

ShotSpotter is the brand name for a service that uses acoustic sensors and software to detect the sound of a gunshot and its origin. It’s a proprietary system sold by SST Inc., a Silicon Valley-based firm founded in 1994. The company promotes the system as a way to alert police to gunshots that otherwise would go unreported.

About 90 cities in the U.S. and around Latin America have installed ShotSpotter in their efforts to track and reduce gun violence. The firm’s CEO has his eyes on a much larger market, though, and wants the company to get into as many as 1,500 U.S. cities. SST isn’t the only company that sells acoustic gunshot-detection technology, but it has had the most success in marketing it for local law-enforcement use.

In Wisconsin, Milwaukee first installed ShotSpotter in 2010 and expanded it in 2014. The city’s system shows up briefly in the new Wisconsin Public Television documentary, Too Many Candles: Milwaukee Gun Violence, in which officers explain how they respond to its alerts in high-crime neighborhoods. Additionally, Beloit installed ShotSpotter in 2009, but its system went dormant in 2012 after running into technical problems and budget limitations.

ShotSpotter operates on the principle of triangulation. Police departments install audio sensors atop buildings in its service area. When a gun is fired in an audible vicinity, multiple sensors pick up on the shot’s sound and transmit data to the SST Incident Review Center in California, where the company’s proprietary software analyzes the data to triangulate where the noise originated. SST sends its determination back to the client city’s local police dispatcher, who, in turn, sends officers to the scene. Ideally, the whole process — from the gunshot to the arrival of officers — takes just a few minutes.

Wisconsin Public Television

A Milwaukee police officer views the ShotSpotter data display for the city.

Most clients, including the Milwaukee Police Department, do not monitor entire cities with ShotSpotter, but set it up to monitor specific areas that suffer higher rates of gunfire.

The ShotSpotter system has its limitations — it only picks up on outdoor gunshots, and other loud, abrupt noises have yielded false alarms — but ideally it can help police officers reach the scene of a shooting in mere minutes, whether or not anyone reports it to authorities. This goal is one of ShotSpotter’s main value propositions.

In Milwaukee, the ShotSpotter service has broad support. Democratic and Republican lawmakers from across the state worked together in 2014 to provide money for the system, although the Legislature earlier had reduced funding for it. In 2014, the Legislature added $175,000 to $140,000 from the Milwaukee Police Department and $175,000 from Milwaukee County; ShotSpotter continues to be funded with a mix of state, county and city monies.

The system initially covered about 2 square miles with the city’s highest incidences of gun violence. After the first year, SST said its data showed that residents were reporting 14 percent of recorded gunshots to police. Officials including Milwaukee police Chief Ed Flynnand legislatorshave continued to cite that 14 percent figure. SST in turn highlights a Milwaukee TV news report referencing that percentage, though without the crucial context that it reflects a specific area with high rates of gun violence and where police-community relations are likely to be poor.

Since being established, ShotSpotter’s coverage within Milwaukee has expandedto about 11 square miles in areas of the city’s north and south sides. Milwaukee Police Department spokesperson Sgt. Tim Gauerke said in a recent email conversation that he did not know whether the percentage of unreported gunfire in these areas of Milwaukee has changed since 2011.

At a general level, the effectiveness of ShotSpotter at reducing crime is difficult to measure, in part because reasons for decreases or increases in violent crime cannot be reduced to a single factor. Law-enforcement experts and social scientists have had little opportunity to independently evaluate the service, in part because SST’s own business practices impede evaluation of its product.

Evaluating ShotSpotter is difficult

SST CEO Ralph Clark agreed gun violence is a complex problem.

In working with client cities where gun violence has decreased, “we have to be respectful to not make the case that it’s only because of ShotSpotter that we’ve seen these reductions,” he said in an interview. SST pulls in a lot of data from cities that use ShotSpotter — even producing slick annual reports about gun violence — and argues the system helps reduce crime rates by assisting officers to respond more quickly to scenes of gunshots.

The company guarantees police departments in its contracts that ShotSpotter will detect 80 percent of outdoor gunshots. “It really gets down to operational effectiveness,” Clark said. “We consider ourselves to be a part of hopefully a comprehensive strategy.”

Anecdotal evidence shows police respond to ShotSpotter alerts and capture armed suspects at the scene. Gauerke said an April 21, 2016, arrest of suspectsresulted from a ShotSpotter alert. Despite other examplesof such outcomes, the frequency of Shotspotter-related arrests or whether police keep firm data on such correlations is not clear.

“I am not certain if we keep that record,” Gauerke said when asked whether Milwaukee Police Department tallies arrests resulting from ShotSpotter alerts. “If we do, it is not data I have readily available.”

Reveal, a podcast produced by the Center for Investigative Reporting, dug into the efficacy of ShotSpotter on a national level in an April 25, 2016, episode. “Unfortunately, in practice, police often come up empty-handed because people rarely stick around after firing a gun,” Reveal noted. For the podcast, that’s evidence ShotSpotter plays at best a minor role in aiding criminal convictions and reducing violent crime rates.

Clark disagreed with this assessment, saying the goal of ShotSpotter alerts is not about simply yielding immediate arrests of suspects with guns in their hands. “I think they didn’t ask the right question,” he said of Reveal‘s reporting.

Rather, Clark said, an appropriate measurement of ShotSpotter should be longer-term in nature: A positive evaluation should include reductions in crime, more arrests that flow indirectly from ShotSpotter alerts and improved community relations as more officers tracking the alerts show up in high-crime areas and have more positive interactions with residents. Clark said ShotSpotter admittedly can’t claim all the credit for these trends, but the service can still strengthen a police department’s broader strategy to address gun violence.

That said, SST does tout the immediate alert-to-arrest scenario when it can. The company has re-posted news items about such incidents on its website. One press release from October 2014 cited such arrests as a benefit, offering quotes from police officials in several cities using the system.

“In 2013, we started tracking the number of arrests and guns we’ve recovered due solely to ShotSpotter alerts — it’s making a huge difference,” Springfield, MassachusettsPolice Superintendent John Barbieri said in the release. And Camden County, N.J., Police Chief Scott Thomson offered this comment: “We had our first gun arrest within two hours of implementation of this technology, and in the second incident, we were able to apprehend a suspect test firing a high powered rifle.”

The long-tailargument Clark makes is compelling — crime and policing are complex, with broader changes involving numerous factors that are difficult to track over time — and he has advocated it consistently during his time as SST’s CEO. For instance, when officials in Charlotte, North Carolina, stopped using ShotSpotter and criticized it as ineffective in helping them solve crime, Clark responded with skepticism to the Charlotte Observer.

“What I hear in (the Charlotte-Mecklenburg Police Department’s) response is a very strong focus on homicides and catching people in the act of homicides,” he said. “I think that is a very limited way of thinking about the nature of gun violence. Really what you want to do is prevent (gun violence), not convict after someone has killed somebody.”

Clark explained that when a city first installs ShotSpotter, SST dispatches trainers to spend a week teaching its law enforcement officials how to use the system and talking about how to incorporate it into a broader policing strategy. These tactics include finding ways police can combine ShotSpotter data with other information like intelligence about gangs. Another company focus is community relations. “Probably the most powerful thing (police) can do is having a community observe them responding to these shootings,” Clark said. (On its website, SST highlights a 2014 WisconsinEye interview with Milwaukee’s Chief Flynn, who advocates each of these goals.)

Clark would eventually like SST trainers to start going on ride-alongs with officers in ShotSpotter cities. He emphasized that police departments shouldn’t just treat the system as a plug-and-play affair that solves problems single-handedly. “The highest and best use of this technology is as a service delivery and enhancement tool for law enforcement,” he said.

Based on its public statements, though, SST is trying to have it both ways on the short-term alert-to-arrest scenario, promoting it as an immediate benefit while arguing that these incidents shouldn’t be used to measure ShotSpotter’s overall effectiveness.

Transparency versus viability

Another reason that gauging ShotSpotter is difficult is that very few people have done independent research on the system’s efficacy.

Jennifer Doleac, an economist at the University of Virginia and the Brookings Institution, and one of very few ShotSpotter experts not associated with SST or a client police agency, points out that there’s an obvious statistical obstacle in comparing ShotSpotter-monitored areas with “control” areas. Such an analysis would theoretically require some cities or areas to install the system, but not dispatch officers based on its alerts — understandably a hard sell for cash-strapped police departments and local governments, as well as being unethical.

But Doleac has communicated with SST about the need for such a comparison. “They’re open to the idea but it hasn’t been at the top of their priority list,” she said.

Understanding ShotSpotter’s performance requires data from the field, but intellectual property rights block access. SST has carved out a gray area in the public records landscape and requires researchers to pay for the raw data.

A government agency may use ShotSpotter, but SST owns the data its systems generate, and therefore the company maintains the data aren’t public record in the same way as a police report or a law enforcement agency’s internally generated statistics are considered. Doleac said SST’s ownership of the data is written into the contracts it signs with law enforcement.

SST owns ShotSpotter data for Milwaukee, Gauerke confirmed, though he declined to release a copy of the city’s contract with the company, saying it contains proprietary information. This secrecy might come as a surprise to Milwaukee City Clerk Jim Owczarski, who did not comment specifically on the city’s contract with SST. Rather, he said he generally tries to avoid such provisions when dealing with technology contractors.

“Every piece of software I buy, I want to make sure I retain the records,” Owczarski said.

The status of ShotSpotter data might surprise a lot of other people around the country, Doleac said.

“The police departments that sign these contracts aren’t necessarily looking for this when they’re looking through (them),” she said. “I think the very first thing is to make sure the city owns the data that ShotSpotter is producing, so that they can share those data with objective researchers who can use them to study policy effects in the community.”

But that stance doesn’t align with SST’s business strategy, and CEO Clark isn’t shy about saying so. He encourages client agencies to release public information that uses ShotSpotter data in bold strokes — identifying broader trends in gun-violence rates or specific problem areas.

As for the raw data, like specific time and location details of individual gunshots the system detects, SST wants to keep a tight hold on it and develop a revenue stream by selling this information to academic researchers and public agencies like the FBI and U.S. Bureau of Alcohol, Tobacco, Firearms and Explosives.

“If the data begins to leak out to these organizations en masse and in an electronic form, it devalues the big data that we could otherwise monetize with the FBI and ATF,” Clark said.

SST and police agencies that use ShotSpotter have said they keep a tight lid on the raw data to protect proprietary information about the company’s software and to prevent people from figuring out the locations of individual sensors, which often are located on top of private buildings with the consent of property owners. Milwaukee has kept the confidentiality of those property owners in mind when releasing some ShotSpotter information, said Assistant City Attorney Peter Block.

“We’ve given out data in terms of the number of shots on a certain day or a week in a general vicinity,” he said.

Security concerns are distinct from SST’s interest in making money from the data, but the company’s policies are “much more the latter issue,” Clark noted.

Local police agencies around the country do share internally generated crime data with the FBI, but if a local police department sent its raw ShotSpotter data to the agency, Clark would consider that free-ridership and a breach of contract. He uses the subscription legal-research service LexisNexis as a metaphor.

“If you were to license LexisNexis, you can’t have the city license LexisNexis and then have an agency in D.C. say, ‘Well, we don’t want to pay for LexisNexis,'” Clark said.

Owning ShotSpotter data also makes a powerful marketing tool. A visit to the ShotSpotter bloggives the sense that SST wants to come off as a data-driven authority on gun violence as much as a company selling a service. The company produces in-house white papers and an annual “National Gunfire Index” (for example, the 2015 edition) based on data its systems gather around the nation. SST even offers clients a consulting serviceto help agencies track down federal and state grant funding. This approach and the company’s resources might reassure police departments struggling with violent crime and shrinking budgets.

While independent investigations of ShotSpotter have been conducted, including the recent Reveal podcast, considerable public discussion revolves around news reports on alert-to-arrests, supportive comments from law enforcement officials and SST’s own considerable body of promotional materials. ShotSpotter dominates not only this part of the lawenforcement technology market but is also subtly playing a growing role in public conversations about reducing gun violence.

Doleac sees a missed opportunity in the status quo. In a February 2016 essay for the Brookings Institution, she wrote that greater ShotSpotter data transparency would help the agencies that use it build greater trust with the communities they serve. Doleac agrees with Clark that police following up on ShotSpotter alerts can have a positive impact by getting out of their patrol cars, talking with residents in high-crime areas, showing real concern and building trust. But she also wrote that releasing detailed ShotSpotter data will “improve research and empower citizens to make their communities safer.”

“Some cities will give you aggregate statistics — total gunshots per month, for instance. That is not enough,” Doleac wrote. “They should release the time and location of each incident, the way they do for reported crime. If those data aren’t public, insist that they amend their contract with ShotSpotter so that your city owns those data in the future.”

Despite understanding SST’s need to generate revenue, Doleac doesn’t agree that selling data will be good long-term business for the company. “I think it’s a poor business strategy, frankly, for them to not release this data,” she said. Still, in the course of her research, including visiting SST’s headquarters, Doleac has grown convinced that SST’s heart is in the right place. She added that the company must strikeachallenging balance between business needs and its role in a complex public issue.

Editor’s note: Wisconsin Public Radio airs “Reveal” on a weekly basis. This report has been updated with clarification on Doleac’s comments regarding SST Inc.