Flock Safety and the Expanding Frontiers of AI-Powered Police Surveillance

Flock Safety’s latest suite of search tools empowers law enforcement agencies to track individuals through visual AI analysis, moving well beyond the company’s traditional reliance on automated license plate recognition (ALPR). By shifting from static vehicle data to dynamic, descriptive person-tracking, these tools have become the focal point of a burgeoning controversy involving dozens of police stalking cases across the United States. Through an analysis of the software’s underlying code and its operational interface, it is clear that while Flock has introduced new guardrails intended to curb abuse, these mechanisms function primarily as audit trails rather than proactive prevention measures.

The evolution of Flock’s technology represents a significant leap in the capability of local law enforcement to monitor public spaces. Previously, police utilized ALPR to identify vehicles associated with specific investigations. Today, an officer can define a geographic "geofence" within a city and task a network of cameras with an automated, continuous search for any individual fitting a specific, plain-language description—such as "person wearing scrubs" or "individual with specific tattoos." This shift has raised profound concerns among civil liberties advocates, who argue that the company is deploying AI for tasks that no current technology can perform with sufficient accuracy or neutrality.

A Timeline of Escalating Scrutiny

Over the past eighteen months, the rapid adoption of Flock’s technology has met a cold, reality-based correction from both the public and local governments. The timeline of this backlash began in earnest as reports of systemic misuse surfaced:

• Early 2023: Evidence emerges of law enforcement officers across various jurisdictions utilizing government databases—including, in some instances, nascent camera surveillance tools—to stalk ex-partners or monitor individuals based on personal interest.
• Late 2023: Investigations by outlets like WIRED and 404 Media highlight that Flock’s "FreeForm" search, which allows for natural-language queries, is being used to track people based on clothing and physical characteristics, rather than just vehicle plates.
• Spring 2024: A series of high-profile incidents triggers national attention. Notably, a Texas deputy is revealed to have searched more than 83,000 camera records to track a woman seeking an abortion, prompting a formal inquiry from members of Congress directed at Flock CEO Garrett Langley.
• Summer 2024: Illinois state officials discover that Flock had permitted federal immigration agents to access state-funded camera data, a practice determined to be in direct violation of state statutes.
• August 2024: In response to the growing crisis, Flock announces a suite of mandatory software changes, including shortened data retention periods, the requirement of case codes for every search, and the implementation of automated auditing tools.

The Anatomy of Surveillance and the "Guardrail" Fallacy

Flock’s software operates on a complex backend that remains largely opaque to the agencies using it. When an officer inputs a search term, the system’s AI evaluates the request against categories of sensitive content. The company has publicly touted these "baked-in" protections as a defense against bias and misuse.

However, a technical examination of the software reveals a bifurcated approach to content moderation. The system categorizes potential search terms into three buckets: allow, warn, and block. Terms related to race, religion, and offensive content are generally blocked. Yet, the category of "political, social, and cultural expression"—a domain protected by the First Amendment—is treated only with a "warning."

This distinction is critical. When an officer searches for terms associated with protests or cultural expression, the system allows them to proceed after checking an acknowledgment box and providing a comment. For experts like Tom Bowman, policy counsel at the Center for Democracy and Technology, this is a glaring vulnerability. "Political and cultural expression is one of the most protective areas under the First Amendment," Bowman notes, "and it seems like that actually has the least concern from a policing perspective."

Furthermore, the "warning" system functions as a deterrent only for officers who are not actively seeking to bypass the rules. According to Deepak Kumar, an assistant professor of computer science at UC San Diego, such logs are useful for post-hoc administrative review but fail to provide the real-time prevention necessary to stop an officer intent on misconduct. "It functions less as a deterrent and more as a record," Kumar observes.

Data Misuse and the "Bad Apple" Problem

The systemic nature of the abuse is underscored by recent data from The Washington Post, which identified at least 50 law enforcement officers in the U.S. charged or accused of misusing license plate readers. In nearly half of these instances, the target was an ex-partner or a romantic interest.

The documented cases of misconduct are pervasive. In one instance, a Milwaukee officer searched for a woman he was dating 124 times, labeling each search as an "investigation." In another tragic case, a Georgia police chief who queried his ex-girlfriend and her daughter 600 times ultimately faced criminal charges and took his own life. These incidents suggest that the problem is not merely a technical oversight but a cultural issue within departments that lack the resources or the will to audit the massive amounts of data flowing through their systems.

While Flock maintains that it provides the tools for oversight, the responsibility for enforcement remains with the local agencies. "Vendors can recommend safeguards," says Don De Lucca, a former police chief, "but departments have to set the rules." The challenge is that these departments are often overwhelmed by the sheer volume of data, leading to a reliance on the software’s own "Audit Assistance" features—which, as of recently, were set to "off" by default.

Broader Implications for Civil Liberties

The integration of AI into public surveillance networks has transformed the nature of privacy in America. When a system can track individuals across thousands of cameras based on vague, subjective descriptions, the distinction between a targeted criminal investigation and general, dragnet surveillance collapses.

Critics argue that the current focus on "bad-apple" officers ignores the structural danger of the technology itself. Jay Stanley, a senior policy analyst at the ACLU, characterizes the moderation system as a "speck on the back of a giant surveillance machine." By allowing police to probe databases for "your-imagination-is-the-limit fishing expeditions," Flock has inadvertently created a tool that, even when functioning "correctly," inherently challenges the democratic expectation of anonymity in public spaces.

The fundamental tension lies in the fact that no automated moderation system can perfectly anticipate human intent. Whether an officer is searching for a suspect in a violent crime or an ex-partner in a personal dispute, the machine treats the request based on the technical parameters set by the developer. Because Flock’s AI models operate on company servers, external oversight is impossible. Neither the public nor independent researchers can verify the accuracy of the model or the criteria it uses to flag or ignore specific queries.

The Path Forward: Accountability vs. Utility

As 2024 draws to a close, Flock’s pledge to make its auditing tools mandatory will serve as a test case for whether technical constraints can mitigate the social costs of mass surveillance. If successful, the changes could provide a template for other technology firms operating in the high-stakes world of public safety.

However, the skepticism remains high. The history of law enforcement technology—from the 2016 AP investigation into database abuse to the recent reports regarding ICE and CBP—suggests that without rigorous, independent, and transparent oversight, surveillance tools will continue to be repurposed for personal agendas.

For now, the balance between public safety and individual privacy remains tilted in favor of the former. As cities grapple with whether to continue their contracts with Flock, they face a fundamental question: Is the utility of AI-powered surveillance worth the risk of providing every officer with the potential to turn their badge into a digital stalking tool? Until the industry can prove that its "guardrails" are more than just a record of failure, that question will likely remain at the center of the national conversation on police technology.