According to leaked documents, the Obama administration's use of metadata to identify and target terrorists in Pakistan would misidentify over 99,000 innocent people.
The SKYNET program, named after the antagonist in the Terminator movie series, is used to examine the cellular network metadata of over 55 million people in Pakistan and flag suspicious patterns to target for “counter-terrorism” operations like kidnapping, interrogation or drone assassination.
The drone assassination program, begun in 2002 under Republican president George W. Bush, has been greatly expanded under Democratic president Barack Obama. The Bureau of Investigative Journalism conservatively estimates the death toll from these targeted killings at over 3,200.
An unknown portion of these strikes were based on automated programs like SKYNET. According to the former director of the National Security Administration (NSA) as well as the CIA, retired General Michael Hayden, “We kill people based on metadata.”
Documents reviewing the metadata targeting program were leaked by NSA whistleblower Edward Snowden and then published by The Intercept last year. Patrick Ball, a director of the Human Rights Data Analysis Group, told Ars Technica in an interview that the methods used by the NSA in SKYNET are “ridiculously optimistic” and “completely bullshit.”
In order to process the records for millions of cell phone users the NSA uses a process called “machine learning,” where the computer is fed a set of “known terrorists” and then looks for new targets by how closely an individual's behavior matches the known set. It is essentially the same method used by corporations to target Internet advertising.
Overall the NSA examines 80 different variables such as travel patterns, co-travelers, SIM card swapping, phone contacts and turning off a phone which the NSA sees as an attempt to evade surveillance. The assumption of the NSA is that the behavior of “terrorists” significantly differs from innocent people in consistent ways.
The method the NSA uses to measure success however is fundamentally unsound. To test the program, the NSA began by taking a random sample of 100,000 people and adding seven “known terrorists.” The NSA then gave the program six of the individuals marked as terrorists to analyze and tasked it with finding the seventh. The exceedingly small seed of “known terrorists,” makes the results of the test unreliable.
This is combined with the fact that the “terrorists” tested were handpicked as belonging to the same network, while the ordinary people were randomly selected, removing a significant portion of their social network from the test. Applying the same program to the full database would result in a significant increase in false positives.
In the leaked documents, when the NSA applied their method to the full data set of 55 million, only one of the “known terrorists” was in the top 100 suspects, and only five of them were in the top 500.
According to the NSA slides, they expect to falsely categorize 0.18 percent of the sample as terrorists. This may sound like an exceedingly small number, but when applied to the data of 55 million people, it comes out to 99,000 innocent people wrongly accused. This is much larger than the Haqqani Network, one of the largest groups targeted by the US which is estimated to have anywhere from a 4,000 to 12,000 members, significantly smaller than the number of expected false positives.
The absurdity of the NSA's method is exposed in a slide that presents Al-Jazeera 's longstanding Islamabad bureau chief, Ahmed Zaidan, as the highest rated target. The top secret slide lists him, in sharp contradiction, as a member of both al-Qaeda and the Muslim Brotherhood, two opposed organizations. As a journalist, Zaidan has traveled extensively through Pakistan speaking to all sides of various conflicts.
Disturbingly, Zaidan appears to have already been assigned a watch list number as an al-Qaeda member before being flagged by SKYNET, demonstrating significant inaccuracies in the NSA's set of “known terrorists.”
It is unclear what steps, if any, exist between SKYNET flagging someone as a terrorist and someone being targeted for drone assassination. What is clear from Hayden's statement is that some people are killed based on metadata and the use of programs like SKYNET.
Within Pakistan where SKYNET operates, the drone assassination program is run with fewer restrictions than in other countries. In 2013, Obama ostensibly tightened the guidelines for targeted killings requiring the CIA to have evidence that the target was an “imminent threat” and that they have “near-certainty” that no civilians would be hit. The drone program in Pakistan has been operating under a waiver that explicitly removes those guidelines.
Overall the drone assassination program is thoroughly brutal and inaccurate. Other leaked documents showed that out of 235 people killed by Operation Haymaker in Afghanistan between 2012 and 2013, only 35 were targets; the rest were bystanders.
Obama has relied heavily on assassination during his presidency in order to avoid the political difficulties of actually trying people accused of terrorism. Many of those detained for terrorism at Guantanamo are being held on circumstantial or inadmissible evidence gathered from CIA torture or illegal NSA spying.
According to the former head of the Defense Intelligence Agency, Lt. Gen. Michael Flynn: “The drone campaign right now is really only about killing. When you hear the phrase 'capture/kill' capture is actually a misnomer.”
Increasingly, the Obama administration is relying on metadata and signals intelligence for identifying “terrorists” and selecting drone targets. In Somalia and Yemen, over half of the intelligence used to select targets is based on electronic signals like phone calls that do not directly identify the target.
The Department of Defense announced last August that it was going to expand its drone program by 50 percent over the next four years and Italy has recently agreed to allow US drones to operate out of a base in Sicily to target Libya. Enormous civilian casualties can be expected from the growth of these mechanized killing operations.