– Thank you so much everybody, for joining us tonight to this Science and the Public, for the Holtz Center. First, my name is Samer Alatout, and I’m the Director of the Holtz Center for Science and Technology Studies. Now I will introduce– I’m not going to introduce the speakers, but I will introduce the introducer of the speakers. [laughing] My colleague, Cecelia Klingele, and who is a professor in the law school at UW. After receiving her JD from the University of Wisconsin Law School, in 2005, Cecelia Klingele has a great kind of profile, right? Served as a law clerk to Chief Judge Barbara Crabb of the United States District Court, for the Western District of Wisconsin, Judge Susan Black, of the United States Court of Appeals for the Eleventh Circuit, and Associate Justice, and this was impressive, really, to me, although every, all the judges are, you know, with respect to all of them, but Associate Justice John Paul Stevens of the US Supreme Court. She returned to the University of Wisconsin in 2009 as a visiting assistant professor, and has been on the permanent faculty, a permanent faculty member in 2011. Professor Klingele’s academic research focuses on criminal justice administration, with an emphasis on community supervision of those on conditional release. She served as Associate Reporter for the American Law Institute’s Model Panel Code, sentencing revision, sentencing revision, External Co-Director of the University of Minnesota Robina Institute’s Sentencing Law and Policy Program. And past co-chair of the Academic Committee of the American Bar Association’s Criminal Justice Section.

So with that, please welcome Cecelia and the rest of the panel for, for this, and enjoy the talks. [applauding] – Thank you, Samer, and you can only guess with that nice of an intro for me, how fantastic our actual panelists are going to be tonight. It is my pleasure to introduce them, and I will actually do that first, and then make a few opening remarks about the theme of tonight’s presentation. So first, Simon Cole joins us from the University of California at Irvine, and where he’s professor of Criminology Law and Society, and serves as Director of the Newkirk Center for Science and Society. Professor Cole specializes in historical and sociological study of the interaction between science, technology, law, and criminal justice. He is the author of Suspect Identities: A History of Fingerprinting and Criminal Identification, which was awarded the 2003 Rachel Carson Prize for the Society, by the Society for Social Science, Social Studies of Science. He is co-author of Truth Machine: The Contentious History of DNA Fingerprinting, as well, and has spoken widely on the subject of fingerprinting, scientific evidence, and science in the law. He’s also consulted as an expert in the field. He’s written for many general interest publications, including the New York Times, and the Wall Street Journal, and currently focuses on the sociology of forensic science and the development of criminal identification databases and biometric technologies.

His teaching interests focus on forensic science and society, surveillance and society, miscarriages of justice, and the death penalty. Our second panelist is Margaret Hu, who is Associate Professor of Law at Washington and Lee University School of Law. Her research interests include the intersection of immigration policy, national security, cyber surveillance, and civil rights. Previously, she served as senior policy advisor for the White House Initiative on Asian Americans and Pacific Islanders, and also served as special policy counsel in the Office of Special Counsel for Immigration-Related Unfair Employment Practices, Civil Rights Division, in the US Department of Justice. As Special Policy Counsel, she managed a team of attorneys and investigators in the enforcement of anti-discrimination provisions of the Immigration and Nationality Act, and was responsible for federal immigration policy review and for coordination. She is also the author of a forthcoming book, The Big Data Constitution: Constitutional Reform in the Cybersurveillance State. So we have an action-packed evening in front of us. I am honored to kick it off, though I will keep my remarks brief, so that we get more time from our esteemed guests. It was a little ironic, I thought, that I was asked tonight, to speak on the subject of Big Data in criminal justice, and the ways in which it might affect racial disparities.

What I find ironic about it, is that most of the time, when we talk about data in criminal justice, we’re not lamenting that there’s too much of it, but that there’s not enough. For those who work in the field, you already know what I mean, and for those outside of it, let me give you some sense. The collection of data in criminal justice in the United States is complicated by many factors. Among them are the fragmentation of criminal justice agencies. Every police agency, every county clerk of courts office, every district attorney’s office, every iteration of public defenders, and they exist in many different iterations around the country, every correctional agency and jail maintains separate databases in this country. There’s no regularized norm around what data is collected, how it’s reported, whether it’s audited, how it’s reviewed or stored, backed up, maintained, made accessible. Any researcher who has filed freedom of information request acts has often been greeted with the response from criminal justice officials, “Well, I don’t think we can get you that. “We’d have to go through every single file “and pull it out, cause it’s written by hand, somewhere! “And probably sorted in archive if it’s been maintained.” That fragmentation of data, and the lack of adequate controls, usually leads us, again to lament the inability to gather aggregate data about the functioning of our criminal justice agencies, and the way they are either positively or negatively affecting our communities, as measured by any number of different metrics. While we have some ways of gathering some aggregate statistics, mostly through the Department of Justice, and the FBI’s Crime Statistics, there are many flaws to those, as well, and good reason to question the reliability of at least some of that data.

So, again it’s funny that we’re here tonight, to talk about the opposite problem, in many ways. And that’s the problem created by drawing on Big Data, the compilation of large amounts of information, gathered about individuals not necessarily aggregated across police systems. More commonly aggregated by private data collection agencies, and then sold to law enforcement or other interested parties, or generated based on criminal records information that we might have in state court databases. So if criminal justice system actors are so reticent to collect data, and be held accountable for it, what explains the draw, and I would say, the increasing trend to rely on it, when it comes to things like risk prediction algorithms, or hot spot, or predictive policing algorithms? I think that’s best explained by the configuration of the criminal justice system, itself, and the laws that govern it. Again, for those of you who are familiar with the system, you’re aware that although there is of course law, constitutional, statutory, administrative, that governs the behavior of system actors, from police, through prosecutors, through judges and correctional agents. In fact, many of the day-to-day decisions made by those actors are not dictated by any statute, but are rather governed by a principle we call discretion. That is the legally-authorized authority of system actors to select between multiple, equally-permissible legal options. For the beat officer, that means that when you see the child spray painting the wall, you can either tell him to go home, or you can give him a ticket. Or you could arrest him, you could refer him for charges, or not.

You could take him to his parents, right? Any number of choices, all of them equally legally permissible. The same is true, not only for arrest decisions, but for the decisions about whom to surveil. About what charges to leverage against an individual defendant. About whether to set bail, and if so, what the amount to be, or the conditions of release. What sentence to impose for an individual who’s been convicted of a crime, and what kind of supervision or custody to give to those who are already being punished for a criminal sentence. Those are hard decisions. They don’t have clear, right or wrong answers. They require the balancing of many difficult and complicated moral, sociological, and other factors, outside of the law, that often leaves police officers and sentencing judges awake at night, wondering if they’ve done the right thing. The idea that we could outsource those really hard decisions to a math problem is super appealing.

Math sounds so objective, and fair, and clear-cut, and the desire to assuage our guilty consciences, or at least our anxiety about getting the right answer in a difficult and complex situation, is such that I think data feels concrete, and safe, and reassuring to many in the criminal justice system. Now I’m oversimplifying. There are also those who are inherently skeptical. Particularly in criminal justice, and particularly when we talk about the law, and the legal process. Because, as all of us know, if lawyers and judges were good at math, we’d be doctors. [chuckling] That’s for you Jerry. [audience chuckling] Of course, there are some in law with mathematical talents that far exceed my own, but the reality is there is something about feeling safe that math is certain, and not quite understanding the magic mix that goes into the data-crunching behind many of these numbers that makes it particularly appealing in a complex and difficult enterprise like the administration of criminal justice. But there are, of course, dangers of over-relying on data. And I do mean over-relying, because, certainly, there are many positive things to be found in numbers, that check our intuitive gut sense of what’s happening on the ground, or who is being affected in what ways by the decisions that we make.

Data plays an important role and in many ways, we need to get better about collecting it. But the ways we use it matter, and there are several dangers that I will throw out, and then turn the stage over to those who can unpack it for us much better than I can. The first is a failure to often recognize that the data themselves are often flawed. When we rely on information in systems, whether that information is used as it is first presented, or whether it’s first put through a complex equation to generate a new number or prediction for us, the quality of that data matters. And the reality is that turning something into a number doesn’t change inequities in the collection of that data. Or problems in the quality that exists. Take for example, predictions about criminal recidivism. Those are ones with which I’m particularly familiar, because they often affect sentencing decisions and correctional supervision. In those cases, we strive, with algorithms, to generate aggregate predictions of about what individuals with similar backgrounds to a particular defendant we believe are likely to do in the future when it comes to re-offense.

But, there are a lot of problems with that. First of all, and most simplistically, those data don’t tell us anything about actual human behavior. All they tell us about is human behavior that’s gone awry, and been detected by law enforcement. The reality is that the universe of criminal behavior is much, much, much, much broader, than that of detected criminal behavior. And as a result, depending on where you live, and how old you are, and whether you’re a guy or a girl, you are more, or less likely, to be detected committing crimes. And if we’re relying on information about crime detection, that’ll tell us about the likelihood perhaps, of someone like you being detected again in the future, but it doesn’t tell us anything about the actual behavior of you or the general population. Or very little. Second, I think there’s a danger of misunderstanding the limitations of the data themselves. Again, in the risk prediction context, often, risk of future offense in the sentencing arena, is predicted out as whether a person has a low, a medium, or a high risk of recidivism.

And usually when I poll lawyers and judges, they tell me they can’t tell me what the exact percentage is, that means low, medium, or high, but they’re sure there is one. Not true. In fact, most of these data are not absolute. They’re comparative. They’re comparing populations against each other for the frequency of predicted re-offense. In other words, if you live in a really, really dangerous place, it may be [laughs] low-risk people are actually higher risk than they might be in a different population. In so far then, as what we are trying to do in the criminal justice system, is not only, protect people from future risk of harm, but maybe more importantly, hold people accountable for actual decisions that they have made in the past to harm others. These data may not tell us all that we think is important about either moral culpability, propensity to offend, or to change, character, the possibility of future growth. And if those are the things that matter to us at sentencing, then, bad news, guys.

We can’t outsource it. And so I hope today’s conversation will help us, first of all, better understand all those numbers, cause I don’t understand them, and maybe some of you are with me on that. But also better understand what the limitations and the possibilities of this information, ways in which it may be having an inequitable effect on some members of our community, and ways that hopefully, we can use it to make our system better, and more fair. So with that I’m happy to turn it over to Simon and Margaret. [applauding] – Thank you so much for the wonderful privilege to join you this evening, for this very important conversation, and very grateful to Lynn and Samer at the Holtz Center, and to Cecelia for that very generous introduction, and for contextualizing these issues in such a brilliant way, thank you so much. So what I wanted to do today is focus on Big Data in discrimination. And, really, feel very fortunate to be here because I just published an article with the Wisconsin Law Review called Crimmigration-Counterterrorism, where I talk about the conflation of crime, immigration, and counter-terrorism, or national security rationales, through programs such as the Muslim Ban, and extreme vetting. I also have another article that I just published on the Muslim Ban and extreme vetting called Algorithmic Jim Crow, that was published in Fordham Law Review. So I wanted to start today, by talking about extreme vetting as a way to help us wrap our minds around modern governance that involves Big Data, and the rationales that support it.

Then I wanna go into a little bit more detail about mass surveillance, justifications, Big Data intelligence gathering methods, Small Data surveillance that we used to have in the Small Data world, versus the Big Data, cyber-surveillance schools that we now have at our disposal. And then, if there’s time, get to the Snowden Disclosures. So, extreme vetting, what is it? Extreme vetting is a way to understand the modern landscape that we have, with the ubiquity of social media, and online information. So back in December 2015, then-presidential candidate Donald Trump published a Statement on Preventing Muslim Immigration on his campaign website. The statement called for a total and complete shutdown of Muslims entering the United States until our country’s representatives can figure out what is going on. Then, shortly before the election, he announced a proposal for what he called extreme vetting of immigrants and refugees. He later explained, the Muslim ban is something that in some form has morphed into an extreme vetting from certain areas in the world. So the Muslim ban or what is referred to as the Travel Ban, and extreme vetting, should be understood as one of the same, so what is it? So you have under the former administration, the former Director of the United States Citizenship and Immigration Services Office of the Department of Homeland Security, explained that they already started some form of extreme vetting during the Obama administration, that prospective refugees from Syria and Iraq, since 2015, had their Facebook, Twitter, and Instagram accounts checked, and then, the then Department of Homeland Security Secretary, now Chief of Staff, John Kelly, explained, in Congressional testimony, that extreme vetting would be an accounting of what websites that those refugee applicants would be visiting, what telephone contact information they had in their phones, to see who they were talking to, social media information, including passwords. So, further information about extreme vetting was revealed in a media report that the Department of Homeland Security held what they called an Industry Day in 2017, in Arlington, Virginia.

It circulated a document, at that time, called the Extreme Vetting Initiatives, and a host of industry representatives were there. The document stated that, right now, it is difficult for the government to assess threat because the data is collected, that is fragmented, and it’s very time-consuming and labor-intensive to make sense of it. And so they were inviting solicitations of proposals to help do this type of vetting tool that would automate, centralize, and streamline vetting procedures, while simultaneously making determinations of who would be considered a security risk. The system was purportedly supposed to then, predict the probability of an individual becoming a positive member of society, as well as whether or not they had criminalistic or terroristic tendencies. The attendees included for example, IBM, Booz Allen Hamilton, LexisNexis, and other companies, and in the solicitation, it stated that the contractor shall analyze and apply techniques to exploit publicly available information such as media, blogs, public hearings, conferences, academic websites, social media websites such as Twitter, Facebook, LinkedIn, radio, television, press, geospatial sources, Internet sites, specialized publications with intent to extract pertinent information regarding targets, including criminals, refugees, non-immigrant violators, and targeted national security threats and their location. So, basically, what they said was all publicly available information of all persons to be turned into some type of automated tool, some type of algorithm, to assess risk and predict terrorism, so during a follow-up Q and A session, one of the attendees asked whether or not this was basically legal. And it was an anonymous question, and the Department of Homeland Security responded that there’s a prediction that in the future Congressional legislation might address this, but, basically, they’ll continue to do it until they’re told that it’s not legal. And that’s basically where we are. In the legal regime, in our regulatory structure for this type of online collection, or using social media, or using sort of internet digital footprints that we leave behind, that’s publicly available, all of it’s considered fair game.

So, this leads us to what is Big Data cyber-surveillance, and what are some of the justifications for it? So in a Big Data cyber-surveillance world that we find ourselves in, right now, the analytical method is you start with the data. And what that means, for the government, is that the data becomes suspicious. It’s not necessarily that people are suspicious anymore. And this also has led to pre-crime ambitions. So just as the Department of Homeland Security invited these corporations to come up with tools to assess risk and predict terrorism and crime before it occurs, that has led to basically, a minority report type of world. And the way that this was explained by the CIA’s Chief Technology Officer was that, since you cannot connect dots that you don’t have, it drives us into mode of fundamentally trying to collect everything, and hang onto it forever. Forever being in quotes, of course. It is nearly within our grasp, he explained, to compute on all human-generated information. So, he said, “It’s all about the data, stupid.” Revolutionize Big Data exploitation, acquire, federate, secure, and exploit.

Grow the haystack, and magnify the needle, so oftentimes you hear in the intelligence community, that you need to have the biggest haystack possible in order to find the needle. But what some experts have explained is this is a put the haystack before the needle approach to governance and intelligence gathering. And what this does is it pre-supposes that there is a needle in the haystack before you even know that there’s a needle. And so, in a Small Data world, you started with the needle. You started with a person who was a criminal, a suspect, a crime, and then you went vertical into the information gathering. Because, the resource limitations and the technological limitations required a vertical, downward drilling of data to support that original question. But in the Big Data world, you actually flip everything around, and you start with the data. You answer the question after the fact. You look at the data, and then you say, “From this data, can I find a suspect? “Can I find a crime? Can I find a terroristic risk?” And so everything has been flipped upside down, and this has led to basically virtual suspects or we can have a digital avatar of ourselves that becomes the representation of the targeting of the government action.

So Edward Snowden, when he came forward with the Snowden Disclosure said, “Does this method work, or are we just drowning in hay?” Are we building more and more hay, and do we have larger and larger haystacks, that doesn’t necessarily lead us to any conclusive resolution. And also another risk of this, as explained to another intelligence source, is that everyone now is a target. So even though extreme vetting, for example, is presented on just trying to find a target who is potentially a suspect, a criminal, or a terrorist from these refugee vetting procedures, you saw from that extreme vetting day that you Collect it All. It is a Collect it All method, it doesn’t matter what your citizenship is, it doesn’t matter your immigration status. The method requires collecting it all, and everyone who has any digital communications becomes a target. And what this also means, is that our digital devices are the ancillary representatives of us, ourselves, and so, Snowden explained that part of the key inquiry of the intelligence community is whether the phone is suspicious. Not necessarily whether or not the person is suspicious. So, we now have for example in the Obama administration, it was revealed that we have signature strikes, where the identity of the individual of the drone strike is not known. And you had one drone strike operator explain that please people get hung up that there’s a targeted list of people on the kill list.

It’s really like we’re targeting a cell phone. We’re not going after people, we’re going after the phones in the hopes that the person on the other end of the missile is the bad guy. So not knowing the identity, but because you have suspicious data being generated by a suspicious phone, you’re killing the phone, you’re killing the person who’s holding the phone. And you also had this, shortly after the Snowden Disclosures, the former Director of the CIA and the NSA, General Mike Hayden, saying we kill people based on metadata. Metadata is data about data. Time of a call, place of a call, length of a call. In a Small Data world, it would be hard to imagine killing people based on that type of data, but in a Big Data world, everything has switched around, and now, you have the former Director of the CIA and the NSA, saying that metadata can form the basis for actual, lethal consequence. This also has led to pre-crime ambitions. So, for example, the Department of Homeland Security has a test pilot program that they call the Future Attributes Screening Technology Program.

In the news, and the media, it’s referred to as a pre-crime program. Under this test pilot program, you have the Department of Homeland Security saying that they’re going to collect physiological cues such as body and eye movements, eye blink rate, pupil dilation, body heat changes, breathing patterns, as well as linguistic cues, such as voice pitch changes, alterations of the rhythm, and changes in the intonations of speech, to detect malintent. Or, in order to detect this threat risk assessment. And the volunteers of this program were informed that the consequences could range from none to being temporarily detained, deportation, prison, and death. And this is based on things like linguistic cues and physiological cues. And so, something else that we have seen from the Snowden Disclosures is that these types of fragments of data are leading to more and more aggregated information that allows for the government to believe that they are engaging in the most efficacious types of decision making that they think is now possible through these technological advances. And, you have some scholars, such as Jack Balkin at Yale Law School, calling this the National Surveillance State. So he says, “It’s one of the most important developments “in American Constitutionalism, “that is the gradual transformation “of the United States into this National Surveillance State “that is the logical successor of basically “our administrative state and our welfare state.” He said that it’s just going to become a way of governance, and we’re going to integrate surveillance into the way that we just carry out our day-to-day business. You have other experts such as Benjamin Wittes, saying that the problem of the justification of mass surveillance is that, the problem is not so much a rule of law problem, it’s really a technological problem.

That, in the United States we might have one of the most constrained intelligence communities in the world but that the US intelligence ambitions scale up to our geopolitical ambitions. And that we have the most awesomely powerful supercomputing capacities of anyone, on earth, so part of the question now is how is it possible that we have, potentially, the most constrained legal apparatus imposed on us for intelligence gathering, and yet, still have potentially the most pervasive and invasive system. So, I wanted to move to now some of the legal apparatus and justifications that help support this. So for example we have the National Security Presidential Directive 59 Homeland Security Presidential Directive 24. It was signed by President Bush in 2008, and it’s called The Biometrics for Identification and Screening to Enhance National Security. So it directs the military and the federal government to collect, store, use, analyze, and share biometrics, and contextual information. It doesn’t define contextual information, but Microsoft defined it this way. All locations that you go, all the purchases you ever make, all your relationships, all activity, all your health, governmental, employer, academic and financial records, your web search history, your calendars and appointments, all your phone calls, data, texts, email, all peoples connected to your social circle, all your personal interests, and all other personal data. So this helps to explain, for example, this slide from the Snowden Disclosures.

So here you had Sniff it All, Know it All, Collect it All, Process it All, Exploit it All, Partner it All. That basically summarizes the philosophy, or the ethos, of the new Big Data cyber-surveillance systems that we find ourselves in. Here’s a slide from one program, Social Radar, from the US Air Force, and you see that it is Collect it All. It is any possible potential communication, any type of information, that can be gathered, you see military, religious, political, economic health, geography, demography, econometrics, using public sources, polls, and surveillance, and then, using that to see into the future. You also have this slide. Total Information Awareness, which was technically defunded after 9/11, but this is also another Collect it All program. So you see that starts with biometric data, and then it goes to automated virtual data repositories, intelligence data, transactional data, including financial, education, travel, medical, veterinary, country entry, place, event, transportation, housing, resources, government communications. So what I wanted to point out was that here is this Hollerith machine. This was found in the Holocaust Museum.

And you had Edwin Black, a journalist, go into the Holocaust Museum in Washington, DC, and asking the archivist, “What is this IBM machine “doing here in the Holocaust Museum?” And after a multi-year study, he produced his book, The IBM and the Holocaust, and what he found was that through the punch card system, the Third Reich was able to very efficiently, through this data collection, and processing system, execute the Final Solution. And, I wanted to put two posters side-by-side. The one on this side, is the Third Reich’s Hollerith machine poster, and then the other one is the United States poster for the Total Information Awareness Program. And there’s some uncomfortable parallels between the two posters. So I wanna now talk about Big Data intelligence, and datafication. And this brings us to a little bit more information just on Big Data and how does it work? So part of what’s happening is we’re going through a transformational moment in our history, where, as in 2000, 25% of all stored information was digitized. By 2012, 98% of all information was digitized. So as far as the scalability of it for example, imagine a gigabyte as a full-length feature film in digital form, and a petabyte is one million gigabytes, an exabyte is one billion gigabytes, and a zettabyte is one trillion gigabytes. And that’s the type of stored data world that we find ourselves in now.

So, the prediction is that the data, the digital data that we create is expected to double every two years, through 2020. And by the year 2020, we’re going to have 5,200 gigabytes of data for every man, woman, and child on earth. And, what some experts such as Mayer-Schonenberger and Cukier in their book, Big Data: A Revolution That Will Transform How We Live, Work, and Think, explained, is that this converts all social and physical reality into a digital format, and that the transformation of that digital data is converting data into new forms of value. And so, I just wanted to give a few statistics on datafication. So for example Google’s more than 100 petabytes in size, experiences more than 7.2 billion page views per day. Processes more than 24 petabytes of data per day, a volume that’s thousands times more the quantity of all printed material in the US Library of Congress. By 2012, Facebook had already reached one billion users, 35% of all the digital photos are stored on Facebook, with 10 million new photos uploaded per hour. YouTube is more than 1000 petabytes in size, over 72 hours of video uploaded to YouTube every minute, more than four billion views per day. 800 million users upload over an hour of video every second.

Twitter, more than 124 billion tweets per year, grows at 200% per year, and at 4500 tweets per second. So we’re also seeing a datafication of us human beings, through the government. The datafication through biometrics, for example, 70 million fingerprints in the criminal master file at the FBI, 34 million civil prints, over 10 million DNA files, and approximately 70 million digital photos for the facial recognition technology for the FBI. The Department of Homeland Security is similarly moving towards datafication through biometrics. Approximately 300,000 fingerprint scans collected every day, 130 million fingerprint files on record, and the Department of Homeland Security has started to gather the DNA of refugees, at at least two refugee sites. You have the Department of State also gathering digital photos that can serve facial recognition technology, such as 200 million digital photos through the passports, and 75 million digital photos through visas. So bringing this back to the Snowden Disclosures, we can see that biometrics and facial recognition technology is forming the basis of new forms of intelligence and the way in which the government is using biometric identity in order to anchor decision making through a formulation of targeting, that is the combination of biometric and biographic information. So, through one of the Snowden Disclosures it was revealed that the NSA intercepts millions of images per day off of the Internet, including 55 thousand facial recognition-quality images. And it was explained that, it’s not just the traditional communications we’re after, but it is the compilation of biographic and biometric information to implement precision targeting.

So it’s basically the fusion, 24/7, of biometric body tracking, with the 360-degree biographical tracking of us as individuals, through, for example, our digital footprints, and our online personas, in order to facilitate data-driven decision-making. So, I want to just wrap up by explaining… Okay, Big Data is technically defined. So you do have these definitions such as volume, velocity, variety, veracity, value, variability, I don’t know why they focused on the Vs, but they did, when they defined Big Data. But Big Data is much more than that. Big Data really is better understood as a philosophy of governance. Big Data as a philosophy is a theory of knowledge, it’s a theory of decision-making, it transforms knowledge, and it transforms in the eyes of government, what it can do, and what it should do. And this affects all of us, so going back to extreme vetting, extreme vetting is misunderstood as a national security program. Extreme vetting is the new normal of the world that we find ourselves in today.

Extreme vetting, and the forms of Big Data Collection and Big Data cyber-surveillance is going to, eventually, influence every right, privilege, freedom, and liberty that we have. So we’re already at the earliest stages of it, in the way that you have, for example, voting rights, having database screening through the Department of Homeland Security databases before there’s an assessment of whether or not you are the appropriate individual of a certain identity that can vote. So there’s digital mediation of voting, you have digital mediation of your right to work, so, employers are screening employee’s data through the Department of Homeland Security databases. You have determinations of who can enter the border, so freedom of movement, but this is just the earliest stages. Right now, through things like the No Fly List, the No Work List, the No Vote List, these are the earliest stages of the Big Data type of capacities that the government has to mediate our rights and privileges. And so, we cannot continue to see the world through these Small Data eyes. The world that we understand it, as it is, right now, we see through the lens of Small Data because that’s what’s humanly knowable. We are about to transform into a world where the government and those that formulate policies for us, look at the world through Big Data glasses. So what they can understand algorithmically, who they can classify as a risk, and what decisions they can make based on those threat-risk determinations.

And so that’s the future of discrimination. So unlike the world that we once had, for example, under Jim Crow, so this is my thesis in algorithmic Jim Crow, when you had race, you had the classification based on something like skin color, for example, and then you had the delegated screening that was conducted by humans, bus drivers, people who owned theaters, people who owned restaurants. They were supposed to do the screening, they were supposed to isolate individuals and then reject them based on their skin color. In a modern world of Big Data, discrimination is going to be operated technologically. So the classification won’t necessarily be based on race, national origin, religion, though it might correlate with it, it’s going to be based on, for example, statistical data assessing risk, and then you’re gonna have classifications that will not necessarily be human judgment, but will be through an algorithm. The algorithm is going to do the screening, and then that is how you’re going to have deprivations of privileges and freedoms through some type of technological method. And we simply do not yet have the legal tools, or constitutional tools, to mediate that type of new form of discrimination that we’re now starting to witness at the very earliest stages. So why don’t I conclude, here, and thank you so much for this opportunity to speak with you today. [applauding] – We’re talking about Big Data, race, and the criminal justice system.

And I wanna begin just by briefly mentioning that these issues have kind of been endemic in biometric identification since the very beginning. Issues of race, issues of colonialism. You’ll find that biometric identification was first sort of pioneered in the laboratory of the colony, and only then, brought back to the metropole, which is something that Professor Hu in her other work talks about, is still going on, now. And the issue of predicting individual behavior. So, I’ll begin by saying a little bit about race. The first ways of identifying people were through looking at their kind of gross, physical aspects through photographs, or through the anthropometric system, where they took measurements, did meticulous facial descriptions, and looked at peculiar marks and scars on their bodies. And this, this system looked, in great detail at the kind of continuum of human variety. The anthropometric system had more than 20 different shades of brown eyes that it identified people with. But this system was considered by Europeans to be unworkable in the colonies, and here’s a quote, quotation, from Francis Galton, saying, well, we can’t use this system in India, and they said the same about China, because they all look the same to us, so we can’t, and, and so, it’s for that reason that fingerprinting arises in India and not back home, in Europe.

It’s the same in the United States, where fingerprinting was considered extremely appealing for identifying Chinese immigrants in the late 19th century. But it was considered not useful for identifying Europeans, and here’s this great quotation from the San Francisco Police Chief, saying, well, you can use fingerprint, you can use fingerprints for indifferent Hindus and wandering Arabs, but when you’re identifying white people, we need something else. We need to look at their faces. And we had the same thing in Argentina, the other cradle of biometric identification, where race played out in kind of a different way between northern and southern Europeans. We have, in the United States, in 1903, the famous Will West case in which, which supposedly demonstrated that fingerprinting could distinguish between two supposedly indistinguishable African American men, who were then distinguished by their fingerprints. This didn’t actually happen, but it, the important thing is it became the sort of origin myth of why we use fingerprints. Then we fast forward a couple decades to 1925, and we have, in the New York City Police Department, the head of the identification bureau saying, who’s talking about his fingerprint file. And here he’s got supposedly this extremely individualizing biometric technology that can identify people down to the individual level, and distinguish between identical twins, and so on, and he’s saying, “Yes, we have that, “but we also sort our files into three groups. “Black, white, and yellow.

“And we do that just by looking at them, cause we know “those races when we see them.” So we have sort of the coexistence of this individualizing technology and this very crude division of people into three groups. And I’d argue that we’re gonna see this kind of throughout this history. This kind of progression, this effort towards individualization that ends up with grouping. And the whole push to use biometrics in criminal identification was about being able to treat criminals as individuals. The idea was that if people were, if we didn’t, before we had biometric identification, people would use aliases, and then we wouldn’t know how long their criminal records were, and then we wouldn’t actually be able to measure recidivism. Once we had biometric identification, then the people who ran prisons, and police departments, could think that we have a pretty good way of detecting recidivism. And then we can do what we’ve always wanted to do, which is punish first-time offenders and recidivists differently. Now this was sort of promoted as individualized punishment, that we would tailor the punishment to the individual. What actually happened was grouping.

There were two sets of prisons. One for the first-timers, and one for the recidivists. So the whole population was divided into two groups, but you did at least segregate off the first-timers from the recidivists, who supposedly would contaminate them with their criminal ways and you could maybe keep them away from that. It’s important to note, for what we’re going to get to later in our discussion, that even though biometrics was useful for detecting some recidivism, it prevented people from using an alias every single time they were arrested. It didn’t of course it wasn’t, of course, a perfect tool for detecting recidivism, for precisely the reasons mentioned by Professor Klingele, that we don’t, that law enforcement doesn’t detect all criminal behavior. In addition, there are other issues with defining recidivism. What do we mean by recidivism? Do we mean committing the same crime, or do we mean a different crime? Do we mean if you are crime-free for 20 years and then you commit another crime, are you a recidivist? Do parole violations count as recidivism? And does that have something to do with how stringent your parole conditions are? And so on. Fast forward a couple more decades, and people start talking about this fingerprint system is great, and shouldn’t everybody be in a database? Shouldn’t we put the whole population in the database, and this discussion, in this country, took place in many countries, and this country sort of peaked in the late 20s, and early 30s. And Americans rejected it, between 1935 and ’43, efforts to create a full citizen fingerprint database were defeated.

And this in a sense, created what I call the arrestee compromise. It created two groups of people. Criminals, which meant, not just people convicted, but anyone who’d been arrested. ‘Cause if you get arrested, your fingerprints go in the database. Plus, some other people. Civil servants, military people, people who work with children. And then, the rest of us who get the privilege of not being in the database. And so this arrestee compromise is going to come back today, if we fast forward a few more decades, we get another biometric technology, seemingly more powerful in many ways, called DNA. And that brings about a kind of public policy problem.

How big should the DNA database be, and who should be in it? So it’s important to realize, that the smaller your DNA database gets, the less useful it is. And if your DNA database is anything short of everybody, there are gonna be some rapes that are going to go undetected. And, those people are going to rape other people. So there are certainly costs to reducing the size of the database, but those have to be balanced against what we, as a society, think is, is fair. And in this country, and in most other countries, the DNA database public policy problem kind of ended up in the same place the fingerprint one ended up with it, which is on the arrestee compromise. In 30 states, including Wisconsin, and my state of California, the US and the UK, we have arrestee databases. And so it is, it is sort of the same compromise that we struck with, with fingerprints. Now the problem with arrestee databases, of course, is that arrest practices in policing is not race-neutral, and it’s not class-neutral, and it’s not geography-neutral. So the fact of having an arrest seems to have something to do with your race, and it can become a sort of backdoor to race, or as Professor Hu has written, race at the backend of the process.

And in this little, sort of quick and dirty kind of attempt to estimate the racial composition of arrestee databases, population-wide databases, and convict databases, I found that the arrestee database is the one that gives you the smallest number of white people and the largest number of black people, as sort of not surprisingly. And this is kind of exacerbated by the fact that familial searching can now be done, which means that if somebody is in the database, their close blood relatives are effectively in the database, even though they’re not in the database. And so, the racial implications sort of balloon out from there. So the arrestee compromise is in some sense the least fair possible system we could come up with, which is what we’ve decided as a society to do. There are two fairer solutions. One is to have a universal database, and put all of us in the database, and then we can all bear equally the burdens of the privacy violations, and the risk of being wrongly convicted, and the burdens of familial searching, and so on, and a number of people have advocated this, who are list, listed here. Kind of on the principle of anti-discrimination, including Alec Jeffreys, who developed DNA typing, and the law professor Michael Smith, who teaches here, I believe. So go see him, he wants all of your DNA to be in a database. But for good reason.

As an anti-discrimination measure. Or, you could maybe have a convict database, which is possibly justifiable on the grounds that you should have reduced privacy because you’ve been convicted of a crime, not just merely arrested for one. I wanna move now to a slight, slightly different topic that is, I’m gonna try to bring together with DNA in a minute. And Professor Hu writes about this as well, which is the development of a kind of philosophy called forensic intelligence, which we could, perhaps distinguish between from treating forensic information as evidence. So forensic intelligence is kind of a new, a hypothetical, mostly, new approach, towards forensics, that distinguishes from the old approach in the following ways. Whereas evidence was oriented towards law and the trial, forensic intelligence is oriented toward policing, and security. Forensic evidence is reactive, it comes in after a crime has occurred. It tries to solve the crime, and it tries to prove who did it. Forensic intelligence isn’t interested in that.

It’s interested in preventing crimes, and being proactive in linking crimes together. Forensic intelligence is very appealing in a lot of ways. There’s a lot of problems with bias, and unscientific reasoning in forensic science that I talk about on other occasions. And forensic intelligence has a lot in common with people like me who are trying to reduce those issues in forensic science. So, it will be less biased. It uses more, kind of probabilistic and scientific reasoning. So it has a certain appeal. But, let me get to the unappealing aspects of it. So the sociologist Sarah Brayne has written, recently, in her study of Big Data policing, how this kind of approach widens the criminal justice dragnet, but in unequally distributed ways, while appearing to be objective, as Professor Klingele mentioned, having the appearance of being simply math.

That this bias can kind of self-perpetuate the existing biases in the data that is fed into these algorithms. And again the simplest example of this is having an arrest, or not having an arrest. Your chances of acquiring an arrest, if you’re a drug user, are not the same depending on your race, your class, and your geographic location. And so, we have criminal justice algorithms, and we have the Loomis case, here, in Wisconsin, and complaints about the use of criminal justice algorithms to predict human behavior, and as Professor Klingele mentioned these algorithms are in some sense, appealing. And the main reason is concerns about bias in the non-algorithmic ways about in the use of discretion by human beings, which is disturbing. And, as Professor Hu has written, the criminal justice algorithms sort of appear, on their face, to be, actually, equality enhancing. Now, two main critiques of these algorithms are lack of discretion, that same discretion that we’re sort of getting rid of, and lack of transparency. So I’m gonna get to those two in second, but now I’m just gonna add a third kind of tech modality to forensic intelligence and DNA databases, and that’s probabilistic genotyping. So I don’t have too much time to go into probabilistic genotyping.

If you wanna know more about it, you can talk to Professor Keith Finley in the law school, and he’ll maybe tell you a little more. But, forensic scientists are pretty good at handling very clean, single-source DNA samples. But forensic DNA profiling starts to get very complicated when you have mixed samples with multiple contributors to them and small amounts of DNA, and they become very difficult to interpret and there have been problems with biased interpretation of these samples. And so, along comes probabilistic genotyping, which says, let’s have a computer interpret these things, according to a set of rules that we’ve pre-programmed before we got the case, so we’re not gonna be biased towards some particular outcome in this case, and plus, the statistics for these complex mixtures is very difficult anyway, you need a computer to do them anyway. So that’s the kind of appeal of them. On, at first glance, and again, that’s a legitimate appeal. You have a couple of problems, though, so in the bottom left you have the legal battles over the source code. ‘Cause defendants say, “Fine, but I’d like “to see the source code that led to this interpretation,” and the makers of the software say, “That’s the entire intellectual property “of our company, and so you can’t see it.” And the courts have to resolve this, which they usually do in favor of the company. On the upper right, you have the Oral Hillary case, which is a murder case in upstate New York, allegedly committed by a black man who was dating a white woman, in a very white part of the country.

And, with a very small DNA sample that required very complex interpretation, and the two, two different pieces of probabilistic software reached two different results in this case. This got very complicated, and then, in the bottom right, I have a press release from the company, Cybergenetics, one of the main vendors of this probabilistic genotyping software saying, that NIST, the National Institute of Standards and Technology, which announced a study where they would test these algorithms, and measure how accurate they were on known samples, Cybergenetics released this news release saying this was a waste of taxpayer money and anti-science, and that they are sort of trying to create a, manufacturing, for over a decade, NIST manufactured crises in DNA mixture interpretation to amass money and power. So, the two critiques, lack of discretion, and lack of transparency. Here’s an op-ed from the New York Times that focuses on the lack of discretion, and her antidote to these criminal justice algorithms is go back to discretion. Have a good old human judge work these things out. Well the problem with that was some, there were some problems with discretion, and the good old-fashioned human, human judge. So it’s not clear how appealing that is. And here’s the lack of transparency argument. This was published by a computer scientist, Duke, who objects to the proprietary nature of the compass algorithm that was used in the Loomis case, and says, “I have a non-proprietary, transparent, “open algorithm, which is better.” And that is true.

Transparency is better than secrecy, but notice that she still has an algorithm that she says predicts criminal behavior really well. And so I wanna emphasize is what we’re doing is the same old thing. We’re predicting criminal behavior, which we’re measuring by arrest, or contacts with the police, and we’re predicting it from arrest, and contacts with the police. And we are bracketing the problem, the detection problem, of whether we’re actually detecting criminal behavior or we’re just detecting that criminal behavior which led to contact with the police. And so we’re protecting, we’re bracketing the problem, the problems with the data that are both input and output from this algorithm, even if it is transparent, which is better. And so, in conclusion, I want to suggest that with criminal justice algorithms, we’re sort of ending up in the same place, in something kind of like the arrestee compromise, where we use these algorithms to kind of predict dangerousness about people kind of like arrestees, right? They’re people with enough contacts with the state that they start to look dangerous in these predictive algorithms. And so, and that seems like it’s going to be kind of discriminatory through the back door or at the back end. And it kind of brings us back to if we don’t like this, what are our other choices? And again with the DNA database, we had two choices. We had every, we had everybody [laughs] or, just convicts.

Now what would that look like in terms of criminal justice algorithms? Well the everybody would be everybody. But as Professor Hu just told us, it wouldn’t just be our DNA samples anymore, it would be everything about us. So it would be everything about everybody. So that would be fair. We’d all bear the privacy violations equally. We’d all bear them together, but we end up with sort of a total surveillance society, or as Professor Hu said, a National Surveillance State. Our other choice is to sort of go back to something like convicts, and to focus, and to focus on people who we can prove have done crimes. And that we’re reasonably sure did them, and not on people who we predict will do them. And kind of the most, the most extreme advancement of this view, is Bernard Harcourt, who’s written a book called Against Prediction, which, as the title indicates, for kind of the reasons I’ve explained, is against prediction altogether, as an activity, not making it better, doesn’t, believes we shouldn’t do it at all.

So thank you. [applauding]