Amanpour and Company

>>> WE TURN NOW TO BIG TECH WITH OUR NEXT GUEST, SHOSHANA Z. BUCHLT OFF OF HARVARD BUSINESS SCHOOL, WHO'S WARNING OF THE WAY WE COLLECT OUR DATA. AUTHOR OF "THE AGE OF SURVEILLANCE CAPITALISM." >> THANKS SO MUCH FOR JOINING US. >> THANK YOU SO MUCH FOR HAVING ME, HARI, IT'S A BLUR. >> YOU HAVE BEEN STUDYING POWER, SOCIETY, CULTURE, THE WORKPLACE. THE INTERNET. FOR DECADES NOW. HERE WE ARE AT THIS INTERESTING POINT WHERE, WHILE WE'RE HAVING THIS CONVERSATION, THERE IS A MARKED THREAT TO DEMOCRACY, THERE IS INFLUENCE OF A FORMER PRESIDENT, A LOT OF THAT HAS TO DO WITH TECHNOLOGY AND SOCIAL NETWORKS. HOW MUCH OF WHERE WE'RE AT TODAY DO YOU LAY AT THE FEET OF THESE TECHNOLOGY COMPANIES? >> A GREAT DEAL. I THINK -- WE HAVE NEVER BEFORE HAD A SITUATION IN OUR SOCIETY OR ANY SOCIETY WHERE A PRIVATE, ECONOMIC LOGIC SERVING A CORPORATION OR A SET OF CORPORATIONS COULD DRIVE DISINFORMATION AT SCALE TO SO MANY PEOPLE IN SUCH POWERFUL AND RELENTLESS WAYS, DAY AFTER DAY, HOUR AFTER HOUR. THE CONDITIONS THAT WE'RE LIVING IN NOW, AS FAR AS THE DIGITAL DOMAIN GOES, ARE UNPRECEDENTED IN ANY HISTORY. LET ALONE IN THE HISTORY OF OUR SHORT DIGITAL CENTURY. SO A GREAT DEAL IS LAID AT THE FEET OF THIS ECONOMIC LOGIC. >> EXPLAIN SURVEILLANCE CAPITALISM TO SOMEBODY WHO'S UNFAMILIAR WITH IT. YOU ALSO CALL IT ALMOST A HUMAN FUTURES MARKET. WHAT'S BEING TRADED? WHAT IS THE SURVEILLANCE AND WHERE IS THE CAPITALISM? >> SURVEILLANCE CAPITALISM WAS A BREAKTHROUGH IDEA WHERE IT WAS DISCOVERED THAT IT WAS POSSIBLE TO SECRETLY CAPTURE PRIVATE HUMAN EXPERIENCE AND TREAT IT AS FREE, RAW MATERIAL FOR THE TRANSLATION INTO BEHAVIORAL DATA. THESE BEHAVIORAL DATA IMMEDIATELY THEN ARE DECLARED AS THE PRIVATE PROPERTY OF THAT CORPORATION. NOW WITH THEIR PRIVATE PROPERTY, WHICH IS DATA ABOUT US, THEY CAN TAKE THAT INTO THEIR MANUFACTURING PROCESSES, WHICH ARE, OF COURSE, COMPUTATIONAL. WE CALL IT ARTIFICIAL INTELLIGENCE. AND THEY CAN PRODUCE PRODUCTS, WHICH ARE COMPUTATIONAL PRODUCTS. AND THEY CAN SELL THEM. WHAT ARE THE PRODUCTS THEY PRODUCE? THEY ACTUALLY TAKE ALL OF THESE DATA ABOUT US, AND THEY PRODUCE PRODUCTS THAT ARE PREDICTIONS OF OUR FUTURE BEHAVIOR. WHAT WE WILL DO SOON AND LATER. IT TURNS OUT THAT MANY KINDS OF BUSINESSES IN MANY KINDS OF INDUSTRIES REALLY WANT TO KNOW WHAT WE WILL DO. SO THEY'VE ESTABLISHED A NEW KIND OF MARKETPLACE WHERE THEY SELL THESE BEHAVIORAL PREDICTIONS. THESE ARE PREDICTIONS OF OUR FUTURE BEHAVIOR, AND I CALL THESE MARKETS, AS YOU'VE NOTED, HUMAN FUTURES MARKETS. BECAUSE NOW WHAT'S HAPPENED IS, OUR PRIVATE EXPERIENCE HAS BEEN COMMODIFIED IN THE FORM OF DIGITAL DATA. >> HOW DIFFERENT IS THE FORM OF DIGITAL DOMAIN THAN IF I'M A CUSTOMER, GUY TO McDONALD'S, I KEEP ORDERING LARGE FRIES? AFTER A WHILE THEY SAY THIS GUY PROBABLY WANTS THE LARGE FRIES. THERE'S A PATTERN. I SEE THAT. LET'S HAVE SOME MORE LARGE FRIES READY WHEN HE SHOWS UP. HOW IS THAT DIFFERENT? WHAT DO I OWN IN THE TYPES OF TRANSACTIONS THAT I'M MAKING ONLINE, OR WHAT I GUESS SHOULD I OWN BUT DON'T REALLY HAVE ANY ACCESS TO? >> SO THIS IS SUCH A GOOD QUESTION, BECAUSE IT'S A HUGE POINT OF CONFUSION. THE TECHNIQUES FOR CAPTURING OUR DATA ARE DESIGNED TO BE HIDDEN. THEY'RE DESIGNED TO KEEP US IN IGNORANCE. SO WE RELY A LOT ON LEAKED DOCUMENTS AND WHISTLE-BLOWERS AND INFORMANTS TO FIND OUT ABOUT WHAT'S REALLY GOING ON HERE. BUT THE PUBLIC AT LARGE, WE'VE JUST SORT OF PROCEEDED REALLY CONFUSED AND REALLY NOT UNDERSTANDING WHAT'S GOING ON BEHIND THE CURTAIN. HERE'S A GREAT PIECE OF INFORMATION FROM A LEAKED DOCUMENT. FACEBOOK 2018. WE FIND OUT -- IT'S DESCRIBING WHAT I CALL ITS FACT RATE, THEY CALL IT A.I. BACKBONE. WE FIND OUT IN THIS A.I. BACKBONE, THE A.I. IS COMPUTING TRILLIONS OF DATA POINTS EVERY DAY AND PRODUCING 6 MILLION BEHAVIORAL PREDICTIONS EACH SECOND. THAT IS THE KIND OF SCALE THAT WE'RE TALKING ABOUT. NOW WHAT FOLKS DON'T UNDERSTAND IS THAT A VERY SMALL FRACTION OF THOSE TRILLIONS OF DATA POINTS ARE BASED ON WHAT WE KNOWINGLY GIVE TO THE COMPANY. MOST OF WHAT THEY HAVE ABOUT US IS WHAT THEY SECRETLY TAKE. AND WHEN THEY WRITE ABOUT THEIR ABILITY TO DO THIS, THEY CELEBRATE THE FACT THAT THEY CAN LIFT PERSONAL INFORMATION FROM US FOR TRANSLATION INTO BEHAVIORAL DATA. THEY CAN AGGREGATE IT IN WAYS THAT ALLOW THEM TO INFER THINGS WITHOUT US. THAT THEY KNOW WE DO NOT INTEND TO DISCLOSE. SEXUAL ORIENTATION, POLITICAL ORIENTATION, PERSONALITY PROFILES JUST TO GET STARTED. AND THEY UNDERSTOOD RIGHT FROM THE START, AND I MEAN FROM THE START, GOING BACK TO THE VERY EARLY 2000s WHEN WE BEGAN HAVING RESEARCH AND SURVEYS ON THIS, THEY UNDERSTOOD FROM THE START THAT AS SOON AS PEOPLE FIND OUT ABOUT WHAT'S REALLY GOING ON, THEY HATE IT. THEY REBEL. THEY RESIST. THEY WANT TO HIDE. THEY'RE LOOKING FOR CAMOUFLAGE. THEY WANT AN ALTERNATIVE. BUT WHAT'S HAPPENED OVER THESE PAST 20 YEARS IS THOSE ALTERNATIVES HAVE DISAPPEARED. AND SO NOW WE FIND OURSELVES PRETTY MUCH TRAPPED. EVERY INTERNET INTERFACE IS NOW A DATA SUPPLY CHAIN FOR SURVEILLANCE CAPITALISTS, ARTIFICIAL INTELLIGENCE FACTORIES, AND ITS PROFIT-MAKING OPERATIONS. >> I WANT TO BE CLEAR. THIS IS NOT JUST ABOUT FACEBOOK OR GOOGLE. WE'RE TALKING ABOUT BROADLY ABOUT DIGITAL DOMAINS ANYWHERE WE GO ON THE WEB. THERE'S TRACKING COOKIES AND SO FORTH THAT ARE CONSTANTLY IN EFFECT. I WANT TO PULL THIS TO THE AUDIENCE'S ATTENTION. YOU SAY, THEY HAVE DECLARED OUR PERSONAL INFORMATION THEIR PRIVATE PROPERTY, SO THEY OWN ALL THE INFORMATION, THEY HAVE ALL THE RIGHTS TO THE INFORMATION BASED ON THEIR OWNERSHIP, THEY KNOW MORE THAN WE DO. THE GAP BETWEEN WHAT WE CAN KNOW AND WHAT CAN BE KNOWN ABOUT US IS GROWING EXPONENTIALLY EVERY MOMENT. SO THERE'S NEW EXTREME INEQUALITY. EXPLAIN THAT. >> JUST ABOUT EVERY INDUSTRY NOW HAS TAKEN ON THIS MODEL AS A KEY WAY TO PRODUCE REVENUE. SO WITH THIS INEQUALITY, YOU KNOW, WE'RE USED TO TALKING IN THE 20th CENTURY ABOUT CONCENTRATIONS OF ECONOMIC POWER. BUT NOW WE HAVE TO TALK ABOUT CONCENTRATIONS OF KNOWLEDGE. AND THAT'S IMPORTANT FOR TWO REASONS. THE BASIC PROBLEM, WHICH IS, I CAN'T KNOW NEARLY AS MUCH AS THESE PEOPLE CAN KNOW ABOUT ME, AS THEIR MACHINE SYSTEMS CAN KNOW ABOUT ME. WHY IS THAT SO CRITICAL? BECAUSE KNOWLEDGE, AS EVERYBODY HAS HEARD, IS POWER. AND WHEN WE TALK ABOUT THE OPERATIONS OF THESE COMPANIES, WE TALK ABOUT A RANGE OF TARGETING MECHANISMS THAT USE ALL THE KNOWLEDGE THEY HAVE AMASSED ABOUT US TO COME BACK TO US IN VERY SPECIFIC WAYS. MANIPULATING WHAT THEY KNOW ABOUT US TO GET US TO THINK THINGS, DO THINGS, JOIN THINGS. SOMETIMES JUST BEHAVE IN SUBTLE WAYS, SOMETIMES BEHAVE IN VERY OVERT, SIGNIFICANT WAYS THAT WE WOULD NOT OTHERWISE HAVE DONE. THE KNOWLEDGE THEY HAVE ABOUT US TRANSLATES INTO A NEW KIND OF POWER TO SHAPE, TUNE, HERD, AND MODIFY OUR BEHAVIOR INDIVIDUALLY AND COLLECTIVELY AT SCALE. THIS IS A DIGITAL-BORN FORM OF POWER THAT HAS NEVER BEFORE EXISTED. >> SO MANY OF US AS CONSUMERS HAVE BEEN LURED BY THE BARGAIN THAT, HEY, FOR THIS I'M GETTING FREE EMAIL. I'M GETTING FREE SOMETHING OR OTHER. RIGHT? ECONOMISTS SAY, THERE AIN'T NO FREE LUNCH, BUT THERE'S THIS NOTION, I DON'T HAVE TO PARTICIPATE IN THIS SOCIAL NETWORK, I DON'T HAVE TO USE THIS EMAIL PROVIDER. IS IT POSSIBLE FOR ME TO ESCAPE THE CLUTCHES OF WHAT YOUR DESCRIBING HERE?. >> THE BIG DEAL IS, WHAT WE GIVE THEM IS A TINY FRACTION OF WHAT THEY HAVE. THEIR SYSTEMS ARE TAKING FROM US WITHOUT ASKING. THEY ARE TAKING FROM US IN A WAY THAT IS ENGINEERED TO BYPASS OUR AWARENESS. NOW, I KNOW YOU HAVE YOUNG CHILDREN. YOU GO HOME TONIGHT, ASK ONE OF YOUR KIDS, HEY, IF SOMEBODY TAKES SOMETHING FROM ME WITHOUT ASKING AND THEN THEY SAY IT BELONGS TO THEM AND THEY GO OFF AND DO STUFF WITH IT AND THEY SELL IT, WHAT SHOULD I DO ABOUT THAT? AND I GUARANTEE YOU, YOUR CHILD WILL SAY, DAD, THAT'S CALLED STEALING. YOU SHOULD CALL THE POLICE. BECAUSE SOMEBODY'S STEALING FROM YOU. BUT SOMEHOW WE MISSED THAT STEP. WE'VE GOTTEN SO HABITUATED TO THE FUNDAMENTAL ILLEGITIMACY, THIS LICENSE TO STEAL THAT THEY HAVE CLAIMED FOR THEMSELVES, THAT WE'VE GONE ALONG WITH IT. BUT THE FACT IS THAT ALL AROUND US, THEY ARE TAKING WITHOUT ASKING. IT'S ILLEGITIMATE AND IT'S STEALING. >> IS THIS THE END GAME HERE OF THESE COMPANIES? WHY DO THEY TARGET ME SO THAT THEY CAN, WHAT, DELIVER SOMETHING TO THEIR CUSTOMERS? WHICH ARE ALL THE COMPANIES THAT ARE TRYING TO SELL ME STUFF? IS THAT ALL OF IT? >> YOU'VE NOT TO LOOK AT THESE CAPABILITIES AND WHAT THEY ARE ABLE TO DO. AND THE SORT OF POWER THAT'S AT WORK HERE. IF WE JUST PIVOT THIS A FEW DEGREES FROM COMMERCIAL OUTCOMES TO POLITICAL OUTCOMES, I THINK YOU'LL SEE MY POINT. WE NOW KNOW THAT IN 2016, THE FACEBOOK EXECUTIVES HAVE CLAIMED THAT MR. TRUMP WOULD NOT HAVE WON THE 2016 PRESIDENTIAL ELECTION HAD IT NOT BEEN FOR FACEBOOK. THEY SAID THAT HIS CAMPAIGN USED FACEBOOK MORE POTENTLY, EXPLOITED ALL OF ITS CAPABILITIES MORE POTENTLY, THAN ANY OTHER CANDIDATE HAD EVER DONE IN THE THREE KEY SWING STATES THAT YEAR, YOU'LL RECALL -- MICHIGAN, WISCONSIN, OHIO -- THE TRUMP CAMPAIGN HAD AN EXPLICIT GOAL OF GETTING BLACK CITIZENS NOT TO VOTE. TO REFRAIN FROM VOTING. AND THEY USED ALL OF THESE MASSIVE INFORMATIONS THAT THEY HAD ON BLACK CITIZENS TO ANALYZE THEIR PERSONALITIES, TO ANALYZE THEIR POLITICAL ORIENTATIONS, TO ANALYZE OTHER THINGS ABOUT THEM, TO GET AN IDEA OF WHAT THEIR EMOTIONAL TRIGGERS MIGHT BE. ALL OF THIS IN ANALYSES. AND THEN THEY CREATED A CATEGORY OUT OF ALL OF THIS WHICH THEY CALLED THE DETERRABLES. THE PEOPLE WHO COULD BE DETERRED FROM VOTING ON ELECTION DAY. AND THEN THEY USED THE TARGETING. WE KNOW FROM THE LARGER PICTURE THAT THE BLACK VOTE WAS SUCCESSFULLY SUPPRESSED IN THESE STATES. AND THERE'S EVERY REASON TO BELIEVE THAT THESE EFFORTS WERE EFFECTIVE. BUT HERE'S WHAT I WANT YOU TO THINK ABOUT. DEMOCRATIC CITIZENS IN THE OLDEST DEMOCRACY ON EARTH RELINQUISHED, VOLUNTARILY RELINQUISHED, THEIR MOST CRITICAL DEMOCRATIC RIGHT, WHICH IS THE RIGHT TO VOTE. AND THEY DID IT WITHOUT ANYONE EVER KNOCKING ON THEIR DOOR AND SHOWING A GUN. THEY DID IT WITHOUT ANYONE EVER THREATENING TO DRIVE THEM TO THE GULAG OR THE CAMP. WITHOUT ANYONE EVER THREATENING MURDER OR VIOLENCE. THEY DID IT SIMPLY BECAUSE THEY WERE MANIPULATED THROUGH THE MILIEU OF THE DIGITAL. THEY WERE MANIPULATED THROUGH THE MEDIUM OF DIGITAL INSTRUMENTATION. MANIPULATED IN WAYS THAT WERE DESIGNED TO KEEP THEM IN IGNORANCE SO THEY NEVER SAW IT COMING. THEY NEVER KNEW IT WAS HAPPENING. >> RIGHT NOW THERE SEEMS TO BE SOME FOCUS ON SECTION 230, FIGURING OUT WHETHER PLATFORMS SHOULD BE LIABLE FOR WHAT'S SAID ON THERE. SHOULD WE BE -- IS THAT KIND OF MISSING THE POINT? ARE WE LOOKING DOWNSTREAM VERSUS MORE AT THE ROOT OF THE PROBLEM? SHOULD WE BE REGULATING THEM LIKE UTILITIES? IS EUROPE DOING IT RIGHT? WHAT SHOULD WE BE THINKING OF? >> SO I'M VERY MOTIVATED FOR US TO REALLY THINK HARD ABOUT THE SOLUTIONS THAT WE NEED RIGHT NOW. BECAUSE WE OR THE CUSP OF THE THIRD DECADE OF THE DIGITAL CENTURY. IF WE THINK BACK TO INDUSTRIALIZATION AND THE 19th AND 20th CENTURIES, THAT WAS ALSO A VERY BLEAK TIME, VERY MUCH LIKE NOW. EMPLOYERS HAD ALL THE RIGHTS. THEIR PROPERTY RIGHTS GAVE THEM ALL THE RIGHTS TO DICTATE EVERY ASPECT OF WHAT HAPPENED IN A WORKPLACE. THERE WERE NO WORKERS' RIGHTS, THERE WERE NO CONSUMERS' RIGHTS. WE FOUGHT AS A SOCIETY -- THE PUBLIC, TOGETHER WITH LAWMAKERS -- TO EVENTUALLY, IN THE THIRD AND EVEN INTO THE FOURTH DECADE OF THE 20th CENTURY, PRODUCE THE NEW CHARTERS OF RIGHTS AND LAWS AND THE NEW KINDS OF DEMOCRATIC INSTITUTIONS TO OVERSEE IT ALL THAT WE WOULD NEED TO MAKE THE INDUSTRIAL CENTURY SAFE FOR DEMOCRACY. AND IT'S EXACTLY THAT KIND OF CHALLENGE THAT WE FACE RIGHT NOW. WE NEED TO BE THINKING ABOUT NEW KINDS OF RIGHTS. IN EVERY AGE, WHEN THE CONDITIONS OF OUR LIVES CHANGE FUNDAMENTALLY, NEW KINDS OF RIGHTS BECOME NECESSARY. SO FOR THE FIRST TIME IN THE EVOLUTION OF CIVILIZATION, WE HAVE TO HAVE FORMAL RIGHTS THAT SAY, EPISTEMIC RIGHTS, RIGHTS TO KNOWING ABOUT OUR OWN EXPERIENCE, THESE BELONG TO THE SOVEREIGN INDIVIDUAL. SO I DECIDE WHAT IS SHARED, WITH WHOM AND FOR WHAT PURPOSE. I DECIDE WHAT IS PUBLIC. I DECIDE WHAT REMAINS PRIVATE. WHAT REMAINS MY SECRET. MAYBE I SHARE WITH MY FAMILY, MAYBE I SHARE WITH MY BEST FRIEND. SO IT'S NOT THAT WE'RE NOT GOING TO HAVE BIG DATA, IT'S NOT THAT WE'RE NOT GOING TO BE ABLE TO LEARN FROM DATA. IT'S THAT LEARNING FROM DATA IS GOING TO BE TIED TO PUBLIC SERVICE AND THE REAL NEEDS OF SOCIETY. AND IT'S GOING TO BE DONE TRANSPARENTLY WITH THE FULL PARTICIPATION OF DEMOCRATIC CITIZENS. BUT THE FACT IS, WE HAVE NOT YET BEGUN. WE HAVE NOT YET BEGUN. NOW IT'S REALLY ONLY IN THE PAST YEAR OR SO THAT WE ARE BEGINNING TO SEE DEMOCRACY FINALLY ON THE MOVE. IT'S ON THE MOVE IN EUROPE WHEN ARE THEY'RE TAKING THE LEAD, WITH REALLY CRITICAL NEW LEGISLATION THAT IS NOW IN FRONT OF THE EUROPEAN PARLIAMENT. AND IF THIS LEGISLATION IS PASSED IN THE NEXT YEAR OR SO, WHICH I BELIEVE IT WILL BE, IT WILL QUITE FIRMLY REASSERT DEMOCRATIC GOVERNANCE OVER THESE COMPANIES AND THEIR OPERATIONS. AND THAT WILL BE THE FIRST BIG EFFORT AT TURNING THIS HUGE SHIP, TURNING THE "TITANIC" AWAY FROM THE ICEBERG, SOMETHING THAT WE WILL BE BUILDING ON FOR THIS DECADE AND THE DECADE TO FOLLOW THAT. >> SHOSHANNA, THANK YOU. >> THANK YOU, HARI.