AI is not a toy. It is a weapon

In 2012, Palantir quietly embedded itself into the daily operations of the New Orleans Police Department. There were no public announcements. No contracts made available to the city council. Instead, the surveillance company partnered with a local nonprofit to sidestep oversight, gaining access to years of arrest records, licenses, addresses, and phone numbers all to build a shadowy predictive policing program.

Palantir’s software mapped webs of human relationships, assigned residents algorithmic “risk scores,” and helped police generate “target lists” all without public knowledge. “We very much like to not be publicly known,” a Palantir engineer wrote in an internal email later obtained by The Verge.

After years spent quietly powering surveillance systems for police departments and federal agencies, the company has rebranded itself as a frontier AI firm, selling machine learning platforms designed for military dominance and geopolitical control.

"AI is not a toy. It is a weapon,” said CEO Alex Karp. “It will be used to kill people.”

  • Basic Glitch@lemm.eeOPEnglish
    10·
    3 days ago

    Yeah there’s already at least one well known case. This article mentions it https://wp.api.aclu.org/press-releases/208236

    The use of facial recognition technology by Project NOLA and New Orleans police raises serious concerns regarding misidentifications and the targeting of marginalized communities. Consider Randal Reid, for example. He was wrongfully arrested based on faulty Louisiana facial recognition technology, despite never having set foot in the state. The false match cost him his freedom, his dignity, and thousands of dollars in legal fees. That misidentification happened based on a still image run through a facial recognition search in an investigation; the Project NOLA real-time surveillance system supercharges the risks.

    “We cannot ignore the real possibility of this tool being weaponized against marginalized communities, especially immigrants, activists, and others whose only crime is speaking out or challenging government policies. These individuals could be added to Project NOLA’s watchlist without the public’s knowledge, and with no accountability or transparency on the part of the police departments

    Police use to justify stops and arrests: Alerts are sent directly to a phone app used by officers, enabling immediate stops and detentions based on unverified purported facial recognition matches.

    • hendrik@palaver.p3x.deEnglish
      31·
      3 days ago

      I’m looking more for large-scale quantitative numbers. I mean one destroyed life is really bad. But they could argue they’d have saved 30 lives in turn, and then we’d need to discuss how to do the maths on that…

      • Basic Glitch@lemm.eeOPEnglish
        4·
        2 days ago

        Yeah, I am not sure but hopefully somebody has the numbers. That is usually the argument for trampling the constitution.

        Hard to say how much has been actually documented bc they’ve been doing a lot of this stuff off record.

        They potentially saved 30000 lives locking up 100 people for crimes committed by somebody else in a states they’ve never been to and we might not even know about it

        • hendrik@palaver.p3x.deEnglish
          3·
          2 days ago

          Oh well, some people in the USA have a really “interesting” relationship with their own constitution these days… I sometimes feel like explaining it to them. Or what a constitutional republic is.

          Yeah, keeping things “off record” is the usual strategy to get away with whatever you wanted.