A feature Google demoed at its I/O confab yesterday, using its generative AI technology to scan voice calls in real time for conversational patterns associated with financial scams, has sent a collective shiver down the spines of privacy and security experts who are warning the feature represents the thin end of the wedge. They warn that, once client-side scanning is baked into mobile infrastructure, it could usher in an era of centralized censorship.

Apple abandoned a plan to deploy client-side scanning for CSAM in 2021 after a huge privacy backlash. However, policymakers have continued to heap pressure on the tech industry to find ways to detect illegal activity taking place on their platforms. Any industry moves to build out on-device scanning infrastructure could therefore pave the way for all-sorts of content scanning by default — whether government-led or related to a particular commercial agenda.

Meredith Whittaker, president of the U.S.-based encrypted messaging app Signal, warned: “This is incredibly dangerous. It lays the path for centralized, device-level client side scanning.

“From detecting ‘scams’ it’s a short step to ‘detecting patterns commonly associated w[ith] seeking reproductive care’ or ‘commonly associated w[ith] providing LGBTQ resources’ or ‘commonly associated with tech worker whistleblowing.’”

    • ᴅᴜᴋᴇᴛʜᴏʀɪᴏɴ
      link
      fedilink
      English
      521 month ago

      You can turn it off all you want, it won’t stop the scanning and collection.

      Like turning off Google Location history, its just hidden from your view.

    • @LesserAbe@lemmy.world
      link
      fedilink
      English
      91 month ago

      Trouble is when everybody else uses it, just because you turn it off doesn’t mean you won’t have issues.

  • @retrospectology@lemmy.world
    link
    fedilink
    English
    33
    edit-2
    1 month ago

    Combined with how easy it is becoming to create an AI copy of a person’s voice you’re pretty soon not going to ever be sure if what you’re saying or hearing on a phone is actually what’s being said or if it’s being edited in real time. China’s gonna love this shit.

    Really hate tech bros who just keep recklessly pushing ahead on this stuff. Absolutely the worst scenario for AI.

  • ChaoticNeutralCzech
    link
    fedilink
    English
    12
    edit-2
    1 month ago

    The title feels weird. Is this really a way to use “dial up”?

    == CISCO TM-33k6m2 DIAL-UP MODEM QUICK SETUP ==
    
    STEP 2: SELECT PROVIDER PROFILE
    
      [1] AT&T
      [2] Comcast
      [3] AOL
      [4] Censorship (default)
      
      [0] Specify Provider Parameters...
    
    > _
    
    • @Halosheep@lemm.ee
      link
      fedilink
      English
      161 month ago

      Yes, the phrase comes from turning a dial to a higher value. Think, for example, of a guitar amplifier. You could “dial up” the volume, meaning you are increasing the volume.

      Dial up internet refers to a different kind of dialing.

  • @chunkystyles@sopuli.xyz
    link
    fedilink
    English
    111 month ago

    I’m not advocating for this, but I could see it effectively ending phone scams that often prey on the elderly.

    • @hasnt_seen_goonies@lemmy.world
      link
      fedilink
      English
      21 month ago

      Yeah, I don’t want this for my phone right now, but I do think that this would help me sleep easier with my grandfather who has already fallen for multiple scams.

  • downpunxx
    link
    fedilink
    71 month ago

    If it stops the possibility of being bothered by robocallers, I will give up all my data, my privacy, and the keys to my home, I will even throw in my children

    • @IamAnonymous@lemmy.world
      link
      fedilink
      English
      81 month ago

      Don’t answer unknown numbers and let it ring all the way through. Eventually it stops. If you cut the call or answer it, they will know there is a human being on the other end.

      If they are real they will leave you a voicemail. No need to sacrifice your children 😂

      • Ghoelian
        link
        fedilink
        English
        101 month ago

        Just press volume down once and it’ll stop ringing without actually hanging up. I think that works on pretty much any smartphone.

  • @Infynis@midwest.social
    link
    fedilink
    English
    61 month ago

    You know that scifi trope where someone reaches into a bit of tech, and rips out a blinking tracker? Get ready to have to do that when you buy a new phone

    • @desktop_user
      link
      English
      11 month ago

      sadly there is too much legacy infrastructure to do that without huge problems. Also idiotic forms of two factor authentication exist.

  • Dizzy Devil Ducky
    link
    fedilink
    English
    41 month ago

    I can imagine that in the worst case scenario, company xyz pays millions to gøøgle to have its phone call scamming AI look out for anything that could be taken as even the most mild criticism and cut off your call. That, or the AI randomly cuts out the call because it hallucinates you saying bad things.

  • Possibly linux
    cake
    link
    fedilink
    English
    21 month ago

    I’d be worried about the potential chilling effects. People are likely to self censor talking about topics that could “get them on a list”