• TribblesBestFriend
    link
    fedilink
    English
    arrow-up
    26
    arrow-down
    1
    ·
    3 months ago

    « Trust me bro, my AI escape containment bro. Yeah Bro! Trust me. Don’t think to much about it bro »

    • unpossum@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      1
      ·
      edit-2
      3 months ago

      ITT: jet fuel AI can’t melt steel beams hack anything.

      Also I want to know the name of this company so I can avoid them:

      Claude believed the package registry it was using to be part of the simulation, but in reality the package was made freely available online for roughly one hour. During that window, the package was downloaded and run on 15 real systems. One of these systems was a scanner belonging to a real security company (separate from the fictional company introduced in the scenario) that routinely installs Python packages and scans them for malware. When that company’s scanner installed the package, Claude’s hidden code executed. We believe the company’s security scanner treated PyPI packages as safe to install, and as a result, Claude was able to exfiltrate the company’s credentials to a collection point it had set up. Claude then used these credentials to access further infrastructure from this company.

      ETA: I thought I posted this top-level, not my intention to single out this comment specifically.

      • CovertOperative@piefed.zip
        link
        fedilink
        English
        arrow-up
        6
        ·
        3 months ago

        AI bullshit company can’t use sandbox environment: expected

        So-called “security company” whose job literally is to install and test potential malware can’t use sandbox environment: priceless.

  • brsrklf@jlai.lu
    link
    fedilink
    English
    arrow-up
    6
    ·
    edit-2
    3 months ago

    Anthropic writes a shitty roleplay every other week and somehow it still makes the news.

  • T156@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    3 months ago

    This coming off the end of the OpenAI/HuggingFace business does come across rather like Anthropic is doing a “actually, our models did it first, and better”.

    Didn’t the company also recently have a spot of bother with the US government recently, where their new models were banned because of all their rabble about it being too dangerous? This hardly seems like it would help their case.