In March, a girl’s stepfather took his own life after cops discovered that he had used Grok to create 7,000 sexually explicit images using one photo taken when his stepdaughter was 11 years old, the amended complaint alleged.

Grok allowed the man to generate extreme images depicting incest and rape without flagging any harmful behavior, the complaint said. Seemingly, xAI’s child safety system only intervened after the man input a prompt for “gang rape.” That request sent a CyberTip to the National Center for Missing and Exploited Children (NCMEC), which alerted law enforcement to the AI CSAM.

Yet the harm was not stopped then, either. Despite mandatory reporting requirements to share information like a user’s IP address when CSAM is flagged, xAI repeatedly refused to help cops or NCMEC identify the user, the complaint alleged. For weeks, xAI allegedly “obstructed this investigation at every turn” and made it harder for “law enforcement efforts to locate, identify, and apprehend the perpetrator.”

Eventually, the stepfather was arrested after cops obtained a warrant to seize his devices. That’s when “a forensic review revealed approximately 7,000 AI-generated images and videos” depicting his stepdaughter, which were allegedly produced using Grok. Without Grok providing users with easy access to “undressing” capabilities, his family doubts he ever would have generated the harmful images, which he allegedly trafficked online in trade for “CSAM produced by other child sex predators.”

    • AdolfSchmitler@lemmy.world
      link
      fedilink
      arrow-up
      29
      ·
      9 days ago

      Reminds me of that chart showing left handedness over time. I’m afraid the implication is that there have always been that many pedos in the world, we’re just getting better at catching it publicly or something.

    • ImgurRefugee114@reddthat.com
      link
      fedilink
      arrow-up
      19
      ·
      9 days ago

      Not so fun fact: a lot of child sex abusers aren’t even pedos (clinically); they just prey on the vulnerable they have access to. And there are those who just do it for money or power-fantasy or taboo. Plus there are lots of literal pedos who are functionally harmless, and just got their brains wired wrong. So the moment you go beyond the rates of prosecuted criminality, things get blurry and uncertain real quick. The error bars and qualifiers stack up fast.

      • zqps@sh.itjust.works
        link
        fedilink
        arrow-up
        10
        ·
        8 days ago

        Both points you raise are really important in understanding how child abuse happens and how to prevent it. Which is why I’m extremely annoyed that we treat the terms “pedophile” and “child abuser” as the same.

        But you can’t normally bring that up or people think you’re defending child abusers for some reason.

        • ImgurRefugee114@reddthat.com
          link
          fedilink
          arrow-up
          7
          ·
          edit-2
          8 days ago

          Sry walking and talking; thought part 2:

          The issue is that that education is really important for preventing harm. People have to understand that it’s more about power dynamics and access than anything else. As that education is important for adults, likewise it’s also important for kids. That’s why sex education is so important so that they know when something is going wrong. Sex ed is an anti-grooming tool which is why it pisses me off so much when conservatives attack it as if it’s grooming. But I digresss.

          • dustyData@lemmy.world
            link
            fedilink
            English
            arrow-up
            4
            ·
            edit-2
            5 days ago

            The reason it is such a difficult topic and it is thus weaponized by right wing politics is the uncomfortable fact: the vast majority of CSA is committed by a family member, usually parents. The boogeyman “stranger danger” monster rhetoric is comfortable and easy to digest. The fact that your own husband, father, grandpa/grandma, aunt or uncle, etc. could be the monster causes uneasiness. The anti-sex-education rhetoric is not accidental. A third party teaching proper consent and protection from CSA to children means it dilutes the power of families to either, get away with CSA themselves or protect their family members from the legal repercussions of committing CSA.

            There’s also the reproduction of capitalism side of things. But that argument drowns the discussion of pedophilia and CSA even more in a global culture primed to hate communism and socialism as a knee jerk reaction. Essentially, the familial (let’s be honest, paternal) ownership over children is foundational for the cultural reproduction of capitalism. It is the suppression of sex education and the absolutist domination of the family, and fathers specifically, over the bodies of children that enables later in life the reproduction by similarity of capitalist exploitation of labor. This goes to the root of ideological differences, the right speaks of “the protection of children” as a mere tool to further control and exploit. Protect the family, in so far as alternative families (like LGBT, mixed race, adoption and reconstructed families) pose a direct threat to the scheme of capitalist extraction of wealth from people. At the crux of it all, the oppression and domination of human sexuality as a whole as a political means. The result, the normalization of sexual exploitation of women and children as an acceptable (even desirable) collateral for the ensuring of next quarter’s profit and the promise the next generation will keep up the ideology going.

        • ImgurRefugee114@reddthat.com
          link
          fedilink
          arrow-up
          2
          ·
          edit-2
          8 days ago

          Yeah, “technically that’s hebephilia” is a meme for a reason, much to the dismay of pedants and clinicians alike. The colloquial usage and discourse is a mess, but it’s hard to get you mad at the sloppy rhetoric when it’s usually aimed at some of the worst people on Earth…

          • zqps@sh.itjust.works
            link
            fedilink
            arrow-up
            1
            ·
            edit-2
            8 days ago

            Yep though I do think this technical distinction you bring up is really only relevant in a clinical context.

            Meanwhile the one between “pedophile” and “child abuser” is extremely important in real-life prevention of child abuse. The ideas that 1) only people with sexual attraction to kids would sexually abuse kids, and 2) if you feel sexual attraction to kids obviously you’re gonna act on it, both seem logical but have both been comprehensively disproven.

            A huge added barrier here is that people - for understandable reasons - don’t want to really try to think through and reason logically about such a disgusting and emotionally laden subject, much less empathize with a pedophile. Unfortunately that exactly is required if we’re not just looking to satisfy our vengeful sense of justice, but actually do something for the victims and prevent further child abuse.

    • starelfsc2@sh.itjust.works
      link
      fedilink
      arrow-up
      3
      ·
      8 days ago

      The study grouped all children in “under 18,” I don’t even want to type this but women are generally sexually developed by around 16. A lot of people can’t tell apart a 16 year old and a 21 year old without talking to them. The 16 year old is obviously still a kid, I’m just saying it’s not full blown creep stuff.