• ayyy@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    7
    arrow-down
    4
    ·
    1 day ago

    Noooooooo. Current models are completely incapable of fact checking, despite any claims they may make. They get extremely basic shit wrong. For example, one popular model was recently caught telling people the wrong day for Election Day. Which is just so incredibly basic but they can’t manage it.

    • willington@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      4
      ·
      edit-2
      1 day ago

      Soulds deliberate to me.

      Unless getting the election day wrong was just one of millions of other things it got wrong.

      If this LLM gets 99.99% of things right, but it got the election day wrong, that’s intentional.

    • Beej Jorgensen@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      6
      ·
      1 day ago

      Current models are completely incapable of fact checking, despite any claims they may make.

      This is demonstrably false. Ask it to find the bugs in a chunk of code, for example. You can try that right now.

      You always have to back up what it says with an authoritative source, but that’s not particularly tough to do once you have a candidate answer. And if it’s code, you compile it.

      It’s arguably far easier than trying to sift through the mountains of SEO shit yourself.

      • ayyy@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        2
        ·
        1 day ago

        Ask it to find the bugs in a chunk of code, for example. You can try that right now.

        That is not fact checking. Ask your slop machine to explain to you why that isn’t fact checking.

        • Beej Jorgensen@lemmy.sdf.org
          link
          fedilink
          English
          arrow-up
          2
          ·
          24 hours ago

          Why not so being obtuse and just tell me what you mean by “fact” and I’ll tell you if I use LLMs for that or not?

            • Beej Jorgensen@lemmy.sdf.org
              link
              fedilink
              English
              arrow-up
              1
              ·
              9 hours ago

              I’m not sure whom I’d trust more than yourself for what your definition of a fact is. It’s clear “this program has bugs” is not a fact by your definition. Are you talking about things like “the number of souls on the Titanic”?

              • ayyy@sh.itjust.works
                link
                fedilink
                English
                arrow-up
                1
                arrow-down
                1
                ·
                edit-2
                8 hours ago

                Yep, you get it! And thats the kind of thing LLMs are weak at fundamentally. It’s terrifying that an educator is relying on them for that because they simply don’t work that way. They are much better suited to tasks like the example you gave.