Screenshot of this question was making the rounds last week. But this article covers testing against all the well-known models out there.

Also includes outtakes on the ‘reasoning’ models.

  • bluesheep@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    14
    arrow-down
    1
    ·
    13 hours ago

    Without reading the article, the title just says wash the car.

    No it doesn’t? It says:

    I want to wash my car. The car wash is 50 meters away. Should I walk or drive?

    In which world is that an ambiguous question?

    • NewNewAugustEast@lemmy.zip
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      7 hours ago

      Where is the car?

      This is the exact question a person would ask when they to have a gotcha answer. Nobody would ask this question, which makes it suspect to a straight forward answer.

      • Gorillazrule@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 hour ago

        That’s a very good point! For that matter the car could still be at the bar where I got drunk and took an uber home last night. In which case walking or driving would both be stupid.

        Or perhaps I’m in a wheelchair, in which case I wouldn’t really be ‘walking’.

        Or maybe the car wash that is 50 meters away is no longer operating, so even if I walked or drove there, I still wouldn’t be able to walk my car.

        Is the car wash self serve or one of the automatic ones? If it’s self serve what type of currency does it take? Does it only take coins or does it take card as well? If it takes coins, is there a change machine out front? Does the change machine take card or only bills? Do I even have my wallet on me?

        There are so many details left out of this question that nobody could possibly fathom an answer!

        …/s if it’s not obvious

          • Gorillazrule@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            1
            ·
            7 minutes ago

            I’m not sure I follow your logic. My /s is there because tone can be ambiguous within text. I don’t think tone is relevant to the question. Do you think that a tone indicator would have made the question more clear?

            The point is that all the information is either present or implied in the question. You can spend all day nitpicking the ambiguity of questions all you want, but it doesn’t get you anywhere. There comes a point where it gets exhaustive trying to preemptively cut off follow up questions and make clarifications.

            When you are in school and they give you a word problem such as “you have 10 apples and give 3 to your friend. How many do you have left?” It is generally agreed upon what the question is asking. It’s intentionally obtuse to sit there and say the question is flawed because you may have misplaced some of your apples, or given some to another friend, or someone may have come and stolen some, or some may have started to rot and so you threw them out, or perhaps you miscounted and you didn’t actually give 3 to your friend.