I have had this happen pretty often. I think even the wording used in the “prompt” can also influence whether the chatbot hallucinates a solution.
I remember recently asking a third party’s mandatory chatbot support system (if it can’t help, it then forwards you to a human) whether I was able to change the mapping of some fields in an upload to them without the need for a support agent. It walked me through the process but it didn’t work. Eventually, after three failed chatbot modifications to the instructions, I was forwarded to a human that confirmed I was not, in fact, able to do it myself. Such a fucking waste of time.
What gets me is that despite how often LLMs hallucinate a completely incorrect answer like this, for some reason people still blindly trust them and regurgitate their output without telling you where it came from.
You would think after the first time of encountering an output that is verifiably false people would at least have some hesitancy in the output, but they still keep blindly trusting the language guessing machine.
I have had this happen pretty often. I think even the wording used in the “prompt” can also influence whether the chatbot hallucinates a solution.
I remember recently asking a third party’s mandatory chatbot support system (if it can’t help, it then forwards you to a human) whether I was able to change the mapping of some fields in an upload to them without the need for a support agent. It walked me through the process but it didn’t work. Eventually, after three failed chatbot modifications to the instructions, I was forwarded to a human that confirmed I was not, in fact, able to do it myself. Such a fucking waste of time.
What gets me is that despite how often LLMs hallucinate a completely incorrect answer like this, for some reason people still blindly trust them and regurgitate their output without telling you where it came from.
You would think after the first time of encountering an output that is verifiably false people would at least have some hesitancy in the output, but they still keep blindly trusting the language guessing machine.