If you enter the same prompt with the same seed you will get the same vector outputs.
Chatbots take those vector outputs and treat them as a distribution and select a random token. This isn’t a property of the LLMs, it’s a property of chatbots.
The entire comment chain is about LLMs, not chatbots.
A LLM’s output is deterministic.
A from an IT system point of view, a system using LLMs is only non-deterministic if the system adds randomness. The randomness isn’t inherent to the LLM and any added randomness is typically created by a seed which could be replicated across number of system images leading to deterministic output across every image clone.
From the point of view of the typical user the computer may as well be a magical box controlled by a tiny fairy so any talk of deterministic output is irrelevant.
The entire comment chain is about LLMs, not chatbots.
The post is about the actual product they’re selling, not whatever idealized idea of what a ‘proper’ LLM is.
If every LLM sold is sold as a chatbot, then this “ummm ackchully” is irrelevant. It doesn’t matter what LLMs are technically capable of if none of the LLMs sold actually function that way.
The post is about the actual product they’re selling, not whatever idealized idea of what a ‘proper’ LLM is.
Yes, that is what the post is about.
You didn’t click on reply to the post, you clicked reply under my comment.
In my comment, I was talking about an LLM (I checked with myself) and the other person was also talking about LLMs and on up to the top of the comment chain where we started talking about LLMs in IT systems.
From the context of the conversation, you should understand that we’re talking about LLMs, specifically being in IT having to deal with LLMs. The context should tell you that we’re talking about the actual language models and not the end user applications, like a chatbot.
If they aren’t selling non-chatbot LLMs then that’s irrelevant.
Ok, well this is easy then. Every LLM isn’t sold as a chatbot so I’m not sure why you keep repeating this like it is a point.
If every LLM sold is sold as a chatbot, then this “ummm ackchully” is irrelevant.
Your first comment was ‘ummmm ackchully LLMs are only chatbots’ which is both wrong and ironic.
LLM outputs are 100% deterministic.
If you enter the same prompt with the same seed you will get the same vector outputs.
Chatbots take those vector outputs and treat them as a distribution and select a random token. This isn’t a property of the LLMs, it’s a property of chatbots.
If they aren’t selling non-chatbot LLMs then that’s irrelevant.
The entire comment chain is about LLMs, not chatbots.
A LLM’s output is deterministic.
A from an IT system point of view, a system using LLMs is only non-deterministic if the system adds randomness. The randomness isn’t inherent to the LLM and any added randomness is typically created by a seed which could be replicated across number of system images leading to deterministic output across every image clone.
From the point of view of the typical user the computer may as well be a magical box controlled by a tiny fairy so any talk of deterministic output is irrelevant.
The post is about the actual product they’re selling, not whatever idealized idea of what a ‘proper’ LLM is.
If every LLM sold is sold as a chatbot, then this “ummm ackchully” is irrelevant. It doesn’t matter what LLMs are technically capable of if none of the LLMs sold actually function that way.
Yes, that is what the post is about.
You didn’t click on reply to the post, you clicked reply under my comment.
In my comment, I was talking about an LLM (I checked with myself) and the other person was also talking about LLMs and on up to the top of the comment chain where we started talking about LLMs in IT systems.
From the context of the conversation, you should understand that we’re talking about LLMs, specifically being in IT having to deal with LLMs. The context should tell you that we’re talking about the actual language models and not the end user applications, like a chatbot.
Ok, well this is easy then. Every LLM isn’t sold as a chatbot so I’m not sure why you keep repeating this like it is a point.
Your first comment was ‘ummmm ackchully LLMs are only chatbots’ which is both wrong and ironic.