If I’m having a direct discussion with somebody, then I’ll make it clear what I’m talking about, and if they engage in a conversation about ethics, I’ll explain why those things are unethical.
But if I wanted to post a public message to voice my sentiment, I don’t think trying to post a long explanation every time would do any good, most people would probably just skip it when they saw it’s longer than a couple lines of text. So for those cases, I’d say you have to go down to the level of current discourse to engage with the masses on their terms if you want to be heard.
But if I wanted to post a public message to voice my sentiment, I don’t think trying to post a long explanation every time would do any good
That’s a non sequitur. The initial point was “Stop using this term incorrectly, because it spreads misunderstanding and misinformation.”
Nobody said you should write a thesis per reply, and my entire point was to focus on the harms, and the actors, not the tech.
Saying “AI is unethical,” is a truly nonsensical statement. Even if we grant that AI is shorthand for LLM, is still isn’t justified as all I need to do is find an instance of an open source model being trained on public data to refute it, and there are MANY.
Unless you think that using public data to make public software is inherently unethical, which is a hard stance to justify.
If you’re upset with Sam Altman and ChatGPT, then you can simply say “Sam Altman is a thief, and ChatGPT uses stolen data.” Which is infinitely more defendable than “AI is trained on stolen data and is unethical.”
If you’re upset with Sam Altman and ChatGPT, then you can simply say “Sam Altman is a thief, and ChatGPT uses stolen data.” Which is infinitely more defendable than “AI is trained on stolen data and is unethical.”
The issue is, I’d say the same applies to every model that produces useful outputs. LLaMa, Anthropic, Grok, DeepSeek, whatever else is out there. If you tell somebody that the LLM they’re using is unethical, they’ll nust go use another convenient corporate model.
And “public data” is not enough for me, because that typically means scraping copyrighted content from public websites, I’m not aware of a model that uses only data with permission (either explicit or granted by the license) that’s useful, and the people who need to hear about ethical problems with GenAI especially don’t know about that.
The issue is, I’d say the same applies to every model that produces useful outputs.
That you know of. Your lack of awareness is not an indication on the stance of the technology itself. If you have a problem with Grok, as I do, then condemn Grok, Twitter, and that pathetic man child that owns them.
If you tell somebody that the LLM they’re using is unethical, they’ll nust go use another convenient corporate model.
Yeah, so actively fight against these toxic corporate activities. Fuck OpenAI. Fuck Google. Fuck Twitter. But the instant negative reaction to their shared tool is getting quite a few additional people caught in the blast.
And “public data” is not enough for me, because that typically means scraping copyrighted content from public websites, I’m not aware of a model that uses only data with permission
Again, this is an expression of your ignorance, not reality.
OLMo 2 was trained on Wikipedia and other fully public forums. Its training sources and data is fully accessible and open.
GPT-NeoX is an untrained model that you can train yourself. It’s literally just the foundations anybody could use.
Pythia is fully open source, and one of its training sources was GitHub. Which does genuinely bring in to question whether or not you can use GitHub to train AI. Personally, I don’t see how branching a repository is any different than using the code to train a model. But the current anti-AI trend has people EXTREMELY sensitive to this concept.
Again, this is an expression of your ignorance, not reality.
OLMo 2 was trained on Wikipedia and other fully public forums. Its training sources and data is fully accessible and open.
Decided to check out the first example quickly. It’s hard to dig through the information, but following the chain of sources:
In the first stage, which covers over 90% of the total pretraining budget, we use the OLMo-Mix-1124, a collection of approximately 3.9 trillion tokens sourced from DCLM, Dolma, Starcoder, and Proof Pile II.
As part of DCLM, we provide a standardized corpus of 240T tokens extracted from Common Crawl
So, it’s using data from Common Crawl. What data, exactly? That’d be harder to dig up. DCLM has a repository, but they don’t make an effort to point out how, or if, they’re filtering the data.
What I can quickly find is information from Common Crawl itself, which is the ultimate source of data. On that I can immediately see only two things:
They have an opt-out list, and from what I understand, they’re scraping websites by default, providing mechanisms for you to block them or opt-out… If you’re even aware they exist.
According to their stats page, they have 313423 pages scraped from github.com. I doubt github has that many pages of non-user-generated content, so the question is… What data is going in there, who wrote it, and did anybody agree to that? What about other websites from those top domains, such as blogspot.com, wordpress.org, readthedocs.io? I doubt they got permission to use the 17175161 pages of content from blogspot they’ve scraped.
So yeah, maybe there’s a “good” model out there I’d actually accept, but I’ve seen “open” models being released, and I can’t possibly check all of them, just checking the websites for one of the sources for one of the models took 20 minutes here, if I wanted to verify this properly I’d have to setup the tooling to query the terabytes of data for this source, and all others, and even there I’m not sure if I’d find answers.
Okay, so you actually take issue with internet scraping itself? Let me get things straight, do you have a problem with LLMs themselves, or the data used to train them? It seems like an obvious “Yeah, the data is the issue,” to me but I want to clarify because I HAVE seen the other stance before.
And if it is just the data, does that mean your REAL issue is the act of internet scraping itself? Because then you’re also going to have to condemn every search engine and indexing algorithm in the world and go back to pre-indexing days when we just typed in the URL as the only method.
Or is it the monetisation that you don’t like? Because then the models I provided should be exempt because they’re 100% free.
If I’m having a direct discussion with somebody, then I’ll make it clear what I’m talking about, and if they engage in a conversation about ethics, I’ll explain why those things are unethical.
But if I wanted to post a public message to voice my sentiment, I don’t think trying to post a long explanation every time would do any good, most people would probably just skip it when they saw it’s longer than a couple lines of text. So for those cases, I’d say you have to go down to the level of current discourse to engage with the masses on their terms if you want to be heard.
That’s a non sequitur. The initial point was “Stop using this term incorrectly, because it spreads misunderstanding and misinformation.”
Nobody said you should write a thesis per reply, and my entire point was to focus on the harms, and the actors, not the tech.
Saying “AI is unethical,” is a truly nonsensical statement. Even if we grant that AI is shorthand for LLM, is still isn’t justified as all I need to do is find an instance of an open source model being trained on public data to refute it, and there are MANY.
Unless you think that using public data to make public software is inherently unethical, which is a hard stance to justify.
If you’re upset with Sam Altman and ChatGPT, then you can simply say “Sam Altman is a thief, and ChatGPT uses stolen data.” Which is infinitely more defendable than “AI is trained on stolen data and is unethical.”
The issue is, I’d say the same applies to every model that produces useful outputs. LLaMa, Anthropic, Grok, DeepSeek, whatever else is out there. If you tell somebody that the LLM they’re using is unethical, they’ll nust go use another convenient corporate model.
And “public data” is not enough for me, because that typically means scraping copyrighted content from public websites, I’m not aware of a model that uses only data with permission (either explicit or granted by the license) that’s useful, and the people who need to hear about ethical problems with GenAI especially don’t know about that.
That you know of. Your lack of awareness is not an indication on the stance of the technology itself. If you have a problem with Grok, as I do, then condemn Grok, Twitter, and that pathetic man child that owns them.
Yeah, so actively fight against these toxic corporate activities. Fuck OpenAI. Fuck Google. Fuck Twitter. But the instant negative reaction to their shared tool is getting quite a few additional people caught in the blast.
Again, this is an expression of your ignorance, not reality.
OLMo 2 was trained on Wikipedia and other fully public forums. Its training sources and data is fully accessible and open.
GPT-NeoX is an untrained model that you can train yourself. It’s literally just the foundations anybody could use.
Pythia is fully open source, and one of its training sources was GitHub. Which does genuinely bring in to question whether or not you can use GitHub to train AI. Personally, I don’t see how branching a repository is any different than using the code to train a model. But the current anti-AI trend has people EXTREMELY sensitive to this concept.
Decided to check out the first example quickly. It’s hard to dig through the information, but following the chain of sources:
So, it’s using data from Common Crawl. What data, exactly? That’d be harder to dig up. DCLM has a repository, but they don’t make an effort to point out how, or if, they’re filtering the data.
What I can quickly find is information from Common Crawl itself, which is the ultimate source of data. On that I can immediately see only two things:
github.com. I doubt github has that many pages of non-user-generated content, so the question is… What data is going in there, who wrote it, and did anybody agree to that? What about other websites from those top domains, such asblogspot.com,wordpress.org,readthedocs.io? I doubt they got permission to use the 17175161 pages of content from blogspot they’ve scraped.So yeah, maybe there’s a “good” model out there I’d actually accept, but I’ve seen “open” models being released, and I can’t possibly check all of them, just checking the websites for one of the sources for one of the models took 20 minutes here, if I wanted to verify this properly I’d have to setup the tooling to query the terabytes of data for this source, and all others, and even there I’m not sure if I’d find answers.
Okay, so you actually take issue with internet scraping itself? Let me get things straight, do you have a problem with LLMs themselves, or the data used to train them? It seems like an obvious “Yeah, the data is the issue,” to me but I want to clarify because I HAVE seen the other stance before.
And if it is just the data, does that mean your REAL issue is the act of internet scraping itself? Because then you’re also going to have to condemn every search engine and indexing algorithm in the world and go back to pre-indexing days when we just typed in the URL as the only method.
Or is it the monetisation that you don’t like? Because then the models I provided should be exempt because they’re 100% free.