Because they all use the illegal Colossus 2 data centre from SpaceX/XAi/fascism central and that data centre went down.
Google, Anthropic, and OpenAI all have contracts with them.
When that illegally running environmental disaster of data centre goes down, all those services all go over capacity and you get 502 rate limit errors.
Generally no, because most hosting companies would have something baked into SLA/SLO contracts, but all of this shit is done so illegally and shadily now, I wouldn’t put a hard “no” on that possibility.
Not true. I’ve been involved in litigation with both AWS and Google over unsecured comms that were defined as being TLS secured in SLA/SLO contracts and found not to be. Not that anything nefarious was happening necessarily, but the expectation is clear.
Whether these asshats even check for such requirements with a Musk run company right now 🤷
It COULD possibly be that they are logging every exchange happening at the network fabric between the service layers, but nobody knows unless they intentionally take steps to investigate or accidentally prove it.
You’re missing the point, nobody is talking about trusting these companies.
You can tell if a connection is encrypted end to end or not. And if your paying for that service you can sue if you aren’t receiving what your paying for.
In digital security nobody relies on trust if they can help it. And security matters if you want to keep a competitive edge on your competition, so even shitty companies care about that.
I was not talking about your specific lawsuit, or TLS.
Companies trust other companies all the time, and it’s foundational to most all SLA/SLOs. Any time I’ve voiced concerns around how AI companies are using the data we are giving them (like giving them access to our codebase), it’s brushed off as “we have an agreement with them”. It’s just a load of hogwash. They can and will abuse all data they have access to, just as they have done thus far.
In this particular case, we are talking about a data center, and it is not at all reasonable to assume that the data that flows to said data center is in any way protected, especially one run by Musk.
Executives are the biggest cheaters out there. They will only comply if it will hurt them if they don’t, and it’s won’t here, as long as the protection money is produced they don’t have to worry.
I agree with that. But I think “the right” data scientists can infer too much from all those AI queries, and I think Elon can abuse that for his own gain.
The numbers of this madness are fucking scary, 1.5 GW of compute power, and huge arrays of cooler to keep it operating. It’s basically a 1.5GW heater in the open. Oh and gas turbines to provide (part of) the electricity. Capitalism is burning this world down for a buck.
We can’t really produce enough heat through industry to affect the earth globally, if that’s what you meant. It’s insignificant in comparison to what the Sun provides.
However there is undeniably localised issues caused by these insane structures.
They’re oversold though, especially prompt caching and the parameter count war
The US model is that they think more training compute and parameters will result in the winning model, while the Chinese are focusing on RL and parameters efficiency due to compute limits.
The efficiency of Chinese models really is impressive. I generated sooo much code yesterday with Qwen3.6 35B-A3B running on an RTX 5060 Ti 16GB (+ a little CPU offloading). It got the jobs done at ~50 tokens/sec.
(It’s not super complex code, just some scripts that I would not have taken to time to write manually.)
I’d love to upgrade to something with more VRAM, but even my current card has doubled in price since I bought it last year 😬
I meant your LLM stack lol. I just have an RX 6800 XT in my main Linux PC for inference, but it has to share VRAM with the DE. Maybe I’ll set it up for remote development from my laptop instead to free up VRAM.
What are you using? vLLM? llama.cpp? Which params? How much CPU offloading? Do you use draft models? Is it a MoE model? Have you tried llama-swap? Which agentic front-end are you using? I presume you set it up to access it without SSH’ing into the machine, did you do anything special or is it just a raw unsecured open port on the machine to the LAN?
Ohhh lol. Yeah it’s Llama-swap, running llama.cpp for now, but might add vLLM to the llama-swap config to experiment with NVFP4.
I mainly use MoE models so I can get decent speed while using a 150-200k context window. My go-to model has been Qwen3.6 35B-A3B for a while. I tried Qwen3.8 27B, but it was too slow.
Gemma4 26B-A4B also runs nice and fast, but I generally get better results from Qwen3.6. I don’t remember exactly how much CPU offloading is happening, but it’s not much. As long as I can get like 40-50 tokens/sec, I’m usually satisfied enough.
For the coding harness, I’ve been running Pi in an Apple Container (sort of like Podman, but better isolation in a microvm). Though, I recently configured VS Code to use LLMs on my server, and it was actually pretty decent. Still need to explore a bit more, but so far VS Code’s AI capabilities seem much better than they were a year ago (they seemed way behind, back then).
Also, I don’t connect any harness directly to llama-swap. I have another container running Caddy, which acts as a gateway to AI providers. For other services (e.g. OpenRouter), the API key is injected in the Caddy container. I don’t like having API keys or secrets anywhere where LLMs can read them. It’s not so bad for my own self-hosted LLMs, but not cool to send secrets to a server owned by someone else.
How has tool use been for you? I struggled a lot with tool use with Gemma and Qwen, to the point where I needed to build a healing layer.
Regarding the coding harness, I was looking for something CLI-based or JetBrains-based, and I haven’t had much luck getting my local llama.cpp models playing ball with OpenCode. They keep losing context and misusing tools.
I’m not too familiar with Apple containers as I’m running a full Linux stack, but I’ll give Pi a try, seems interesting! Does it work for coding tasks or is it strictly an “orchestrator”?
Because they all use the illegal Colossus 2 data centre from SpaceX/XAi/fascism central and that data centre went down.
Google, Anthropic, and OpenAI all have contracts with them.
When that illegally running environmental disaster of data centre goes down, all those services all go over capacity and you get 502 rate limit errors.
Ah, so Colossus 2 is the target. Remember that, everyone.
Does that mean Musk’s businesses have access to everything people do on the systems that use their data centers?
Generally no, because most hosting companies would have something baked into SLA/SLO contracts, but all of this shit is done so illegally and shadily now, I wouldn’t put a hard “no” on that possibility.
Nobody can verify that those “contracts” are actually being honored. We are all assuming the fascists will self police themselves out of good will.
Not true. I’ve been involved in litigation with both AWS and Google over unsecured comms that were defined as being TLS secured in SLA/SLO contracts and found not to be. Not that anything nefarious was happening necessarily, but the expectation is clear.
Whether these asshats even check for such requirements with a Musk run company right now 🤷
It COULD possibly be that they are logging every exchange happening at the network fabric between the service layers, but nobody knows unless they intentionally take steps to investigate or accidentally prove it.
deleted by creator
Still means jack shit. Companies will lie through their teeth and violate any and all they can get away with.
They are not to be trusted.
You’re missing the point, nobody is talking about trusting these companies.
You can tell if a connection is encrypted end to end or not. And if your paying for that service you can sue if you aren’t receiving what your paying for.
In digital security nobody relies on trust if they can help it. And security matters if you want to keep a competitive edge on your competition, so even shitty companies care about that.
I was not talking about your specific lawsuit, or TLS.
Companies trust other companies all the time, and it’s foundational to most all SLA/SLOs. Any time I’ve voiced concerns around how AI companies are using the data we are giving them (like giving them access to our codebase), it’s brushed off as “we have an agreement with them”. It’s just a load of hogwash. They can and will abuse all data they have access to, just as they have done thus far.
In this particular case, we are talking about a data center, and it is not at all reasonable to assume that the data that flows to said data center is in any way protected, especially one run by Musk.
I hope you’re right, but Elon don’t strike me as the guy whose most compliant to SLA, law, or anything else that might be an inconvenience to him.
He can’t micromanage everything and regular management will try to comply.
Executives are the biggest cheaters out there. They will only comply if it will hurt them if they don’t, and it’s won’t here, as long as the protection money is produced they don’t have to worry.
I highly doubt that he really knows that much, either.
I agree with that. But I think “the right” data scientists can infer too much from all those AI queries, and I think Elon can abuse that for his own gain.
Impossible to know.
There are systems for doing things cryptographically secure, but I don’t know much about that.
Cryptography? Security? FROM A GENAI COMPANY?!
Well yeah, for themselves
Homomorphic encryption? Much slower than “trust me, bro”.
And where is said data center? For research purposes
https://epoch.ai/data/ai-data-centers/directory/colossus-2 says “Memphis, Tennessee, United States” but also provides satellite imagery
The numbers of this madness are fucking scary, 1.5 GW of compute power, and huge arrays of cooler to keep it operating. It’s basically a 1.5GW heater in the open. Oh and gas turbines to provide (part of) the electricity. Capitalism is burning this world down for a buck.
Today I learned that “AI” data centers use more energy than time travel.
i bet they will also travel us to the apocalypse much faster
We can’t really produce enough heat through industry to affect the earth globally, if that’s what you meant. It’s insignificant in comparison to what the Sun provides.
However there is undeniably localised issues caused by these insane structures.
Probably the gas turbines producing C02 and waste methane.
I’m not sure that would effect inference on an already trained model. Data sets are primarily used in training.
It’s kinda surprising,
I know specifically where one of the big ones hosts its models and its not there, but I guess they could have infrastructure in there.
They’re oversold though, especially prompt caching and the parameter count war
The US model is that they think more training compute and parameters will result in the winning model, while the Chinese are focusing on RL and parameters efficiency due to compute limits.
Considering which country is better at building power stations, that’s a fascinating dichotomy.
The US going for brute force when the brute force is more available in China… Priceless irony.
China doesn’t have the near the amount of compute resources, but they can run less efficient servers for cheaper, so it’s a wash.
The efficiency of Chinese models really is impressive. I generated sooo much code yesterday with Qwen3.6 35B-A3B running on an RTX 5060 Ti 16GB (+ a little CPU offloading). It got the jobs done at ~50 tokens/sec.
(It’s not super complex code, just some scripts that I would not have taken to time to write manually.)
I’d love to upgrade to something with more VRAM, but even my current card has doubled in price since I bought it last year 😬
Show me your set up!
There’s not really anything interesting to show. It’s just a home server in a 13 year old desktop ATX case.
There’s no desk, monitor, keyboard, or mouse… But also no cool server rack.
Function over form, and it sits in a spare bedroom out of sight.
EDIT: I found the receipt for the case. It’s a Cougar Volant Black Steel mid tower, purchased in 2013. So my server just looks like this:
Gotta love the black slab.
I meant your LLM stack lol. I just have an RX 6800 XT in my main Linux PC for inference, but it has to share VRAM with the DE. Maybe I’ll set it up for remote development from my laptop instead to free up VRAM.
What are you using? vLLM? llama.cpp? Which params? How much CPU offloading? Do you use draft models? Is it a MoE model? Have you tried llama-swap? Which agentic front-end are you using? I presume you set it up to access it without SSH’ing into the machine, did you do anything special or is it just a raw unsecured open port on the machine to the LAN?
Ohhh lol. Yeah it’s Llama-swap, running llama.cpp for now, but might add vLLM to the llama-swap config to experiment with NVFP4.
I mainly use MoE models so I can get decent speed while using a 150-200k context window. My go-to model has been Qwen3.6 35B-A3B for a while. I tried Qwen3.8 27B, but it was too slow.
Gemma4 26B-A4B also runs nice and fast, but I generally get better results from Qwen3.6. I don’t remember exactly how much CPU offloading is happening, but it’s not much. As long as I can get like 40-50 tokens/sec, I’m usually satisfied enough.
For the coding harness, I’ve been running Pi in an Apple Container (sort of like Podman, but better isolation in a microvm). Though, I recently configured VS Code to use LLMs on my server, and it was actually pretty decent. Still need to explore a bit more, but so far VS Code’s AI capabilities seem much better than they were a year ago (they seemed way behind, back then).
Also, I don’t connect any harness directly to llama-swap. I have another container running Caddy, which acts as a gateway to AI providers. For other services (e.g. OpenRouter), the API key is injected in the Caddy container. I don’t like having API keys or secrets anywhere where LLMs can read them. It’s not so bad for my own self-hosted LLMs, but not cool to send secrets to a server owned by someone else.
How has tool use been for you? I struggled a lot with tool use with Gemma and Qwen, to the point where I needed to build a healing layer.
Regarding the coding harness, I was looking for something CLI-based or JetBrains-based, and I haven’t had much luck getting my local llama.cpp models playing ball with OpenCode. They keep losing context and misusing tools.
I’m not too familiar with Apple containers as I’m running a full Linux stack, but I’ll give Pi a try, seems interesting! Does it work for coding tasks or is it strictly an “orchestrator”?
eh, just
systemctl isolate multi-user.targetCorrect, that’s how I would do it, but then I need another machine to act as a head.
Reads like American vs European/Asian cars
Always ready to try brute force first. And then some other, less palatable options if that doesn’t work, like slightly less brute force.
The Macrohard Memphis site?
And macroharder it seems… Macrohardest next? https://epoch.ai/data/ai-data-centers/directory/colossus-2
Loosely named after MicroSoft (penis).