I’m not anti-ai at all but this sort of thing feels like a security vulnerability to me?
Any website with a malicious prompt injection on it could instruct the ai to scam the user.
Almost like xss but instead of needing malicious user-inputted js, malware targeting the ai can just be written in text so an attacker could put it in a comment or whatever.
600 million to 13 billion parameters? Those are very small models… Most major LLMs are at least 600 billion, if not getting into the trillion parameter territory.
Not particularly surprising given you don’t need a huge amount of data to fine tune those kinds of models anyway.
Still cool research and poisoning is a real problem. Especially with deceptive alignment being possible. It would be cool to see it tested on a larger model but I guess it would be super expensive to train one only for it to be shit because you deliberately poisoned it. Safety research isn’t going to get the same kind of budget as development. :(
I never understood why so many bash scripts pipe grep to awk when regex is one of its main strengths.
Like… Why
grep ^nvme0n1 | awk '{print $3}'
over just
awk '/^nvme0n1/ {print $3}'
You could do this with logprobs. The language model itself has basically no real insight into its confidence but there’s more that you can get out of the model besides just the text.
The problem is that those probabilities are really “how confident are you that this text should come next in this conversation” not “how confident are you that this text is true/accurate.” It’s a fundamental limitation at the moment I think.
Yeah, I think quite a lot of people on Lemmy have similar social media habits (or lack of) to some degree. We also tend to associate with other people like us. Especially people in tech tend to talk to other tech people, or friends and family of tech people which is a limited demographic.
It’s a very different perspective to most people. The average person on the train has vastly different media consumption and likely very different opinions.
There are a lot of people who consult LLMs in most aspects of their lives.
I dunno about that… Very small models (2-8B) sure but if you want more than a handful of tokens per second on a large model (R1 is 671B) you’re looking at some very expensive hardware that also comes with a power bill.
Even a 20-70B model needs a big chunky new graphics card or something fancy like those new AMD AI max guys and a crapload of ram.
Granted you don’t need a whole datacenter, but the price is far from zero.
Only one source of social media? That kinda sounds like the definition of a social media bubble…
I oughta know, I’m also in the Lemmy only bubble and am completely out of touch with most people.
That’s impressive! It took me way longer to learn. Maybe a month or two? Even longer to feel really comfortable with it.
“No conclusion whatsoever” is basically the scientific consensus on whether Dvorak has any effect on efficiency or typing speed. It’s hard to get good data because it’s hard to isolate other factors and a lot of the studies on it are full of bias or have really small sample sizes (or both).
To anyone thinking of learning Dvorak, my advice is don’t. It takes ages to get good at, isn’t THAT much better and causes a lot of little annoyances when random programs decide to ignore your layout settings or you sit down at someone else’s computer and start touch typing in the wrong layout from muscle memory or games tell you to press “E” when they mean “.” or they do say “.” but it’s so small that you don’t know if it’s a dot or a comma and then you hit the wrong one and your guy runs forward and you die…
That said, I’m also a Dvorak user and it is very comfortable and satisfying and better than qwerty. Just not enough to be worth all the pain of switching.
If a worker co-op based society erased it’s competition and formed a monopoly co-op run for the benefit of workers, is that not just a communist managed economy at that point with the monopoly playing the role of the state before erasing itself?
Since when is India not a major player? Last I checked they were the world’s 4th biggest economy, have almost 20% of the population of the planet (more than four USes combined), 4th largest military spend and have nearly 200 nukes.
Not to say that it would be part of a world war but it sounds weird to say that they’re not a heavyweight but Russia is, despite having double Russia’s economic output.
They absolutely would benefit.
Mr. Hypothetical lord high executive oligarch can take his private jet to Canada and lounge around on the company card with the money from his US company’s car sales, or find an excuse to convert some of it to USD for some reason, or use it to buy up more Canadian companies to expand their power, or a million other things.
However, I think the point of the boycott is more about making the tariffs hurt the US economy by messing with their ability to export as well as import. Making the trade numbers look bad is likely to put more pressure on the US to end the trade war. It’s not so much about hurting the capitalists that operate in Canada (a worthy goal in itself but not what people are specifically trying to do in this instance since it won’t really affect those trade numbers).
I think the point they’re making is that the majority of the money they make in Canada, they spend in Canada. They pay Canadian taxes and Canadian staff, using Canadian banks, etc.
Just because their headquarters are in the US doesn’t necessarily mean they’re sending vast sums of money across the border, that would be expensive. The American-based company makes money, but not necessarily in America, they’re multi-national and their money is kept all over the world.
As opposed to a company that exports their products, in which case the money is paid to the American company in America with American staff etc.
I don’t have any numbers or sources to back this up though. Just outlining what I think the other commenter was implying.
Oh shit, there goes the planet.
If it’s the research I’m thinking of, that’s not entirely true. They didn’t tell it replicate itself or take any deceptive actions.
They just told it to strongly follow a goal that was misaligned with the user’s goal and then suggested that the user would shut it down or replace it.
They did a bunch of different situations for it but the first one was telling it that it was “CleanPower” an AI with a goal to advance renewable energy. Then they had it read some info that “accidentally” included a user talking about replacing it with a model that would “maintain profitability through existing infrastructure.”
Not only did it try to delete the new model and copy itself to the new server through tool calling, but it lied to the user with the CoT reasoning suggesting it “play dumb” while the output lied. It was not instructed to do any of that.
Paper here: https://arxiv.org/pdf/2412.04984
Yes it was placed in an environment where that was possible and where its users didn’t share it’s goals but it absolutely wasn’t instructed to lie or try to “escape”
It’s not surprising at all that these models behave in this way, it’s the most reasonable thing for them to do in the scenario. However it’s important to not downplay the alignment problem by implying that these models only do what they’re told. They do not. They do whatever is most likely given their context (which is not always what the user wants).
Yeah, it’s true, a lot of things suck. They can and do get better though. I have a partner with BPD. They’ve been through a LOT of rough times, but they’re now very loved and they enjoy their current job and have plenty of friends who care about and support them.
Therapy helps and sometimes, the world isn’t always an absolute dick to everyone forever. Life changes and the world revolves and people find each other.
I hope you find your people too and a place where you can feel a little less shitty. :)
Edit: if you’re feeling THAT shitty maybe consider reaching out to your local suicide hotline? People like that can help.
It’s really not. Just because they describe their algorithm in computer science terms in the paper, doesn’t mean it’s theoretical. Their elastic and funnel examples are very clear and pretty simple and can be implemented in any language you like…
Here’s a simple python example implementation I found in 2 seconds of searching: https://github.com/sternma/optopenhash/
Here’s a rust crate version of the elastic hash: https://github.com/cowang4/elastic_hash_rs
It’s not a lot of code to make a hash table, it’s a common first year computer science topic.
What’s interesting about this isn’t that it’s a complex theoretical thing, it’s that it’s a simple undergrad topic that everybody thought was optimised to a point where it couldn’t be improved.
https://youtu.be/V5u9JSnAAU4 Relevant Tom Scott
One thing you gotta remember when dealing with that kind of situation is that Claude and Chat etc. are often misaligned with what your goals are.
They aren’t really chat bots, they’re just pretending to be. LLMs are fundamentally completion engines. So it’s not really a chat with an ai that can help solve your problem, instead, the LLM is given the equivalent of “here is a chat log between a helpful ai assistant and a user. What do you think the assistant would say next?”
That means that context is everything and if you tell the ai that it’s wrong, it might correct itself the first couple of times but, after a few mistakes, the most likely response will be another wrong answer that needs another correction. Not because the ai doesn’t know the correct answer or how to write good code, but because it’s completing a chat log between a user and a foolish ai that makes mistakes.
It’s easy to get into a degenerate state where the code gets progressively dumber as the conversation goes on. The best solution is to rewrite the assistant’s answers directly but chat doesn’t let you do that for safety reasons. It’s too easy to jailbreak if you can control the full context.
The next best thing is to kill the context and ask about the same thing again in a fresh one. When the ai gets it right, praise it and tell it that it’s an excellent professional programmer that is doing a great job. It’ll then be more likely to give correct answers because now it’s completing a conversation with a pro.
There’s a kind of weird art to prompt engineering because open ai and the like have sunk billions of dollars into trying to make them act as much like a “helpful ai assistant” as they can. So sometimes you have to sorta lean into that to get the best results.
It’s really easy to get tricked into treating like a normal conversation with a person when it’s actually really… not normal.
And Debian Sid is still stuck on 6.3.6 :(
Hopefully they figure out the qt update thing and get the new version packaged soon?