my experience has been similar for complex tasks but the sweet spot for me is small, well-defined scripts where i can verify the output easily.

like: i needed to parse some music metadata and normalize it across a few formats. gave it a spec, it produced something that mostly worked. i spent maybe 20 minutes fixing edge cases instead of 3 hours writing it from scratch. that exchange of time is what sold me.

for 3d math where correctness is hard to verify at a glance, i wouldn’t trust it either. the tool is only as useful as your ability to test its output quickly.

my experience with LLM’s and numerical computations like with MATLAB or GNU octave, has been poor. I assume its more of an issue that the data isn’t there, MATLAB has it’s own proprietary AI (which I don’t believe is trained on users code) and Octave has no AI associated on it’s end so the major LLM’s only get trained by the data it is prompted by users online or otherwise. Which is why if you prompt it to do a 3D plot, it will almost always pull something out of it’s ass.

your feeling of a “mental-fog” is my experience with AI in general, the language model explains the ideas well, but then the code editor does some obscure move that makes no fucking sense. also, because you’re not programming it and learning from your mistakes it makes you uncertain of your code. its unfortunate to see search engines are going to shit because of AI, because AI is not ready.

The solutions it generated were almost write every time

Did you vibe code this post? 😂

I have a full pro model for Kiro at work. It does actually work, but we have custom MCP servers for all the internal tools, context on how to use these tools, style guidelines, etc. and then on top of that we have a lot of AI context files in the code base to help the AI understand the code base and make the correct changes.

I’ve been using it on a side project and it works if you know how to constrain it. It does get things wrong a lot. But the big thing about it is doing spec driven development where you give it a write up and it makes a requirements doc and a design doc with a lot of correctness properties in them to follow when generating and making the tasks.

I don’t believe people can vibe code unless they can actually code. It’s a whole different way of coding. I still manually edit what it does a lot.

A lot of people explain it like it’s a brand new junior developer. You need to give it as much context as possible, tell it to exactly what you want, tell it what you don’t want, tell it why, etc. and it still may not listen exactly.

I recently started using Pro to debug a problem I couldn’t solve. The one thing I need from it is an extra insight, a second opinion (because I’m the only developer), and it allowing me to let it read the whole folder helps, it identified a problem I didn’t consider because it’s a file outside of where I was looking.

You just didn’t use the right prompts!!!

/s

Also working on some 3d maths.

I’ve used the free versions a bit, but not really to the extent that I’d call it vibe coding. The chat bots often know where to find libraries or pre-existing functions that I don’t know. It’s also okay at algorithms for well defined problems, but it often says be careful not to do something I absolutely need to do or visea versa. It’s very hit and miss on debugging. It’ll point out obvious stuff (typos) reliably, and it can do some iteration stuff usually, but it usually doesn’t pick up on other things. Once in a rare while it will impress me by suggesting I look at a particular thing, and I think it manages this better in new chats, but most complex issues fail for it. I use it as a faster stackoverflow, but you need to be able to work through the code yourself, understand what you’re doing, and test that individual steps are doing what they need to do. The bots can’t really do any sort of planning or breaking down a problem into sub-problems, and they really suck at thinking about 3d stuff.

I haven’t tried any Anthropic models personally.

So far, between the free online chats by OpenAI and DeepSeek, and the smaller models I’ve run on my own machine, the most useful things I have gotten from it were to treat it as an overeager student that lacks the first-hand experience needed to see the big picture, asking it questions that I’m pretty sure I already know the answer to and seeing if 1) it “understands” what I’m getting at and 2) it can surprise me with a viewpoint I hadn’t thought of before.

Using them to double-check my own ideas seems to be marginally useful, especially when there’s no qualified human being whose attention I can borrow. Using them as a sort of semantic web search can sometimes get me what I’m looking for faster than Google. If anything, they’re an opportunity to exercise critical thinking; if I can tell where it’s getting things wrong I can be fairly confident that my own understanding of the problem/subject is pretty solid.

Vibe coding, though? I have yet to see it work out. Maybe as some starting slop so that I can get to work refactoring code (and get the ideas flowing) instead of staring at a blank file.

It’s a tool that you need to learn. Try some of claude.md files people share online for your programming area as a starter. You still need to review what it does but just asking for it to create tests as it creates code does a lot to improve output.

I use AI for researching what existing software or projects exist to help my build up my system that I then suffer through making.

That’s been my experience. It’s always subtlely wrong, its solutions are hard to maintain, and if you spend too much time with it, it starts forgetting what you said earlier. Managers don’t understand the distinction, they already can’t code well, and only test it in small problems where it’s not context-limited, so they’re amazed.

I use it and it works. It doesn’t give you the right result in one shot, but neither does manual coding. You iterate and prompt again and again. In the end, it saves a ton of time. Engineers are definitely going to lose their jobs because fewer people are needed. I know its tough to accept this and people will go through denial. Part of that is saying the AI code is junk. But, you’ll find it can produce junk and quickly fix it into the right solution faster than an engineer can. It sucks, but this is the new reality. The one thing that is cool once you embrace it is that you realize you can customize your favorite apps or even build anything you want from scratch.

producing subtly broken junk

The difference between you and people that say it’s amazing is that you are capable of discerning this reality.

No, I think you do get it. That’s exactly right. Everything you described is absolutely valid.

Maybe the only piece you’re missing is that “almost right, but critically broken in subtle ways” turns out to actually be more than good enough for many people and many purposes. You’re describing the “success” state.

/s but also not /s because this is the unfortunate reality we live in now. We’re all going to eat slop and sooner or later we’re going to be forced to like it.

Vibe coding, in the sense of telling the model to make codebase changes, then directly using the output produced, is 100% marketing bullshit that does not scale beyond toy examples.

Here’s the rub: Claude is extremely useful as an advanced autocomplete, if and only if you’re guiding it architecturally through every task it runs, and you vet + revise the output yourself between iterations. You cannot effectively pilot entirely from chat in a mature codebase, and you must compile robust documentation and instructions for Claude to know how to work with your codebase.

You also must aggressively manage information in the context window yourself and keep it clean. You mentioned going in circles trying to get the robot to correct itself: huge mistake. Rewind to before the error, and give it better instructions to steer it away from the pitfall it fell into. Same vein, you also need to reset ASAP after pushing into the >100k token mark, because the models start melting into putty soon after (yes, even the “extended” 1M-window ones).

I’m someone who has massively benefited from using modern LLMs in my work, but I’m also a massive hater at the same time: They’re just a tool, not magic, and have to be used with great care and attention to get reasonable results. You absolutely cannot delegate your thinking to them, because it will bite you, hard and fast.

For your use case (3D math), what I recommend is decomposing your end goal into a series of pure functions that you’ll string together. Once you have that list, that’s where Claude comes in. Have it stub those functions for you, then have it implement them one at a time, reviewing the output of every one before proceeding.