In addition to what the other reply says, the current state of AI isn’t necessarily the best AI could be. Even with the iterative changes on the LLM-based model, things are improving so fast that it might be safe to shrink the workforce for technical tasks soon.
But I’m sure I’m not the only one that thinks the LLM-focused approach itself is just a local minimum the industry is stuck trying to optimize while another approach that isn’t just a big data “throw everything we can at it and hope it spits out useful results” but something more methodological that encodes our knowledge from experts to give it a head start as well as robust reasoning strategies and logic to let it improve on that starting point as it seeks and adds relevant data in ways similar to how we do science and engineering.
I believe that it’s a race between an AI that truly can outcompete us and societal collapse, because the real reason AI is more difficult to stop than those other three is how easy it is to hide development. The massive data centers are required for the current approach being scaled up for the world to use it. AI research and development can be done on home PCs, especially if you’re more interested in results than speed (in which case you aren’t limited by cores or memory but just by storage and time).
Eh it’s the illusion of speed. Scaling brought enormous returns from GPT-3 -> GPT-4 but it’s been far less significant for every major release since. To compensate for this, every research lab is coming up with new ways to extract value of it of models: CoT, RL, Agent Harness etc
However, these are all hacks to make LLMs more efficient or (try) to make them more reliable. They still have significant drawbacks which will take years (probably decades) to ever get them to the point where they can reliably replace knowledge workers. China knows this and is taking a far different approach to LLM development (not a tankie fyi). Scaling is a horrible idea which will burn billions of dollars with an astronomically low chance of return.
Yeah, while I have some doubts, I believe that LLMs have fundamental issues that will always hold them back. The doubts come because Claude Code seems like they’ve built a system where they are effective at giving it a good context, and it has relatively quickly solved some annoying obscure issues with my environment that I was unable to make any progress on my own with and other LLMs were also useless for.
I still think it’s a series of patches/bandaids to cover up those flaws, but my doubt comes in the form of “what if those patches can get it to average human level or even skilled”. I don’t think LLMs can get to the true innovator level like Einstein and Tesla, but doing competent work is well below that level and at this point I think LLMs might be able to get there.
And I think other approaches could do even better. Not that I know what they are, but just based on the assumption that we haven’t found the ideal approach in the still infancy of what AI could be.
Edit: Funny enough but the current/recent advancements seem to be aimed at eliminating the job of “prompt expert” first.
In addition to what the other reply says, the current state of AI isn’t necessarily the best AI could be. Even with the iterative changes on the LLM-based model, things are improving so fast that it might be safe to shrink the workforce for technical tasks soon.
But I’m sure I’m not the only one that thinks the LLM-focused approach itself is just a local minimum the industry is stuck trying to optimize while another approach that isn’t just a big data “throw everything we can at it and hope it spits out useful results” but something more methodological that encodes our knowledge from experts to give it a head start as well as robust reasoning strategies and logic to let it improve on that starting point as it seeks and adds relevant data in ways similar to how we do science and engineering.
I believe that it’s a race between an AI that truly can outcompete us and societal collapse, because the real reason AI is more difficult to stop than those other three is how easy it is to hide development. The massive data centers are required for the current approach being scaled up for the world to use it. AI research and development can be done on home PCs, especially if you’re more interested in results than speed (in which case you aren’t limited by cores or memory but just by storage and time).
Eh it’s the illusion of speed. Scaling brought enormous returns from GPT-3 -> GPT-4 but it’s been far less significant for every major release since. To compensate for this, every research lab is coming up with new ways to extract value of it of models: CoT, RL, Agent Harness etc
However, these are all hacks to make LLMs more efficient or (try) to make them more reliable. They still have significant drawbacks which will take years (probably decades) to ever get them to the point where they can reliably replace knowledge workers. China knows this and is taking a far different approach to LLM development (not a tankie fyi). Scaling is a horrible idea which will burn billions of dollars with an astronomically low chance of return.
Yeah, while I have some doubts, I believe that LLMs have fundamental issues that will always hold them back. The doubts come because Claude Code seems like they’ve built a system where they are effective at giving it a good context, and it has relatively quickly solved some annoying obscure issues with my environment that I was unable to make any progress on my own with and other LLMs were also useless for.
I still think it’s a series of patches/bandaids to cover up those flaws, but my doubt comes in the form of “what if those patches can get it to average human level or even skilled”. I don’t think LLMs can get to the true innovator level like Einstein and Tesla, but doing competent work is well below that level and at this point I think LLMs might be able to get there.
And I think other approaches could do even better. Not that I know what they are, but just based on the assumption that we haven’t found the ideal approach in the still infancy of what AI could be.
Edit: Funny enough but the current/recent advancements seem to be aimed at eliminating the job of “prompt expert” first.