Eh it’s the illusion of speed. Scaling brought enormous returns from GPT-3 -> GPT-4 but it’s been far less significant for every major release since. To compensate for this, every research lab is coming up with new ways to extract value of it of models: CoT, RL, Agent Harness etc
However, these are all hacks to make LLMs more efficient or (try) to make them more reliable. They still have significant drawbacks which will take years (probably decades) to ever get them to the point where they can reliably replace knowledge workers. China knows this and is taking a far different approach to LLM development (not a tankie fyi). Scaling is a horrible idea which will burn billions of dollars with an astronomically low chance of return.
Yeah, while I have some doubts, I believe that LLMs have fundamental issues that will always hold them back. The doubts come because Claude Code seems like they’ve built a system where they are effective at giving it a good context, and it has relatively quickly solved some annoying obscure issues with my environment that I was unable to make any progress on my own with and other LLMs were also useless for.
I still think it’s a series of patches/bandaids to cover up those flaws, but my doubt comes in the form of “what if those patches can get it to average human level or even skilled”. I don’t think LLMs can get to the true innovator level like Einstein and Tesla, but doing competent work is well below that level and at this point I think LLMs might be able to get there.
And I think other approaches could do even better. Not that I know what they are, but just based on the assumption that we haven’t found the ideal approach in the still infancy of what AI could be.
Edit: Funny enough but the current/recent advancements seem to be aimed at eliminating the job of “prompt expert” first.
Eh it’s the illusion of speed. Scaling brought enormous returns from GPT-3 -> GPT-4 but it’s been far less significant for every major release since. To compensate for this, every research lab is coming up with new ways to extract value of it of models: CoT, RL, Agent Harness etc
However, these are all hacks to make LLMs more efficient or (try) to make them more reliable. They still have significant drawbacks which will take years (probably decades) to ever get them to the point where they can reliably replace knowledge workers. China knows this and is taking a far different approach to LLM development (not a tankie fyi). Scaling is a horrible idea which will burn billions of dollars with an astronomically low chance of return.
Yeah, while I have some doubts, I believe that LLMs have fundamental issues that will always hold them back. The doubts come because Claude Code seems like they’ve built a system where they are effective at giving it a good context, and it has relatively quickly solved some annoying obscure issues with my environment that I was unable to make any progress on my own with and other LLMs were also useless for.
I still think it’s a series of patches/bandaids to cover up those flaws, but my doubt comes in the form of “what if those patches can get it to average human level or even skilled”. I don’t think LLMs can get to the true innovator level like Einstein and Tesla, but doing competent work is well below that level and at this point I think LLMs might be able to get there.
And I think other approaches could do even better. Not that I know what they are, but just based on the assumption that we haven’t found the ideal approach in the still infancy of what AI could be.
Edit: Funny enough but the current/recent advancements seem to be aimed at eliminating the job of “prompt expert” first.