So I have a framework laptop with ryzen ai cpu that uses 48gb of shared ram, and it does run Q4 llms fine enough, but I’m not sure it compares to a real GPU.
On my desktop I have an RX 7900 XTX but I’ve only dabbled in image generation so far, so right now I couldn’t really tell you the difference.
Yeah I was lucky to buy a 6900XT when it was near the lowest price, so I sold that and added a couple hundred for the 7900, seemed like a good future-proofing move, given the times we’re living in.
I think Apple silicon is faster than my generation of Ryzen, but the newest (Strix something?) with the LPDDRGGFASEWARGH5 memory should be faster yet. Of course buying all that memory right now would be quite painful.
So I have a framework laptop with ryzen ai cpu that uses 48gb of shared ram, and it does run Q4 llms fine enough, but I’m not sure it compares to a real GPU.
On my desktop I have an RX 7900 XTX but I’ve only dabbled in image generation so far, so right now I couldn’t really tell you the difference.
24 GB VRAM. Damn, jealous! My 16 GB seems pitiful in comparison 😅
I do wonder if the Ryzen AI CPUs compare with Apple’s UMA. I’m mostly interested in LLM inference for code generation and automation.
Yeah I was lucky to buy a 6900XT when it was near the lowest price, so I sold that and added a couple hundred for the 7900, seemed like a good future-proofing move, given the times we’re living in.
I think Apple silicon is faster than my generation of Ryzen, but the newest (Strix something?) with the LPDDRGGFASEWARGH5 memory should be faster yet. Of course buying all that memory right now would be quite painful.