jeffw@lemmy.world to Technology@lemmy.worldEnglish · 2 days agoOpenAI hits back at DeepSeek with o3-mini reasoning modelarstechnica.comexternal-linkmessage-square15fedilinkarrow-up168
arrow-up168external-linkOpenAI hits back at DeepSeek with o3-mini reasoning modelarstechnica.comjeffw@lemmy.world to Technology@lemmy.worldEnglish · 2 days agomessage-square15fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up1·1 day agoDense models that would fit in 100-ish GB like mistral large would be really slow on that box, and there isn’t a SOTA MoE for that size yet. So, unless you need tons of batching/parallel requests, its… kinda neither here nor there? As someone else said, the calculus changes with cheaper Strix Halo boxes (assuming those mini PCs are under $3K).
Dense models that would fit in 100-ish GB like mistral large would be really slow on that box, and there isn’t a SOTA MoE for that size yet.
So, unless you need tons of batching/parallel requests, its… kinda neither here nor there?
As someone else said, the calculus changes with cheaper Strix Halo boxes (assuming those mini PCs are under $3K).