Until recently, I always bought computers with 16gb RAM. As a developer, this was important, but also necessary for optimal performance. Specially with tools like Docker, and IntelliJ needing the resources they do. I don't mention Chrome, because that is a choice people make. And I make different decisions. 🙂
I am seeing that with the advent of Large Language Models, and integration of GenAI tools in the core operating systems, the need for better models, and more memory will show up very soon. Also, I am seeing that the 70b (10+ gb) models are way better than the 3b (1.5+ gb) ones. I know the small ones are only going to get better from here, but remember that the large ones will benefit from that technology too.
Plus, supporting large context windows will be a requirement, and the "hosted" versions will get increasingly cheaper (like Gmail made "unlimited" email storage mainstream). But, we have to be careful, and remember that one fine day, Google did come back and say that unlimited only means 17gb.
Also, as developers, we tend to work better when we are disconnected ( Airplane mode anyone ? ). So, having a model that works when on the road / plane is always a good idea. And for this reason alone, I prefer running the models on device.
But running these models does not come cheap. With 16gb RAM, I can run the 3b models today, for code completion, blog writing, etc. But, as time passes, I want more out of them. I also have situations where there are multiple models loaded. One for Chat, another for Tab completion. And we will now also get MacOS and Windows run a LLM for OS features. So, memory will start vanishing faster than we will know.
So, my feeling is that my next machine will be a 32GB one. And Apple makes it even more attractive by changing the nomenclature of memory available, from 16gb to 18gb, and from 32gb to 36gb. More the merrier, right ?!
Top comments (0)