If you count only the cost for you, maybe it doesn’t consume water, but your toy still guzzled lakes as it was training. Plus, the hardware to run a full sized LLM is expensive, so you bragging about how it costs nothing is like a millionaire preaching to gamblers that it’s better to just be rich than try to win at the slots
I specifically said “full sized”, a pc with modern gpu and more than 32gb of vram is not a regular computer that most gamers have access to. If you are running a 7B model on a gtx 1080 or even an rtx 3060, you are not running a full LLM like the ones you would get from a subscription service
I pay for my electricity. It uses roughly the same amount of power when I’m running an LLM as it would if I was playing a game. It’s negligible.
And contrary to all the breathless headlines about water-guzzling data centers, my computer doesn’t consume any water at all when I run an LLM.
If you count only the cost for you, maybe it doesn’t consume water, but your toy still guzzled lakes as it was training. Plus, the hardware to run a full sized LLM is expensive, so you bragging about how it costs nothing is like a millionaire preaching to gamblers that it’s better to just be rich than try to win at the slots
It’s a regular gaming PC. Are you going to dismiss all gamers as “millionaires”?
I specifically said “full sized”, a pc with modern gpu and more than 32gb of vram is not a regular computer that most gamers have access to. If you are running a 7B model on a gtx 1080 or even an rtx 3060, you are not running a full LLM like the ones you would get from a subscription service
Yes, I know. You’re saying that a 32GB graphics card is millionarie hardware? You’ve got a weird view of the cost of these things.
A reminder that lemmy was developed by Marxists and is still commie heavy to this day