• 1 Post
  • 8 Comments
Joined 1 year ago
cake
Cake day: August 8th, 2023

help-circle




  • IDK, looks like 48GB cloud pricing would be 0.35/hr => $255/month. Used 3090s go for $700. Two 3090s would give you 48GB of VRAM, and cost $1400 (I’m assuming you can do “model-parallel” will Llama; never tried running an LLM, but it should be possible and work well). So, the break-even point would be <6 months. Hmm, but if Severless works well, that could be pretty cheap. Would probably take a few minutes to process and load a ~48GB model every cold start though?


  • I’ve tried a couple rolling distros (including Arch), and they always “broke” after ~6 months to a year. Both times because an update would mess up something with my proprietary GPU drivers, IIRC. Both times, I would just install a different distro, because it would’ve probably took me longer to figure out what the issue was and fix it. I’m currently just using Debian stable, lol.