LinkedIn is a strange place. There was a post the other day from an AI startup founder who had dropped a bundle on a top end Mac Studio with 512GB unified memory that apparently “ripped”, and would replace his $1000 a month API bill. So I went down a bit of a rabbit hole on the current state of open models that can be run locally vs the frontier alternatives, a bit about quantisation and why the gap is likely to persist for the foreseeable future. So maybe of interest if you’re into that kind of thing… TLDR; you’re likely lumbered with an API bill for the foreseeable but a hybrid approach is becoming increasingly common. https://iain.so/a-10k-mac-studio-wont-replace-your-api-bill submitted by /u/iainrfharper
Originally posted by u/iainrfharper on r/ArtificialInteligence
You must log in or # to comment.