I have been trying to figure out what to do with the RAM heavy box. Its a 1U Dell r640 w/dual xeon platinum 8268’s, and 1.5tb of 2666 ram. it has 8x2.4Tb SAS 2.5" drives so not a lot in the way of storage. No GPU, trying AI anyway, token count is horrendous… But it works. Grok 2, 512K Context, -t 40 + NUMA, 4.73t/s prompt, 1.35t/s gen… web search enabled… Do the Tesla GPU’s fit off the stock risers in 1U servers or am I going to have to cut the top of this? Anyone have a similar build? Any recommendations? I’ll be adding a GPU ASAP but interested in what other people trying to claw their way in are up to… submitted by /u/Creative-Type9411
Originally posted by u/Creative-Type9411 on r/ArtificialInteligence
You must log in or # to comment.