Hello Community, Apologies in advance if this is not the right Subred for this. I am a student pursuing AI research. I have been using Windows Gaming laptops and PCs with Nvidia GPUs for LLM work with CUDA Acceleration. In my work, I now need to work on slightly bigger models and I need 16GB or 24GB GPU. Windows laptops with RTX 5080 and 5090 are extremely expensive at the moment, so I was thinking of switching to Mac. I am considering 48GB M5 Pro MacBook Pro. I want to know how mature is the Apple Silicon’s Metal MLX and MPS ecosystem? Is it anything compared to CUDA? I have the below specific questions: I work with 1-7billion parameter LLMs and create new architectures like Mixture-of-experts variations, complex reinforcement learning policies, LoRA and Full-finetuning on datasets, running different compression algorithms, Multi-agent systems, MCP servers, RAG systems. I use CUDA PyTorch, bitsandbytes, FlashAttention-style kernels, DeepSpeed, Triton, xFormers. Are there good alternatives for these on Mac? I use a lot of agentic coding tools like GPT Codex and Claude Code. How proficient are they in coding using Metal MPS and MLX? I know they are good in CUDA Accelerated libraries as I use them on daily basis but I have no idea about how good frontier models are in Metal. I do a lot of training work, not just inference experiments. So if an LLM is getting trained on Macbook, how is the performance in multi-tab browsing (I usually have 100-150 tabs open in Microsoft Edge) and documents/LaTeX related tasks? Note: I know training and inference speed will be lower on Mac. It will work for me if GPU acceleration with neural accelerators is available on Mac instead of dumb CPU brute-force. I care more about library availability and sufficient memory to allow me work with models that I want to. submitted by /u/ig_DRAX
Originally posted by u/ig_DRAX on r/ArtificialInteligence