by pritambarhate on 5/14/2024, 3:35:45 AM with 1 comments
Has someone here come across a case study on training LLMs on Apple M2 Ultra Neural Engine? I wanted to know how it would compare to training LLMs on GPUs like H100.
Considering the cost and shortage of H100, can Mac Pro Ultras be used for training LLMs? I mean people are trying to do it on SuperComputers using CPUs (https://news.ycombinator.com/item?id=40348371) surely someone must have tried using Apple Silicon Neural Engine.
I tried searching for it, but didn't find anything proper.
The Neural Engine itself on those is pretty much non-comparable to something like an H100. The GPU is much more suitable for training, or even the CPU probably.
Considering the cost and shortage of H100, can Mac Pro Ultras be used for training LLMs? I mean people are trying to do it on SuperComputers using CPUs (https://news.ycombinator.com/item?id=40348371) surely someone must have tried using Apple Silicon Neural Engine.
I tried searching for it, but didn't find anything proper.