> Relationship with CVNets

> CoreNet evolved from CVNets, to encompass a broader range of applications beyond computer vision. Its expansion facilitated the training of foundational models, including LLMs.

We can expect it to have grown from here: https://apple.github.io/ml-cvnets/index.html

It looks like a mid-level implementations of training and inference. You can see in their "default_trainer.py"[1] that the engine uses Tensors from torch but implements its own training method. They implement their own LR scheduler and optimizer; the caller can optionally use Adam from torch.

It's an interesting (maybe very Apple) choice to build from the ground up instead of partnering with existing frameworks to provide first class support in them.

The MLX examples seem to be inference only at this point. It does look like this might be a landing ground for more MLX specific implementations: e.g. https://github.com/apple/corenet/blob/5b50eca42bc97f6146b812...

It will be interesting to see how it tracks over the next year; especially with their recent acquisitions:

Datakalab https://news.ycombinator.com/item?id=40114350

DarwinAI https://news.ycombinator.com/item?id=39709835

1: https://github.com/apple/corenet/blob/main/corenet/engine/de...

It's interesting that Apple also actively develops https://github.com/apple/axlearn, which is a library on top of Jax. Seems like half the ML teams at Apple use PyTorch, and the other half uses Jax. Maybe its split between Google Cloud and AWS?

They also mention in the README:

> CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data

This is the first I’m hearing of that, and the link seems broken.

Built on top of pytorch.

How does this compare to MLX? As far as I understand MLX is equivalent to PyTorch but optimized for Apple Silicon.

Is this meant for training MLX models in a distributed manner? Or what is its purpose?

What's the advantage of using this over something like Huggingface Transformers, possibly with the MPS backend?

I would love an LLM agent that could generate small api examples (reliably) from a repo like this for the various different models and ways to use them.

Does this support training on Apple silicon? It’s not very clear unless I missed something in the README.

I went through their folders, they have have a lot of classes that just inherit from pytorch and torchvision classes and seemingly do nothing new. All optimizers, schedulers and most layers do that. They do however have a reasonable amount of blocks, i.e. specific combinations of layers from various papers, similar to monai.networks.blocks. Out of "building pieces" they also have a few newly implemented losses, metrics.

What library would you recommend for neural net training and inference on Apple M1? I want to use it from C++ or maybe Rust. The neural net will have 5M params at most.

> OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework https://arxiv.org/abs/2404.14619

Apple is pushing for open information on LLM training? World is changing...

What hardware would one need to have for the CoreNet to train efficiently?

The style is not very different than NeMo(nvidia)/fairseq(Facebook)/espent(oss) etc..

Ok, why would anyone use this when you have industry standard methods already?

h1: CoreNet: A library for training deep neural networks

Pretty funny that Apple engineers use Homebrew too.

anyone have this hosted so anyone can try this out?

Looks at Apple: CoreNet Looks at Microsoft: Net Core

My inner trademark troll demands a bucket of popcorn.