I feel like there's several industries that are practically computer science yet don't utilize open source effectively. Data science is definitely one, but the video game industry definitely comes to mind.

You could argue game engines are notoriously complex, but the Linux kernel would like a word.

Collaborative incremental improvement of models would be extremely disruptive. While this happens via research, it's massively inefficient, particularly as pretrained models get larger and span multiple modalities.

There is definitely a problem re: large parameter models, the issue is I don’t think throwing software dev tools at this is the right solution.

The constraint is largely hardware. The incremental post training done via transfer learning is generally not broadly applicable to many use cases.

I'm curious how Deepmind's MOE models Perceiver and Switch might play into managing a open distributed model.