Ask HN: Monorepos for Data Engineers?

by bradhe on 5/9/2025, 1:57:18 PM with 1 comments
Hey folks, I'm working with a data engineering team trying to consolidate all their misc projects into a single monorepo. I'm used to working on monorepos in the context of regular software engineering (e.g. at my company, our monorepo is Golang, Rust, and Next.js and we use moonrepo to manage everything); however, not so much in the context of data engineering. It doesn't help that I'm not as Pythonic as I should be :). Wondering if there are known best practices I should be aware of for managing Python monorepos in the context of data engineering?

  • by baobun on 5/9/2025, 2:13:58 PM

    I don't think there's that much to it. You probably want to have all your packages run on the same Python version anyway but also for unified CI, I guess. Consider that your next.js project could have had some data pipeline on the backend and that your new project may grow some webapps. It's all the same in the end.

    Keep it simple.