by davidmr on 11/28/2018, 5:06:03 PM
by pinewurst on 11/28/2018, 5:41:24 PM
That would have to be the worst job in the world - keeping Lustre going as an Amazon service, with management that utterly lacks understanding and sympathy.
by fold_left on 11/28/2018, 5:15:21 PM
finally, something that can handle node_modules! :)
by damnhungry on 11/28/2018, 5:12:30 PM
I don't mean to be cynical but for the last few days some one is bombarding with lot of amazon news
This was inevitable, and I’m sure the engineering from Amazon’s side is impressive because Lustre is an absolute beast to run well at scale, but I’m not sure how great an idea it is for most people.
Coming from an academia HPC background and then moving into the private sector, I’ve mostly come to believe that parallel filesystems (especially POSIX-compliant ones) are rarely the right solution outside of MPI simulations. Like NFS, it makes it extremely easy and attractive to implement anti-patterns like using the filesystem for IPC or generating a bazillion files and then needing to reduce them to move to the next stage of the pipeline. In my experience, it’s rare that people don’t regret doing that sort of stuff in the long run.
That said, I’m sure the AWS team knows their customers and what they’re doing better than I do!