With chiplets being a thing these days, I guess this is really about where in the overall system is the best place to put the package boundary and socket/pins? And then another approach to that same question would be what apple is doing with in-package ddr5 (which I think I heard amd is copying with a custom line for... Microsoft / azure I think it was?).

My understanding is that one factor pushing hyperscalers towards dual-socket was the cost of the network fabric - for the longest time, having two CPU sockets per NIC / per leaf switch port was the overall system price/performance sweet spot for many workloads. More sockets required more expensive CPUs, while single-socket servers need twice as many NICs and twice as many top-of-rack switchports.

With newer/faster ethernet standards you still need twice as many NICs but you can often split the lanes coming out of a switch chip and use a Y cable.

This article reads like an AMD advertisement for their EPYC processor line.

EPYC brings plenty of NUMA complexity in a single socket unfortunately. If you just want to solve system performance riddles then one socket is plenty. I seem to recall that Facebook publicly announced that they switched their web server systems to 1 socket more than 8 years ago. Since that time Netflix has written several times about how they carefully keep the sides of a 2S server from interfering with each other and I always wondered why they bother, why they don't just saw the system in half and save themselves the trouble.

Not mentioned in this is the issue of memory scaling.

DRAM price per GB has been roughly flat for well over a decade - consumer prices hit $4/GB in 2011, and have fluctuated around there ever since - most of the drop in real cost since then has been due to declining value of that $4. Prices for large enterprise/hyper scalers are probably similar, as it’s a low-margin commodity market.

Two sockets gets you more memory channels and more DINNS, but as memory price causes the RAM/CPU ratio to drop, and single-channel bandwidth increases with DDR5, that becomes less important.

Of course that’s one of those things you can’t really say to customers, kind of like “you don’t really need 250hp in a passenger sedan”.

If price didn't matter, what would be the best performing CPU available today? With lots of PCIe lanes and high single-thread performance?

If you want to inference big LLMs on CPU, you really want those 2x 12 memory channels a Dual EPYC system offers you for decent speeds.