Nicely written article.

A few minor nits:

* As such, the development lifecycle is incredibly long. It starts with prototyping in an FPGA (Field Programmable Gate Array), in which chip designers can program their required functionality and confirm compatibility*

I’ve been involved with development of multiple generations of ASICs including 7nm and now 5nm. Going all the way back to 40nm, we never prototyped in FPGAs. Most designs will not fit into a single FPGA and designs involving multiple FPGAs are a headache and the design diverges rapidly from your target due to all the overhead associated with coordinating multiple FPGAs.

Prototype might be something you do for a very simple design and a new or inexperienced team.

If you have a good experienced team, the tooling is good enough that simulation and verification are all that’s needed. However verification is a very extensive process and the most time consuming part and can take 9-12 months. 7nm tape-outs are extremely expensive. So you want to get it right. Most designs have multiple options built in on risky sections so that a work-around for a bug can be done in a metal spin. Furthermore, most of these chips have significant FW/SW associated with them as well. If you think about it, that’s quite a lot to deliver in 2 years into something as reliable as a network processor.

When you consider all of the above, Broadcom does an amazing job going from TH3 to TH4 in only 2 years. They are tough to beat because they have such a good process that they can crank out these incredibly complex chips on a reliable cadence. Please note that this includes analog for the SERDES which is its own development.

> Technically, the use case in the network industry should be called an ASSP (Application Specific Standard Product), but network engineers are simple people, so we prefer to call it an ASIC.

Strictly speaking, ASIC only refers to "a custom chip for a specific customer for a narrow application". If the chip is available to the general public, it's not an ASIC but an ASSP. So a Wi-Fi chipset [0], an Ethernet controller or an LCD driver on the public market are all ASSPs, never ASICs. But colloquially speaking, for many people, "ASIC" is a synonym for all custom chips, so almost everything is ASIC, even if the chip is a general-purpose CPU, many will call it "ASIC" if it's someone's original design. Loose term, but it's not really bothering anyone, so this usage remains. Oh, and the word "chipset" has a similar problem. It originally refers to the ASIC/ASSP on the computer motherboard (originally IBM PC's I/O controller, but also graphics and sound), but nowadays it's often used as a synonym of "ASSP", in other words, "highly integrated chip designed for a specific application in mind", hence Wi-Fi chipset, 5G chipset...

Great intro article, saving this one for folks at work!

For anyone confused why so many low buffer chips exist it depends greatly on the network needs. In some deployments going for deep buffers can actually make things slower overall. E.g. in enterprise healthcare I've really gravitated towards Trident (Trident 3 currently) level chips for most of the LAN/MAN and sometimes RAN as the feature sets are there and there isn't a strong driver for deep buffer per se just traffic priority for phones and a certain app once in a great while. I.e. it doesn't matter if you've got 16 GB of deep packet buffer built up, the telemetry packet needs to make it out the 10 Mbit pipe to the remote clinic now not at some point in the future. For WAN across regions or internet we move into some of those Juniper ASICs though and do rely on deeper buffer for the MANY concurrent sessions which are fighting understanding loss at higher latencies.

Another interesting hardware rabbit hole to go down is the modular transceivers that go into these kinds of switches to provide different kinds of copper/optical connections instead of always being a fixed RJ-45 port like most would be used to at home. Crazy levels of engineering, especially at the higher end. I saw a really interesting presentation last year from Juniper on their use of silicon photonics for high density low cost 400 gigabit Ethernet and beyond (pdf warning) https://www.juniper.net/assets/us/en/local/pdf/nxtwork/silic...

I looked up one of the devices mentioned here: Juniper’s 2U-sized PTX10002. Turns out the list price for PTX10002-60C-AC-R is $720,000.00. :-o

The traffic volumes these devices are intended for are truly mind-boggling to me (as someone currently building a small 25Gbit network with sub-$1k parts).