"The reason why we have Linux, and BSDs, and XNU is that they all provide the same baseline API, which was defined from the outside [by POSIX]. The coordination problem was pre-solved, and what remained is just filling-in the implementation."

But that is not at all how Posix operates or has operated. Posix standardises common denominators between existing implementations. The fact that we now have strlcpy(3) and strlcat(3) in Posix 2024, is not because Posix designed and stipulated them. Rather, they appeared in OpenBSD in 1998, were found useful by other *nix-es over time, spread, and were finally taken aboard by Posix that standardised what was already out there and being used! This to me is the very opposite of the point the author is trying to make!

I'm not entirely clear why/how this is an open source issue?

My assertion: Inertia of user base is by far the largest predictor of what will stick in a market. If you can create a critical mass of users, then you will get a more uniform set of solutions.

For fun, look into bicycles and how standardized (or, increasingly not) they are. Is there a solid technical reason for multiple ways to make shoes that connect to pedals? Why are there several ways to shift from your handlebars? With the popularity of disk brakes, why don't we have a standard size for pads?

I think there are a lot of things that we tell ourselves won from some sort of technical reasoning. The more you learn of things, the less this seems true, though.

Not, btw, that there aren't some things that die due to technical progress.

Open source has the best kind of coordination. If there's a real use-case for two things to work together, you or someone else can implement it and share it without anyone's permission. Meanwhile in proprietary land, people sometimes build things that nobody wanted, and also leave out features with high demand. Proprietary optimizes for the profit of some individuals; open source optimizes for maximum utility.

Thus far, open source has optimized for maximum utility for individuals who can write code... but AI may be changing that soon enough.

Look at Microsoft Windows. You can still run Access '97 because all the DLLs with the GUI widgets from Win '95 are still there. Until 2012 or so, Microsoft would regularly come out with a whole new widget set every few years. You can see this most notably in the settings UI for Windows where there are some screens in the "modern" UI and plenty of dialogs that still come from the '95/NT 4 era.

Since then Microsoft has had no real answer for "how do I write desktop applications for Windows?" other than "use Electron".

(If they were still introducing new widget sets they'd be converting the 'modern' dialogs to something 'postmodern' while still having Win '95 dialogs in there)

This post displays the same ignorance as that joke about Kissinger complaining that Europe didn't have a phone number.

Open source is a movement: it's neither an individual nor a committee. And people join it because it is a movement with no central authority.

Alternative view - Open Source projects have the freedom to do what they want to do. Which in turn gives me the freedom to choose how I set up my desktop environment. How many changes have been pushed on Windows users over the last 15 years with the only option being to change and get security updates or stay on an old insecure version?

And while there are lots of desktop environments for Linux you can usually run applications targeting one in any of them (I use Gnome's file manager in Enlightenment as it supports accessing CIFS shares directly).

> The underlying force there is the absence of one unified baseline set of APIs for writing desktop programs.

It's called the Common Desktop Environment.

To me it's about how low-bandwidth communication channels limit collaboration.

Linux and FOSS grew up (congrats!) and the important work got super big and complex.

Unfortunately, online communication - typically the channel preferred by FOSS projects - has much lower bandwidth than teams working full time in an office. Thus limiting the "depth" of collaboration.

It's not all bad. FOSS does have some profound success to be proud of. For small and well-defined projects "benevolent dictator for life" works! Anything one person can lead - a desktop tool or a library - FOSS produces really good outcomes. But above say, a package manager or a distro.. things get wonky.

Again it's not all bad. FOSS is rolling up its sleeves. Folks are organically doing the "go down every avenue people are motivated to go down" thing. You could call it Darwinism. But many motivated communities lack resources to go far enough to reach value. Motivation stalls during projects (we're human!), and FOSS rarely comes with a motivation-boosting paycheck. Plenty of valiant efforts don't reach value and it's never malicious. It's OK!

So is there a way to better concentrate efforts?

If the paths are long, it follows that the community should tackle fewer paths. The path(s) taken should be well defined, charted in advance as much as possible, and not uncovered bit by bit - or the work will take decades.

Growing an entire ecosystem around one path forward (or a few) requires alignment. Can enough trust be fostered in leaders to get people to work on a shared vision?

A vision of what Linux on the desktop should/could converge to is the kind of problem that, if Linux were a company, would be bet-the-company strategic. A company can't afford to go down two paths. So it might lock its smartest people in a room to hash out one true strategy. Or have one smart person dictate one vision and align everyone on it.

Can that be done for FOSS?

In the bounds of a single project it has been proven that it can. But what about an entire ecosystem?

Solving the coordination problem in FOSS is one of the grand challenges of humanity. If we solve it, I think it will effect a tectonic shift with far reaching implications and fixes major socioeconomic problems like wealth concentration. Eg: a FOSS alternative to Visa, and of course Windows/MS Office.

> There was a decade of opportunity for OSS to coordinate around an IDE protocol, but that didn’t happen, because OSS is bad at coordination.

Its also because a lot of the key people in Open Source, and senior hackers generally, don't actually use IDEs.

We should encourage more of the younger generation over to powerful configurable editors such as Emacs, rather than locking everybody into VSCode/JetBrains/etc.

It is a feature not a bug.

What author describes is about dominance.

Most OSS is fragmented because of different ideas and different people.

People nag about .NET not having much outside OSS because .NET devs (I am one of them) will not use stuff that doesn’t have MSFT badge.

You don’t want such power in general OSS. Lack of coordination is sign of no dominant entity and that is the feature.

This is kind of the same reason I gave up on the Linux desktop and went for macOS. When I first learned about Linux I was thinking "Sweet!. This is going to kick Microsoft's ass!", but this was 30 years ago and instead of a kickass desktop OS, we got 1000 mediocre ones.

This feels very right. The problem is there are few entities invested enough in Linux as a consumer platform, that have the motivation to push things forward. To make the decisions on what their "Reference" system is.

Valve is maybe the closest?

> But there was no one to coordinate Linux on desktop.

Freedesktop.org?

Maybe open source doesn't need to coordinate. Perhaps users and developers should demand standards and interoperability from their platforms. Perhaps that's why we have things like Electron, Unreal Engine and Unity. One way or another we'll coordinate on something.

idk. i don't really follow the argument. large projects in open source coordinate internally and engage externally when they need to- i suspect that isn't all that different from what you'd see in a large megacompany like apple or microsoft.

open source people create reusable interfaces. i'd argue they go one step further and create open and public internet communities with standards, practices and distribution/release channels.

I wonder if the author has seen codebases for enterprise apps that are a behemoth, designed by a committee with low developer retention.

In contrast, open source software led by the same handful (typically just one guy) of people over years/decades are well coordinated by the BDFL(s) and have a clear direction.

I think the term the author is looking for is "opinionated". The mantra is "if you don't like it fork it". The apparent lack of coordination is a feature of open source,not a bug.

Successful and popular projects rarely seem uncoordinated. The Linux kernel coordinates thousands of devs over mailing lists. Git was created to facilitate that, and now everyone uses to coordinate development.I would even dare say modern dev coordination is spearheaded by open source projects.

> There’s no single entity that can coordinate the API, in contrast to Windows and MacOS. > > But then, how can Linux exist? How does that square with “never break the user space?”

The “never break the user space” philosophy is limited to the kernel, and the single entity coordinating that realm is called Linus.

> I use NixOS. And NixOS isn’t a problem — it’s a solution.

> The past ten years saw a big shift in how we are writing software: baseline level of “interactive static analysis"

While I am making no judgement on what is 'better', the author's choices have impacts, and sometimes not all projects can work within the costs imposed by static analysis.

For example, remember that Rice's theorm generalizes HALT, and that you always have to under or overestimate with SA. Either introducing over constraints or missing things.

It is horses for courses, sometimes the added friction is worth it, other times it is damaging.

The question of if Nix is a problem is a context specific question.

Be careful about making default assumptions and patterns more than what they are, no matter how sensible they are as a default.

https://bugzilla.mozilla.org/show_bug.cgi?id=259356

21 years old bug. XDG is such an obvious improvement, yet getting all projects on board is taking forever.

Open source, like private property, markets, science, and Satoshi consensus, enables people to benefit from the work of others that they don't have a means of coordinating with—for example, because they're dead, or they don't trust you, or they're hard to get along with. If you already have a way of coming to terms with somebody, you don't need open source. Different engineers inside Google can benefit from each other's work without making it open source because they're coordinated by Google's management. It's people who aren't coordinating with Google who benefit from them making it open source.

It's an important question. Paid enterprise enforce a certain level of coordination (supposedly overcoming the pain of dealing with people and ideas you don't necessary like). Open source imposes no such things, so efforts grow when and where they want..

I think the definition of linux is much broader than what is considered today a platform for the IDE. It's kind of like the IDE is the cart, and the kernel is the horse, but 30 years later, linux is an engine with a cabin virtual machine, rather than a desktop per se. the parts interact at a different level now.

I think that that two of our more instructive examples here might be Wayland and late stage GNOME -- namely, how did these two projects get as big as they did, despite being both 1) past compatibility breaking and 2) not obviously good?

    > But then, how can Linux exist? How does that square with “never break the user space?”

This somewhat reminds me of arguments I've heard about waterfall vs agile over the years. Planning for ways to plan, and everything falling apart when faced with hard decisions and thick skulls.

Key sentence here:

> But it is also clear why JetBrains didn’t do LSP — why would they? While the right solution on the technical grounds, you aren’t going to get paid for being technically right.

https://github.com/pandas-dev/pandas/pull/58623

The selection and standardisation comitee for open source is the usage data. Make public what is used where under what circumstances, standards emerge .

Ehh I don't buy that the market was ready 10 years earlier (in 2006) for open-source LSP implementations.

You gotta have someone write those language servers for free, and the language servers have to be performant. In 2006 that meant writing in a compiled language, which meant that anyone creating a language server for an interpreted language would need to be an expert in two languages. That was already a small pool of people.

And big multiplayer OSS platforms like GitHub didn't exist until 2008.

We need more people writing RFC style documents.

@dang

Why editorialize a question mark into titles like this? What does it signify? We are not talking about a factual assertion of debatable veracity, but a statement of opinion along the lines of "considered harmful". Disagreement and debate is the obvious expected result of such a statement. HN admin using special mod powers to editorialize a title which is neither misleading nor clickbait, simply to indicate skepticism, comes across as petty and not a little hypocritical in light of the "don't editorialize titles" site guideline.

Must... make... cathedral... at... all... costs... why... no... cathedral... halp...

http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral...

Who is being incentivised to reduce the friction of interoperation?

Coordination is hard. People who are good at coordinating are not necessarily the same people who are happy to contribute their time to FOSS. And FOSS may need to coordinate in ways that vertically integrated companies do not.

Coordinating between loosely aggregated volunteer projects is not the same as coordinating between vested stakeholders either. I would guess that most FOSS projects are more invested in their own survival than in some larger objective. Teams within a company are (presumably) by definition invested in seeing the company mission succeed.

The GNOME / KDE example mentioned elsewhere in this thread is interesting because these are two somewhat equivalent co-existing projects. Any coordination between them is surely not their highest priority. Same with all of the different distros. The each exist to solve a problem, as the fine article says.

I wonder how much the problem is actually "open source can't standardise on a single solution." Let one thousand flowers bloom, sure. But don't expect a homogeneous user experience. The great thing about standards is there are so many to choose from. xkcd 927. etc.

Seems to be a pattern - take a general then apply to open-source. You know what else can't coordinate - most of the corporate world.

The article doesn't bring up any critical issue that the world world opensource should suddenly deal with - feels more like a morning rant after his shower - which is also how the post starts - it's basically on the front-page because its title is trying to be provocative.

Yet most closed source stuff depends heavily on open source.

Apple coordinates internally, since macOS works with Apple hardware. Windows can drive coordination among hardware vendors. In the Linux world, many organizations and projects share power; there is not the same focal power on having a consistent end user OS (dependencies, configuration). Declarative and deterministic build systems at the OS level allow different groups to package their subcomponents reliably. As various configurations get socialized, it gives choice to tradeoff between customization and popularity/vetting.

I would love to tell people about linux for their desktops, but the main issue I have with it is the fact that people who are interested in it ask me one question regarding Linux distributions:

“Which one?”

This is pretty much the cause of a 90% drop off of interest in Linux on the desktop.

I could say use Ubuntu (and I do) to some of the people who I’m close with that are interested in Linux, but they discover Lubuntu, or Linux Mint and Debian, then they get easily confused and give up.

And that is not even getting into the updates and the packaging and heaven forbid anything breaks.

I was truly shocked at how bad the experience is when you are using and RPM based distribution and a program is only available as a DEB

> I suspect that I have an outdated version of hotspot Linux profiler, but I can’t just go and download a fresh release from GitHub, because hotspot is a KDE app, and I use NixOS.

KDE (not to be confused with the Plasma desktop) is just a bunch of C++ libraries that can work on a variety of desktop environments and even OSes (though Hotspot being a perf report alternative is clearly meant for use with Linux).

I just went and downloaded the latest CI build from[0] and it ran just fine on my openSUSE Tumbleweed, running Xorg with Window Maker. I do have a bunch of KDE apps installed, like Kate (my currently preferred text editor), Dolphin (the file manager i use whenever i want thumbnails, usually for videos and images), Spectacle (for screenshots), Falkon (i use it as a "clean" browser to test out things), etc so i also do have the KDE libraries on my system, but that is just a `zypper install` away. Or an `apt-get install` or `pacman -S` or whatever package manager your distro uses, i've used a bunch of them and they all pretty much behaved the same. I'd expect Hotspot to be installable in the same way in any of them (and i'd expect the AppImage to have these libraries bundled in anyway so you probably wont need them[1]).

If there are issues with NixOS (i don't know, i haven't tried it) i think it might actually be a NixOS issue and not a KDE issue.

[0] https://github.com/KDAB/hotspot/releases/tag/continuous

[1] EDIT: i checked with --appimage-extract, it contains pretty much everything

but open source foundation provide some guides/events/programs to coordinate.

I am reminded of someone I read recently decrying as a loss GNOME adopting systemd components as a critical dependency because they want alternatives to systemd.

... and this a layer of open source flexibility I never wanted. I don't want alternatives to core system management; I want one correct answer that is rugged, robust, well-tested, and standardized so that I don't have to play the "How is this service configured atop this service manager" game.

Fake title - original does not contain a question mark

Something uncoordinated struggles to co-ordinate. More news at ten

The OP defeats his own argument. LSP was a collaborative effort that benefited from a degree of coordination that only hierarchical organizations can provide, yet it still sucks ass.

OP blames FOSS for not providing an IDE protocol a decade earlier, but doesn't ask the rather obvious question of why language-specific tooling is not only still around, but as market-viable as ever. I'd argue it's because what LSP tries to do is just stupid to begin with, or at least exceptionally hard to get right. All of the best language tooling I've used is ad-hoc and tailored to the specific strengths of a single language. LSP makes the same mistake Microsoft made with UWP: trying to cram the same peg into every hole.

Meanwhile, Microsoft still develops their proprietary Intellisense stuff because it actually works. They competed with themselves and won.

(Minor edit: I forgot that MS alone didn't standardize LSP.)

What? How is this even at top? Some no-name program is not getting an update or is not perfectly installable and suddenly it's an open source problem? Stop being an entitled prick

Many moons ago Scott Alexander wrote a critique of Marx. It starts by arguing that if capitalists can be said to produce anything, it's coordination. Coordination, Scott argues, is a thing every bit as real as coal or food or legal services. People need to manufacture it, and we call them executives/investors/marketing staff, and others want to buy it. When we buy coordination we call it brand value or similar. Open source has a notable absence of coordinators, because producing coordination is hard and non-fun, so without a capitalist market there's not much incentive to do it. Same reason desktop Linux historically struggled with anything that wasn't hobby programming (art, UI design, etc... eventually Red Hat and others hired such people using server profits).

The Linux kernel and GNU in general are projects that hacked around that problem by just copying the decisions of other people who were coordinated by capitalists (UNIX vendors), which worked long enough to bootstrap the ecosystem until some of the key people could be coordinated by Red Hat and others who monetized indirectly. But at every stage, the coordination was being produced by capitalists even though it was hard to see.

In other places where the mimic-and-support model didn't work, open source really struggled. This is most obvious on the desktop. Even there, ultimately this approach has been adopted for large chunks of it. If you play games on Linux today it's because people copied the Win32 API i.e. the coordination was produced by capitalists like Bill Gates.

Now Alex mentions LSP and JetBrains. The reason JetBrains didn't do the LSP isn't because of value capture. After all, IntelliJ has been open source for a long time. Other IDEs could easily have embedded it and used its plugins. The reason JetBrains use a Java API is because it's a lot more productive and effective to design in-process APIs than network protocols. As long as you aren't crossing runtime boundaries they're easier to write, easier to test, easier to reason about statically (especially w.r.t. concurrency), and much more performant. You can exchange complex object graphs in a shared address space and coordinate them using locks. All this is a highly effective way to extend an IDE.

Microsoft did the LSP because they took a bunch of energetic developers who only wanted to do web development, so they used Electron. Also for reasons of sticking with the crowd, .NET being pretty useless for cross-platform desktop stuff... it's not just that experience with desktop programming is fading away. But browsers were never designed for the challenges of large scale desktop programming, in fact they weren't designed for building apps at all. So they don't let you use threads, static typing via TypeScript is an aftermarket hack, V8 has very low maximum heap sizes, and there are many other challenges with doing a clean JetBrains style architecture. To their credit, the VS Code team leaned into the architectural limits of the browser and did their best to turn it into advantages. They introduced this notion of a backend that could run independently of the frontend using a 'standard' protocol. This is technically not really different to the IntelliJ API being open source, but people like the idea of protocols more than embedding a JVM and using stuff in a company-specific namespace, so that created a lot of community good will and excitement for them at the cost of many technical challenges.

Those challenges are why JetBrains only use the LSP style approach for one of their IDEs, which due to historical reasons doesn't share the same architectural approach as all the others. And it's also why, if you look at the Rider protocol, it's some fairly advanced state sync protocol thing, it's not a plain old HTTP RPC style protocol.

Given that both are open source and both are produced by teams of paid developers working in an office coordinated by capitalists, it's probably not right to identify this as an open source vs proprietary difference. It's purely a technical one to do with JVM vs web as foundational platforms.

Whats wrong with LSP?