NixOS has packages for llama-cpp and ollama: https://search.nixos.org/packages?channel=23.11&from=0&size=...

Edit: And Arch packages ollama officially - https://archlinux.org/packages/?sort=&q=llama&maintainer=&fl... - and a few things in the AUR - https://aur.archlinux.org/packages?O=0&K=llama

One thing about LLMs is that they are 6GB+ (and much larger for "smart" ones) just sitting in the background. They suck power and produce heat like nothing else, and they are finicky, especially at smaller sizes.

Running one as a background desktop assistant is whole different animal than calling a Microsoft API.

None. Microsoft has Copilot in preview mode in Windows and it's not very integrated apart from a chat window. I doubt GNOME/KDE will be able to dedicate enough resources to adding an assistant that is well integrated with the desktop environment any time soon.

A search in Fedora yields a single GSoC project[0] limited in scope to NetworkManager and it's not clear if anyone actually is working on that.

If the use case you're interested in is actually having the LLM doing things for you in SaaS applications, that wouldn't need deep integration but, considering Google is yet to deliver a Google Drive client for Linux, I wouldn't hold my breath waiting for a native Linux AI-assisted assistant.

Your best option right now is to interface with the assistants through their web interface and hope they have plugins/extensions to interact with things you want.

Other than that, some people have built prototypes running LLMs locally that talk to things like Home Assistant. But again, no deep desktop integration.

0 - https://docs.fedoraproject.org/en-US/mentored-projects/gsoc/...

Not as far as I know, but llamafiles are almost easier to get running than native packages.

https://github.com/Mozilla-Ocho/llamafile

Unrelated, but is there something like Bonzi Buddy for linux? Not the spyware part, just the friendly looking clippy-esque character that can tell you about your new e-mails, weather, or whatever? I kind of wish I had something like that.

Not bundled in a distro-specific way, but ollama fairly straightforward to install and use:

https://github.com/ollama/ollama/blob/main/docs/linux.md

Shameless plug: I’m writing one for Ubuntu Touch and I’m planning on adding a desktop version as well.

My frontend side is very weak so it’s going to be very barebones but contributions are welcome once it’s stable:

https://github.com/gessha/llmtest

Why? They take lots of space and lots of computing power. Linux has always been about lightweight and a bundle only containing essential things. You can always install one if you need it but as it stands right now LLMs are not useful enough to warrant their bundling in a distro. Just my 2 cents

There is one you can install via Flatpak. That will work in most distros.

https://flathub.org/apps/io.github.Bavarder.Bavarder

Wouldn't it be ironic if chatgpt assistant on linux gives linux end-user desktop dominance? lol... Microsoft would be like, "you fools, you've doomed us all"

Perhaps this: https://github.com/yusufcanb/tlm?

it is not distro bundled (yet), but I have it running on my Fedora Linux 39 running on a NUC with 16GB of RAM. Performance is good enough for me.

Jan.ai has a deb package. https://jan.ai/

Could they just package ollama and build a Python GTK interface around the CLI interface?

Check out https://gitlab.com/mdev1974008/doda - Pure terminal UI. Its just python.

Linux options (possibly not bundled) from another thread

https://news.ycombinator.com/item?id=39533494

I'm wondering if there's any AI assistant that could run on a headless RPI? And is easy to install?

When Linux comes bundled with AI I will lose all my faith in humanity and never use a computer again.

It's a fair assumption ChromeOS will be getting Gemini soon.

pip install llm # among other, to run local or not. Yet, KDE or Gnome are yet to integrate or develop a nice API for/ to any of these.

lm studio has a good linux version and gives you everything you'd ever need.

Open interpreter?