Want to run your local AI models on Linux? This app makes it simple

I’ve been using AI as one of my go-to tools for research for some time now. Although there are plenty of options that can be accessed via a web browser (such as Opera’s Aria, the first AI service I ever used), I prefer desktop tools because I can install and use local LLMs — so all of my queries remain private. With a desktop tool, I can easily switch between local LLMs like Llama, DeepSeek R1, Mistral Instruct, Orca, GPT4All Falcon, and more.

Until now, my default Linux AI GUI had been Msty, but I’ve found a new app that’s just as good. That app is GPT4ALL is available for Linux, MacOS and Windows.

My two favorite AI applications on Linux and how I use to get more done.

GPT4ALL is a powerful tool that can be used to perform standard queries, act as a personal assistant, or write code. It has a great UI, which is easy to use and fits in with the look and style of your desktop.

I like the fact that GPT4All allows me to select the compute device used for text generation. My System76 Thelio, for example, has an AMD Radeon RX RX 7600 GPU. GPT4All detects the GPU and lets me choose between Vulkan AMD Radeon 7600 (RADV Navi33) or Vulkan AMD Radeon 7600 (RADV Raphael_MENDOCINO). I can also choose my default model, configure the number CPU threads, and enable a system-tray app. I can enable a locally API server to allow GPT4All to be accessed via http://localhost:4891. It consumes more resources when used in server mode.

Let me show you how to install GPT4All on Linux. Installing GPT4All for Ubuntu-based Distributions

What you will need

: Currently the only supported Linux distributions that are based on Ubuntu are those. I’ll show you how to do this using Pop!_OS Linux.

Show more

chmod u+x gpt4all-installer-linux.run

./gpt4all-installer-linux.run

Note that you do not need to use sudo for the installation. If you try to install GPT4All using sudo, the installation will fail. Click through the steps of the installation wizard.

How to feed your files to a local AI in order to get better, more relevant answers

The installation should complete without a hitch.

Using GPT4All

When you first open GPT4All, you’ll need to either opt in or out of anonymous usage analytics and anonymous sharing of chats. I would recommend you opt out. Once you’ve done that, you’ll find yourself on the main GPT4All window, where you’ll need to install a local model.

To install your first model, click Models and then click Add Model. In the resulting window, scroll down and locate the model you want, such as Llama 3.2 3B Instruct. When you find it, click Download to start the process. Remember, these models can be fairly large (the smallest being Llama 3.2 3B Instruct at 1.79 GB).

Once the model has been downloaded and installed, you’ll find it listed in the Models section.

Also: How to run DeepSeek AI locally to protect your privacy – 2 easy ways

Click on the Chats icon in the left sidebar. If you’ve only installed a single model, it’ll be listed at the top center. If you’ve installed multiple models, select the one you want to use from that drop-down.

Type your query in the “Send a message” field and hit Enter on your keyboard. GPT4All will get busy working to answer your query. If you’ve enabled follow-up suggestions, they’ll be listed beneath the answer. I like the follow-up suggestions because they help me dive into some seriously insightful (and sometimes fun) rabbit holes. On my System76 Thelio, I barely noticed the extra compute overhead required by the suggested follow-ups, so it’s worth enabling.

And that, my friends, is all there is to getting GPT4All installed on Linux — and the basics for using this helpful tool.

Want more stories about AI? Sign up for Innovationour weekly newsletter.

www.aiobserver.co