How to install the latest software on your linux distro

If you need a software which doesn't appear among packages in the repository for your Linux distribution at all or it's way too old you have several options. 

Assuming you don't want to switch your apt from Debian/stable to Debian/testing or Debian/sid.

Include Debian backports repo

If you're using Debian, there's an official Debian-approved way to get packages from Debian/testing being installed to stable without bringing the whole bunch of dependencies. Read this with care - https://backports.debian.org/
 
In short you just need to add a repo to your /etc/apt/sources.list.d/ and then run apt update to get your local apt cache updated. Starting from that point you should be able to find & install some updated packages while using apt. Don't forget to use -a parameter to see all versions like:

apt list kate -a
 
For those like me, who're running Debian 11 Bullseye on a weak SBC (single-board-computer) with limited amount of CPU and RAM, keep in mind that it will be a preferred way to install newer version of software. Just because Debian guys did their best to build newer software packages using as much of existing stable libs as they could. 
 
All the other methods listed below will bring a lot of new libraries to your machine. Even though they will be isolated from your system libraries, they will be loaded during the start of your "alternatively installed application" which means such apps will be taking more CPU and RAM compared to their counterparts from your distro repository.
 
If you're reading all that with a grim, sitting behind modern Core i7 / Ryzen 7 and 16 Gb of RAM then just go ahead.   
 

Install binary packages from testing / unstable / sid (Debian-only)

This approach received a controversial feedback from power users, as it might break or harden your system upgrades in future. But if you don't want to use anything apart from Debian official repos you might want to give it a try.
 
Before you go any further, it makes sense to check it first, if the package you're looking for is actually appearing in testing / unstable / sid streams - https://packages.debian.org/search

 
The below is a copy-paste from http://jaqque.sbih.org/kplug/apt-pinning.html page, I'm leaving it here just in case if the page goes away:

Apt-Pinning for Beginners

Why apt-pinning?

Do you run Debian? Have you ever gotten annoyed at how Debian Stable always seems to be out of date?

I will show you a way that you can have apt mix-and-match between Stable, Testing, and Unstable sources. This will allow you to run a mostly-Stable system, but also track the latest and greatest of those packages that you are most keenly interested in.

Why do this? Stable is covered by the Security Team. Testing and Unstable are not. For non-critical services, like perhaps your mailer, or your window manager, this is not so important, and the newest versions may have additional features that are desired. It is these packages that are perfect for pinning to a version, other than Stable.

sources.list

The first step is to set up your /etc/apt/sources.list to include your typical Stable, plus the Testing/Unstable sources that you want.

A simple sources.list may look like this:

#Stable
deb http://ftp.us.debian.org/debian stable main non-free contrib
deb http://non-us.debian.org/debian-non-US stable/non-US main contrib non-free

#Testing
deb http://ftp.us.debian.org/debian testing main non-free contrib
deb http://non-us.debian.org/debian-non-US testing/non-US main contrib non-free

#Unstable
deb http://ftp.us.debian.org/debian unstable main non-free contrib
deb http://non-us.debian.org/debian-non-US unstable/non-US main contrib non-free

You would probably want to add your mirrors, security.debian.org, and perhaps the appropriate deb-src lines. Here is a copy of my actual sources.list.

preferences

The next step is to create/edit your /etc/apt/preferences file. preferences is where the apt-pinning takes place. Normally, the highest version of an available package wins, but we will override that.

A simple preferences file may look like this:

Package: *
Pin: release a=stable
Pin-Priority: 700

Package: *
Pin: release a=testing
Pin-Priority: 650

Package: *
Pin: release a=unstable
Pin-Priority: 600

Note the decending values. Since Stable has the highest pin-priority, it will be installed preferentially over Testing or Unstable.

My actual preferences file is what you see above.

apt-get update

Now we are ready to apt-get update. This will add the new repositories to apt's list.

E: Dynamic MMap ran out of room

You may find that you receive an error like the following:

E: Dynamic MMap ran out of room
E: Error occured while processing sqlrelay-sqlite (NewPackage)
E: Problem with MergeList /var/lib/apt/lists/ftp.us.debian.org_debian_dists_woody_contrib_binary-i386_Packages
E: The package lists or status file could not be parsed or opened.

This is caused because apt's cache is too small to handle all of the packages that are included with stable, testing, and unstable. This is also very easy to fix. Add the following line to /etc/apt/apt.conf

APT::Cache-Limit "8388608";

Thanks to R (Chandra) Chandras for pointing out this problem

Installing new packages

To install a new package, it is just as it ever was, apt-get install <package>. If the package exists in Stable, then that is what it will grab. If the package exists only in Unstable, then from Unstable it will be gotten.

What if the package exists in both Stable and Unstable, but we want the Unstable version? There are two ways we can do that, each with a slightly different syntax, and each with a slightly different effect.

  • apt-get install <package>/unstable

    This will install the unstable version of the package, and try to meet any dependencies from Stable. This may not work, but it will tell you why:

    # apt-get install zsh/unstable
    Reading Package Lists... Done
    Building Dependency Tree... Done
    Selected version 4.0.6-7 (Debian:unstable) for zsh
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    distribution that some required packages have not yet been created
    or been moved out of Incoming.
    
    Since you only requested a single operation it is extremely likely that
    the package is simply not installable and a bug report against
    that package should be filed.
    The following information may help to resolve the situation:
    
    Sorry, but the following packages have unmet dependencies:
      zsh: Depends: libc6 (>= 2.2.5-13) but 2.2.5-11.1 is to be installed
    E: Sorry, broken packages
    
  • apt-get -t unstable install <package>

    This will install the Unstable version of the package, and try to meet any dependencies from Unstable. This may produce better results.

    # apt-get -t unstable install zsh    
    Reading Package Lists... Done
    Building Dependency Tree... Done
    The following extra packages will be installed:
      libc6 libc6-dev libc6-pic libdb1-compat locales 
    The following NEW packages will be installed:
      libdb1-compat 
    5 packages upgraded, 1 newly installed, 0 to remove and 394  not upgraded.
    Need to get 11.6MB of archives. After unpacking 606kB will be used.
    Do you want to continue? [Y/n] 
    

That's it!

Armed with a complete sources.list and a minimal preferences, you can go ahead and mix-and-match between the various Debian releases.

Have fun!


manually backporting a package 

This is kind of the same as above, but the only difference is that you're building all these packages from testing/sid from sources yourself. And finally you build a package for your current debian version. It might work for some simple packages, which are not requiring a lot of dependency, but it's easily turning into hell with packages of more dependencies.

Anyways, you asked - you get. 

https://wiki.debian.org/SimpleBackportCreation

deb-get

Basically deb-get is a shell script which knows where to download proprietary deb packages in a way to simplify it's installation down apt level.

makedeb

Basically it's the same idea as with AUR, but this one is for Debian - https://mpr.makedeb.org/

Be twice carefully when using it - noone really checks anything in there, so packages can easily contain backdoors or any kind of malicious stuff.


Flatpak

 
Over the course of last couple of years, there are three new distribution methods appeared for Linux: Flatpak, AppImage and snap packages. They're quite similar one to another, the main idea is that application binaries are shipped together with all (or mostly all of) the libs they depend on, so you get a Mac OS X experience with its self-contained-apps. Or they're bringing their own shared components, like "GNOME Application Platform version 42" which takes like 700+ Mb:

root@orangepi4-lts:~# flatpak list --columns=name,origin,application,version,size
Name                                             Origin             Application ID                                Version           Installed size
Mesa                                             flathub            org.freedesktop.Platform.GL.default           21.3.9            290.2 MB
Photos                                           flathub            org.gnome.Photos                              43.0               18.5 MB
GNOME Application Platform version 42            flathub            org.gnome.Platform                                              724.1 MB
Shotwell                                         flathub            org.gnome.Shotwell                            0.30.16            18.6 MB


Useful commands:

# see what remotes (repositories) do we have configured
flatpak remotes --show-details

# add remote repos
flatpak remote-add --if-not-exists gnome-nightly https://nightly.gnome.org/gnome-nightly.flatpakrepo
flatpak remote-add --if-not-exists flathub-beta https://flathub.org/beta-repo/flathub-beta.flatpakrepo
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo

# see what apps are existing in the remote repo 
flatpak remote-ls --app gnome-nightly

# see installed apps and components
flatpak list
flatpak list --columns=name,origin,application,version,size


# search and install app
flatpak search shotwell
flatpak install org.gnome.Shotwell

# install app from specific source
flatpak install gnome-nightly org.gnome.Epiphany.Devel

# run installed application
flatpak run
org.gnome.Shotwell


Also there's one important thing you need to know about flatpak. Applications are isolated from the system in a sandbox, so if you install some new application it gets almost no privileges - it cannot "talk" to other apps over dbus, it cannot access disks, it cannot this and it cannot that. In order to allow an app to use specific privileges you can use run modifiers:

# run application with overriding default sandbox permissions
flatpak --filesystem=host run org.gnome.Shotwell

Or you can configure it in a way to have these privileges by default, so you don't have to provide them for every run:


# see what privileges does the app have
flatpak override --show org.gnome.Photos

# configure permissions to be permanent
flatpak override org.gnome.Photos --socket=system-bus --filesystem=host

# and then you can run it as normal
flatpak run org.gnome.Photos



I did not find an easy way to allow a flatpak application to use all privileges, like if I entirely trust some app and want to allow it to bypass all these restrictions, and it's a big miss of flatpak.

There's an application called flatseal you can install, which makes privileges handling a bit easier task:

flatpak install com.github.tchx84.Flatseal
flatpak run
com.github.tchx84.Flatseal

AppImage

URL: https://www.appimagehub.com/

Unlike flatpaks, which are still attempting to properly separate various dependencies to be downloaded & installed separately, so they could be at least reused and not bloat up HDD usage, appImage is doing quite the opposite - everything is just a one big file. 

There're couple of places you can go to, to get appImages:

The thing is, most of the appImages are created for x86_64 (aka amd64), so you won't be able to run them straight away. But there's a solution, called box64.

SNAP

 
This is the most "hated" way, because it's originating from Canonical (a company who're making Ubuntu) and it's claimed that the repo is non-transparent, so you cannot easily get your app into it.

Personally I don't share that point of view. To me, anything which allows you to have an app of the most recent version you need
in a prompt manner is fine. We all came from Windows, right, so why to moan?

Installation is quite simple, following official guide:
apt install snapd
systemctl start snapd # or just reboot
snap install core # this is mandatory
This is how they suggest to test it - to install a hello-world app from snap and run it. Make sure you didn't have hello-world before, otherwise your test won't be a clean one:
 
snap install hello-world
snap run hello-world
 
You can install the GUI frontend called "Snap Store" where you can find snap packages to download:

snap install snap-store
snap run snap-store

If you don't want any GUI you have an option to explore the snap repository with your web browser - https://snapcraft.io/

You can also use snap CLI interface:

snap                       # short help on all commands
snap list # see installed snaps
snap find <package name> # search for a package
snap info <package name> # get extended info about package
snap install <package>
snap run <package>
snap

Keep an eye on how much space will be wasted in /snap dirs. For me it was 427 Mb after just installing core and hello-world

sudo du -hs /snap /var/lib/snapd/ /var/snap/
326M /snap
101M /var/lib/snapd/
28K /var/snap/

But after installing Kate (which brought another number of dependencies with it) it grew by  500 Mb! 

sudo du -hs /snap /var/lib/snapd/ /var/snap/
993M /snap
346M /var/lib/snapd/
52K /var/snap/


Now let's compare the memory consumption. Running Kate which came from distro: 

/usr/bin/kate &
ps -auxfwww | grep kate
pi 95425 2.3 2.8 1617448 114432 pts/2 Sl+ 00:01 0:02 \_ kate

Now running snap version:

snap run kate &
pi 95885 16.2 3.0 503204 118696 pts/2 Sl+ 00:04 0:02 \_ /snap/kate/130/usr/bin/kate

At first glance the difference is not that much. But don't get yourself into the trap. We also need to consider what amount of memory was consumed in "shared" area.

Memory topic in Linux is bit more complex, to say the least, to make a clean comparison side-by-side like we just tried. Some shared libraries might remain in the "shared" area even if noone is using them, just in case if someone start to use them again. The cleanest we can do is to flush memory by rebooting our pc. So let's do it like this:

1) restart to have a clean slate and flush memory from unused shared libraries
2) make a memory measurement after a restart (with htop)
3) start kate from distro
4) make another measurement
5) repeat points 1-4 but this time with starting kate from snap

These are my figures:

(fig 1. Just rebooted. Before running Kate from repo)
 
(fig 2. Kate from repo was started)
 
(fig 3. Rebooted again. Before running Kate from snap)
(fig 4. Kate from snap was started)


 

You see it yourself, Kate version from repo took 50 Mb of resident memory and 20 Mb of shared.

Whereas snap version took the same 50 Mb of resident + 300 Mb shared. And it's slightly less snappier on my weak hardware, than version from Debian repository because of the CPU overhead. 

I'll say it again: if you're running KDE and Kate from repository, it will be reusing a  lot of KDE/Qt libraries as they will likely be already in memory by the time you start it. When you're using Kate from Snap/AppImage/FlatPack it will be loading its own libraries. A lot of libraries.

I forgot to mention that the snap thing is also running a deamon in your system, called snapd. It's taking just a fraction of CPU but memory footprint is impressive.

But still it's nice to have this option. It has user-friendly design and it's very easy to use.

Converting ArchLinux / AUR package to appImage

This is relatively new way, which became possible because of this project - https://github.com/hanzala123/arch2appimage

The idea is simple: to take the ArchLinux package, or AUR package and "pack" it into appImage format. Why to take Arch package, you might ask? Because Arch is known for bringing latest and greatest versions of all the software packages to its users, so if you're looking for a fresh version of the application, chances are great you'll find it in Arch.

Before you go, first check if the package you're looking for is appearing in Arch repo - https://archlinux.org/packages/ 

Or among AUR packages - https://aur.archlinux.org/packages

Sounds promising? Let's give it a try on our Debian 11 Bullseye Stable running on aarch64 Orange Pi 4 LTS:

git clone https://github.com/hanzala123/arch2appimage.git
cd
arch2appimage
# make sure we're using python of version 3.8 and higher
python3 --version
# install python dependencies
python3 -m pip install -r requirements.txt
# install the additional pacakge we need
sudo apt install desktop-file-utils
# let's try to install something
python3 arch2appimage.py

I installed dragon (a video player) but it created an appImage for x86_64 architecture which is not very handy :)

NIX

There's an interesting alternative to cross-platform package managers like flatpak, appImage, snap, homebrew and even docker-containerized applications called nix

In a nutshell it utilizes the same idea of taking all the dependencies together with app and deploy it to your system, it just does that in a slightly different way than other package managers.  

Off course, every solution like this comes with a cost, so be prepared it will download a tonns of libs to your machine, even if the package you wanted to install was a tiny itself, but having a lot of dependencies. 

It's better to install nix with using your package manager, as later on, after you play with it long enough and decide to remove it, that will be hardsome, if you installed it following nix approach.

I.e. do this:

apt install nix-bin nix-setup-systemd

And not this:

sh <(curl -L https://nixos.org/nix/install) --daemon

Unfortunately the very first package I tried to install - RetroArch - was a failure. It stopped somewhere in the middle while leaving  a lot of trash behind:

nix-env -iA nixpkgs.retroarchFull

Then I tried to install the latest Konqueror, Just wanted to see how its latest KHTML engine works. With nix you can do it by installing a package in a temporary nix "env" via nix-shell command, so the package won't be installed to your permanent nix environment:

nix-shell -p libsForQt5.konqueror
konqueror

Pls remember to clean out the nix stuff as it tends to grow bigger: libsForQt5.konqueror

nix-collect-garbage

So I'm just skipping it so far, but keeping in mind. 

No comments:

Post a Comment

Start here

Disable Firefox from updating itself and flash those annoying "Restart to Keep Using Firefox" messages on you

I recently switched from Brave to Firefox. Just because Brave appeared to be some proprietary shit, even though they're masking themselv...