Linux Desktop Migration Tool

When I got a new work laptop in July I decided that after all these years on Linux and countless hardware refreshes it was time to finally automate the data migration. I looked around and asked around, so that I could build on something instead of starting from scratch. But all I found were very personal scripts with hardcoded values, specific use cases… nothing generic enough.

So I decided to write something from scratch after all. And instead of writing something for myself like everyone else does, I started writing it as a tool that could be used by anyone or used as a base for custom migration scripts. I gave it a name (Linux Desktop Migration Tool) and created a proper project.

It’s a shell script because it’s quick to prototype and easy to extend. I wish it was a desktop app, or even something integrated into GNOME as for example part of GNOME Initial Setup, but something with a nice UI written in a modern toolkit is beyond my skills and time I can spend on it. The shell script may not have the best user experience, but it gets the job done.

I’m primarily targeting Fedora Silverblue because that’s what I use and install for people around me. But at the moment it should work on any modern desktop distribution. And I want to keep it that way if possible, it’s just that I’m not interested in use cases that aren’t relevant on Silverblue. At least until I complete features that are on my todo list.

What can it do? At the moment it can copy over the contents of XDG directories (Documents, Pictures, Downloads…) or any arbitrary home directories. It can reinstall flatpak applications on the new machine, copy over their data, migrate existing Toolbx containers. In the future I’d like to dig into migrating desktop settings, certificates, keyring etc.

I’m trying to avoid the “copy files over and hope for the best” approach. Otherwise I would just have a one line script to copy over the entire home partition. Wherever possible, I use export-import functions. And reinstalling flatpaks is also a cleaner way than just copying directories with binaries as I recently learned when I needed to migrate data from an x86 laptop to an aarch64 Macbook.

If you need to migrate data from one desktop machine to another, check out the tool. Perhaps you will find it useful. Suggestions for improvements or even merge requests are welcome. 😉

How is Linux used by FIT BUT students

The Faculty of Information Technology of Brno University of Technology is one of the two top computer science schools in Brno, Czech Republic. Our development office of Red Hat has intensive cooperation with them including educating students about Linux and open source software. To find out more about how they use Linux, we ran a survey that collected answers from 176 students which is a pretty good sample. I promised to share results publicly, so here they are:

The following chart shows the distribution of responders by year of school. The survey was primarily targeting students in the first year which is why they make up over 50% of the responses.

The following chart shows how many students had experience with a Linux distribution prior their studies at the university. 46% did which shows a pretty good exposure to Linux at high schools.

And now what desktop OS students use primarily. Windows are dominating, but Linux is used as a primary OS by roughly one third of students. macOS is only at 10%. Although we gave responders an option to specify other OSes, no one submitted, for example, BSD.

The following chart shows in what form students use Linux primarily (as either a primary or secondary OS). 44% of students have it installed on their desktop/laptop. 31% use Windows Subsystem for Linux. School programming assignments have to run on Linux, so if they want to stick with Windows, WSL is the easiest way for them. Virtualization is at 9% and remote server at 13% (I suspect it’s mostly uni servers where students can test their assignments before submission).

And here come shares of Linux distributions. Responders could pick multiple options, so the total is over 100%. Basically the only relevant distributions among FIT BUT students are Ubuntu, Fedora, Arch Linux and Debian.

Ubuntu has a clear lead. It’s the default option for WSL where it is on vast majority of installations, so I wondered what the share would be without WSL.

Without WSL the gap between Ubuntu and the rest of the pack is smaller. And since I’m from the Red Hat desktop team I also wondered what are the shares among students who indicated they use Linux primarily on desktop/laptop.

When it comes to desktop computers and laptops the shares of Fedora and Ubuntu are almost the same. That shows two things: 1. Fedora is strong on the desktop among local students, 2. being the default option in WSL gives Ubuntu an advantage in mindshare. Fedora is not even officially available for WSL, but even if it was, it wouldn’t probably change much because other distros are available in the Microsoft Store and only one student of out 50+ who primarily use WSL responded that they use something else than Ubuntu. WSL is probably used by users who want some Linux in their Windows and don’t care much which one it is, so they stay with the default.

We also asked students what prevents them from using Linux primarily. By far the most frequent answer (80%) was “Software I use is not available for Linux”, followed by “I don’t like the UX and logic of the OS” (28%) and “Compatibility with my hardware” (11%). Some students also responded that they simply hadn’t had enough time to get familiar with Linux and are staying with what they know. Other reasons were marginal.

Copr repo with the latest GNOME Software

We’ve been working hard for the last months to improve GNOME Software both in Fedora Workstation and Silverblue, especially its reliability. One thing we identified as a problem is a long feedback loop. I took months before bug reporters and designers could easily test changes made based on their feedback.

That’s why Milan Crha, the new GNOME Software maintainer, created a Copr repository that includes the latest development version of GNOME Software. If you want to help us with testing, install GNOME Software from the Copr repository and report issues. They won’t be overlooked. Milan is a very responsive maintainer.

Mozilla makes Firefox Beta available on Flathub

I’m glad to see that Mozilla has made a significant process with offering Firefox as a flatpak. Having Firefox as a flatpak was one of our long-term goals.

Three years ago we started a testing flatpak repo with Firefox Developer Edition and soon after that we added Firefox Nightly. For a long time it was the only source of Firefox for Flatpak out there. The user base grew into thousands, a level our hosting could barely deal with. Lately we haven’t had much time for its maintenance and at least the nightly builds were often broken.

That’s why from the very beginning we worked with Mozilla to make official Firefox builds available as flatpaks. The effort was later on picked up by Endless.

Now it brings first fruits, Mozilla is already shipping Firefox Beta in the beta channel on Flathub. You just need to enable it by installing this file: https://flathub.org/beta-repo/appstream/org.mozilla.firefox.flatpakref

I think it may already be useful for Silverblue users who have relied on our testing repo if they didn’t want to use package overlay.

There are still a few things to polish before making the stable Firefox available in the stable channel. One of them is localization files which has always been a difficult thing with Firefox. Mozilla has traditionally provided official localized builds for each language. This is not how localizations are typically handled in Flatpak or even in Linux distro packages.

For Fedora Firefox RPM we had to write a script that on startup automatically loads a particular localization file in the form of an addon. I suppose the official Firefox flatpak will work in a similar way.

Last year we also started providing Firefox flatpak built from Fedora packages. That has been the only stable Firefox for Flatpak around. And even after Mozilla releases their official Firefox for Flatpak, we will stay committed to it because there is demand among Fedora users for Firefox that is verified and built by the Fedora Project and it’s also a requirement for software included in Fedora anyway. So if we want to have Firefox as a default, pre-installed browser e.g. in Silverblue we’ll have to build the flatpak ourselves. It also gives us flexibility to ship crucial fixes and features important to our users faster than upstream (e.g. Firefox in Fedora already runs natively on Wayland by default, not on XWayland like upstream Firefox).

In the future, users will have a choice. They can either stick with the default Firefox provided by us, or switch to the one provided directly by Mozilla in a more convenient and secure way than the current tarballs with binaries are. We will also keep maintaining our testing repo for those who are interested in Nightly and Developer Edition. And we’ll see if there is sufficient interest in it to continue.

 

F30 release parties in Prague and Brno

As it’s our tradition since Fedora 15 we’ve organized Fedora 30 release parties in the Czech Republic. Normally the Brno one is earlier, but this time the Prague took place one week before the Brno one – on June 4.

The Prague party was again held in offices of Etnetera, a Fedora-friendly software company. I was worried about the attendance because at the same time the biggest demonstration since 1989 (100+k people) was taking place in Prague. A lot of our friends went there. A lot of old faces didn’t show up, but they were replaced by quite a few new faces (which I think it’s partly due to posting an invitation to the biggest Czech Linux group on Facebook), and in the end the attendance was the same or a bit higher than last time – around 30 ppl.

F30 release party in Prague

We’ve prepared 5 talks for visitors. I started with news in Fedora Workstation and also added a pack of news in Fedora Silverblue. We try to make the release parties as informal as possible, so the talks should not be lectures where one is talking and the rest is listening in silence. My talk was again mixed with a lot of discussion and instead of 30-40 min, it took 1h20m.

Then Petr Hráček introduced the project he’s working on Packit. As someone who maintains packages in Fedora I find the idea interesting because in package maintenance there is a lot of work that can be automated and if there is a tool that can help you with that, great! The only thing that limits my enthusiasm about Packit is that it relies on having YAML files in the upstream repo. And you know how some upstream projects are dismissive to hosting any downstream-specific files…

The next two talks were delivered by Fedora QA guys – František Zatloukal and Lukáš Růžička. František talked on how they test Fedora, what tools they use and how you can help them. Lukáš talked on how to report bugs the useful way.

The 5th talk that was supposed to be on GNOME Builder was cancelled because we were considerably over time, but its author – Ondřej Kolín – promised that he’d change it into an article on mojefedora.cz.

To continue with the bad luck, the release party in Brno a week later had a time conflict with another demonstration against our prime minister. And this time it had an impact on the attendance, around 40 people showed up and we normally get twice more. I hope he will go away, so that there are no longer any demonstrations against him that lower attendance of our release parties 🙂

The party took place in the offices of Red Hat and the program of talks was exactly the same as in Prague.

F30 release party in Brno

F30 release party in Brno

At both parties we also had cool swag for participants. Especially brand-new Fedora hadbook that arrived from a printing-shop just before the Prague party.

New Fedora handbooks

Get notified of new upstream releases

If you maintain a distribution package, or app bundle, or container image, it’s handy to know when components you use in it have new releases.

There is a really useful service Anitya that resides on release-monitoring.org. It watches almost 20 thousand projects for new releases and notify about them.

I maintain several packages in Fedora and the Fedora Project already makes it really convenient for me. It uses Anitya and opens a new bug against your package every time there is a new upstream release which I close once I update the package. But not every project gives you this service.

I also maintain several apps on Flathub which doesn’t provide such a service (yet). And it’s even more important to know about new upstream releases because besides the apps themselves I also have to maintain their dependencies which are not available in runtimes. Especially Evolution has quite a few of them.

Anitya gives you an API, so you can write your own service that will be checking with Anitya. But I’m a lazy person and why to write something that already exists, right? The Fedora Project has a web app called Fedora Notifications which can send notifications of all sorts of events to your email or IRC. You don’t have to be a Fedora package maintainer or anyhow involved in the Project. All you need is a Fedora account (FAS) to log in.

You pick Email Notifications (or IRC, whichever you prefer), click Create New Filter, then you pick Anything regarding a particular “upstream project” rule, and add projects you want to monitor separated by commas. Then click Add this rule and you’re good to go. This is what my rule for components I maitain in Flathub looks like:

You’ll be getting notifications of new upstream releases to your email and it’s up to you how you’ll act on them. I typically check what kind of release it is, if there are any security or important bug fixes. I try to release those ASAP. Otherwise I plan them for future app releases and test with them beforehand.

I can imagine that there can be a nice automation built on the top of the Anitya API. If it detects a new release, it updates the manifest, triggers a build, and sends you info how it went. Maybe one day 🙂

Better Bluetooth sound quality on Linux

Over a year ago I got my first serious Bluetooth headphones. They worked with Fedora well, they paired, connected, sound was directed to them. Just the sound quality was not overwhelming. I learnt that because of limited bandwidth of Bluetooth a codec with audio compression has to be used. There are quite a few of them to pick from: AAC (very widely supported because it’s the only one iPhone supports, partly freely available), AptX (also very widely supported, but proprietary), AptX-HD (enhanced AptX with a higher bitrate, also proprietary), LDAC (probably the best codec available, highest bitrate, available in Android, supported mostly by Sony devices), MP3 (also possible, but supported by virtually no devices). And also SBC which is a native Bluetooth, first generation compression codec with rather bad sound quality.

My headphones supported SBC, AAC, AptX, AptX-HD, LDAC, so all the advanced codecs. Sadly on Linux it fallbacked to the basic SBC because no other was available for Bluetooth, and headphones for €200 produced rather underwhelming sound. I mostly listen to music on Spotify. Listening to it on my headphones meant transcoding OGG 320 kbps to SBC and losing a lot of sound quality.

Then I recalled that Sony released LDAC as open source in the Android Open Source Project. And they really did because you can find libldac released under Apache 2.0 License there. So it could possibly be made available on Linux, too. Bluez was also able to negotiate LDAC with the end device. What was missing was a plugin for PulseAudio that would utilize the codec and encode the stream into LDAC before sending it over Bluetooth to the headphones.

Today I learnt that such a plugin had been finally created.  And besides LDAC it also supports AAC, AptX, and AptX-HD. Those are patent-protected codecs and the plugin relies on ffmpeg to support them, so it’s not likely they will be available in Fedora any time soon. But libldac is already in Fedora package review and is waiting for the final legal approval. The plugin currently depends on ffmpeg, but if it were made optional, we could at least ship LDAC support by default in Fedora Workstation.

I thought we could also support AAC because its decoder/encoder is already available in Fedora, but I learnt that it only supported the original AAC format while what devices support these days is HE-AAC which is still protected by patents.

Anyway, someone already built packages of both the plugin and libldac and I installed them to test it. And it worked on Fedora 29 Workstation, LDAC was used for Bluetooth stream encoding:

I don’t have bat ears, but I could recognize a difference in sound quality immediately.

If I’m not mistaken it makes Linux the first desktop system to support LDAC. And with support for other codecs it will make it the OS with the best Bluetooth sound quality support because all other systems support only a subset of the list, hence fewer headphones/speakers at their best sound quality.

Fedora at LinuxDays 2018 in Prague

LinuxDays, the biggest Linux event in the Czech Republic, took place at the Faculty of Information Technology of Czech Technical University in Prague. The number of registered attendees was a bit lower this year, it could be caused by municipality and senate elections happening on Fri and Sat, but the number got almost to the 1300 mark anyway.

Besides a busy schedule of talks and workshops the conference also has a pretty large booth area and as every year I organized the Fedora one. I drove by car to Prague with Carlos Soriano and Felipe Borges from the Red Hat desktop team on Saturday morning and we were joined by František Zatloukal (Fedora QA) at the booth.

František and me at the booth.

Our focus for this year was Silverblue and modularity. I prepared one laptop with an external monitor to showcase Silverblue, the atomic version of Fedora Workstation. I must say that the interest of people in Silverblue surprised me. There were even some coming next day saying: “It sounded so cool yesterday and I couldn’t resist and install it when I got home and played with it in the night…” With Silverblue comes distribution of applications in Flatpak and there was a lot of user interest in this direction as well.

Reasons to use Fedora.

I was hoping for more interest in modularity, but people don’t seem to be so aware of it. It doesn’t have the same reach outside the Fedora Project as Flatpak does, it’s not so easy to explain its benefits and use cases. We as a project have to do a better job selling it.

The highlight of Saturday was when one of the sysadmins at National Library of Technology, which is on the same campus, took us to the library to show us public computers where they run Fedora Workstation. It’s 120 computers with over 1000 users (in the last 90 days). Those computers serve a very diverse group of users, from elderly people to computer science students. And they have received very few complaints since they switched from Windows to Fedora. Also they’ve hit almost no problems as sysadmins. They only mentioned one corner case bug in GDM which we promised to look into.

Carlos and Felipe checking out Fedora in the library.

It was also interesting to see the setup. They authenticate users against AD using the SSSD client, mount /home from a remote server using NFS. They enable several GNOME Shell extensions by default: AlternateTab (because of Windows users), Places (to show the Places menu next to Activities)… They also created one custom extension that replaces the “Power Off” button with “Log Out” button in the user menu because users are not supposed to power the computers off. They also create very useful stats of application usage based on “recently-used” XML files that GNOME creates to generate the menu of frequently used applications. All computers are administrated using Ansible scripts.

Default wallpaper with instructions.

The only talk I attended on Saturday was “Why and How I Switched to Flatpak for App Distribution and Development in Sandbox” by Jiří Janoušek who develops Nuvola apps. It was an interesting talk and due to his experience developing and distributing apps on Linux Jiří was able to name and describe all the problems with app distribution on Linux and why Flatpak helps here.

On Sunday, we organized a workshop to teach to build flatpaks. It was the only disappointment of the weekend. Only 3 ppl showed up and none of them didn’t really need to learn to build flatpaks. We’ll have the same workshop at OpenAlt in Brno and if the attendance is also low, we’ll know that workshop primarily for app developers is not a good fit for such conferences.
But it was not a complete waste of time, we discussed some questions around Flatpak and worked on flatpaking applications. The result is GNOME Recorder already available in Flathub and Datovka in the review process.

The event is also a great opportunity to talk to many people from the Czech community and other global FLOSS projects. SUSE has traditionally a lot of people there, there was Xfce, FFMPEG, FreeBSD, Vim, LibreOffice…

 

Story of GNOME Shell Extensions

A long time ago (exactly 10 years ago) it was decided that the the shell for GNOME would be written in JavaScript. GNOME 3 was still looking for its new face, a lot of UI experimentation was taking place, and JavaScript looked like the best candidate for it. Moreover it was a popular language on the web, so barriers to entry for new contributors would be significantly lowered.

When you have the shell written in JavaScript you can very easily patch it and alter its look and behaviour. And that’s what people started doing. Upstream was not very keen to officially support extensions due to their nature: they’re just hot patching the GNOME Shell code. They have virtually unlimited possibilities in changing look and behaviour, but also in introducing instability.

But tweaking the shell became really popular. Why wouldn’t it? You can tweak your desktop by simply clicking buttons in your browser. No recompilations, no restarts. So extensions.gnome.org was introduced.

The number of available extensions grew to hundreds and instability some of them occasionally introduced seemed like a fair price for the unlimited tweakability. In the end when the Shell crashed it was just a blink. Xorg held up the session with opened clients, the Shell/Mutter was restarted and the show could go on.

In 2016 GNOME switches to Wayland by default. No Xorg and also nothing to hold up the session with opened clients when the Shell crashes. There is only Mutter as a Wayland compositor, but unfortunately it runs in the same process as GNOME Shell (a decision also made 10 years ago when it also looked like a good idea). If the Shell goes down, so does Mutter. Suddenly harmless blinks became desktop crashes with losing all unsaved data in opened applications.

I read user feedback and problems users are having with Fedora Workstation (and Linux desktop in general) a lot on the Internet. And desktop crashes caused by GS extensions are by far the most frequent problem I’ve seen recently. I read stories like “I upgraded my Fedora to 28 and suddenly my desktop crashes 5 times a day. I can’t take it any more and I’m out of ideas” on daily basis. If someone doesn’t step in and say: “Hey, do you have any GS extensions installed? If so, disable them and see if it keeps crashing. The extensions are not harmless, any error in them or incompatibility between them and the current version of GS can take the whole desktop down”, users usually leave with the experience of unstable Linux desktop. It hurts our reputation really badly.

Are there any ways to fix or at least improve the situation? Certainly:

1. Extensions used to be disabled when the Shell crashed hard (couldn’t be restarted). Since on Wayland it’s the result of every crash, we should do that after every GS crash. And when the user goes back to GNOME Tweak Tool to enable the extensions again, she/he should be told that it was mostly likely one of the 3rd party extensions that made the desktop crash, and she/he should be careful when enabling them.
2. Decoupling GNOME Shell and Mutter or/and other steps that would bring back the same behaviour like on Xorg: GS crash would not take everything down. This would require major changes in the architecture and a lot of work and GNOME Shell and Mutter developer community has already a lot on their plates.
3. Discontinuing the unlimited extensions, introducing a limited API they can use instead of hot patching the GS code itself. This would be a very unpopular step because it’d mean that many of the existing extensions would be impossible to implement again. But it may become inevitable in the future.

Why I use Flatpak for 3rd party apps

There are more reasons why to run applications as flatpaks. Someone wants to have the latest versions as soon as possible. For me as a user of Fedora which provides up-to-date versions of apps this is not a big motivation. Someone wants to run apps more securely. Again I usually trust software provided by Fedora and Flatpak sandbox is still not as strictly enforced as it should ideally be.

But where I really prefer using Flatpak to RPM packages are 3rd party applications. I’m usually running development versions of Fedora. Pre-releases on my work machine and Rawhide on my home laptop. They have been pretty stable for me, including applications in Fedora repositories. Unfortunately it’s not the case for 3rd party applications. Their authors usually don’t follow distro development closely and although many of them bundle as much as possible to avoid problems with changing dependencies, things break.

I used to use the Spotify client as a package from Negativo17 repos. But when I upgraded to F27, something broke and it stopped working. I’m pretty sure it was fixed later on after people started reporting it, but I didn’t want to stop using Spotify for the time being and didn’t have time to debug and report the issue. So I switched to Spotify flatpak and it has worked for me ever since.

The problem I had with very early stages of pre-released Fedora and Rawhide is that dependecies of GStreamer plugins in RPMFusion were usually broken. So I ended up without system multimedia codecs. It was often the case for VLC from the same repository, too. Then I switched to VLC and GNOME MPV flatpaks. Problem solved.

The last example is Telegram. Until recently I was using the official version. It’s not even provided as an RPM package. You have to download an archive, unpack its content to your home, run the binary which creates a desktop file… not very elegant in 2018, but once you do it, it just works. Well… until it doesn’t. I upgraded to F28 and Telegram suddenly took a lot of time to start up. It hung on some font config error until it timeouted and finally started. It easily took 1 minute. So I switched to Telegram flatpak as well. Works like a charm.

So what I really appreciate about Flatpak is that the apps don’t rely on the underlying system, so if the system changes e.g. due to upgrade to a newer major version apps don’t break. As I said it’s not such a major issue for distro-provided apps, but it’s certainly an issue for 3rd apps and Flatpak solved it for me.