Help Us Test Evolution

It was not an easy task to make Evolution run nicely as a flatpak, but Milan Crha managed to do it and we’ve been fine-tuning it for the last 3 years. There are still some use cases that don’t fully work in a flatpak, but they don’t affect most users. Evolution has established itself well on Flathub, too. It has accumulated over 130k installs. There are roughly 12-15k “active” installations.

Some time ago I also started building Evolution for the beta channel on Flathub. When there are already development releases of the upcoming version (it will be 3.49.x this cycle), I build those for the beta channel. If they’re not available yet, I push stable releases there right after the upstream release is done, roughly one week before they go to the stable channel.

So the beta channel always provides the latest and greatest. Be it the latest development version, or freshly released stable version, depending on the phase of the development cycle. It really helps us find problems before they hit average users. For example, the latest release – 3.48.2 – introduced an ugly regression that made basically impossible to view some HTML messages. It was identified and patched while the version was still in the beta channel, so it never reached users on stable.

That’s why I’d like to raise awareness of the beta channel with Evolution. If you’re a bit more advanced user who would like to help with testing and you use Evolution from Flathub, consider switching to the beta channel. You wouldn’t switch to something broken. Milan doesn’t let low quality releases out. It’s for rather rare bugs that would be great to identify and fix before they hit everyone, or for early feedback when UX changes are being done.

All you need to do is to add the beta channel:

flatpak remote-add --if-not-exists flathub-beta https://flathub.org/beta-repo/flathub-beta.flatpakrepo

And install Evolution from it:

flatpak install flathub-beta org.gnome.Evolution

Of course, there are other ways to get the latest release of Evolution. You can build it from source yourself, or build a flatpak directly from the upstream git repo, but the beta channel on Flathub is the most convenient.

EDIT: I forgot the most important thing: where to report problems. All general issues including problems specific to Flatpak as a distribution format belong to the Evolution issue tracker on GNOME Gitlab. Only problems that are specific to Evolution on Flathub should belong to our Flathub issue tracker on Github. I understand it may be difficult for the user to distinguish between a Flatpak problem in general and a problem specific to Flathub. So if you report the former in our issue tracker on Github, it’s fine, too.

Silverblue: pretty good family OS

I’m the go-to IT guy in the family, so my relatives rely on me when it comes to computers and software on them. In the past I also helped them with computers with Windows and macOS, but at some point I just gave up. I don’t know those systems well enough to effectively administer them and I don’t even have much interest in them. So I asked them to decide: you either use Linux which I know and can effectively help you with or ask someone else for help.

Long story short: I (mostly remotely) support quite a few Fedora (Linux of my choice) users in my family now. It’s a fairly easy task. Usually after I set up the machine I don’t hear from the user very often. Just once 6 months and a year typically when I visit them I upgrade the machine to the new release and check whether everything works. But Fedora upgrades became so easy and reliable that recently I usually just found out that they had already done it by themselves.

But there was still one recurring problem: even though they performed upgrades because it was probably a big enough thing to catch their attention they didn’t act on normal updates and I often found them with outdated applications such as Firefox.

I could set up automated DNF updates running in the background, but it’s really not the safest option. And that’s where Fedora Silverblue comes to rescue. Applications run as flatpaks which can be and in fact are by default updated automatically in the background. And it’s pretty safe because the updates are atomic and the app is not affected until you restart it.

The same goes for system updates. rpm-ostree can prepare the update in the background and the user switches to it once the computer is restarted.

So I thought: the user base of Silverblue typically consists of developers and power users used to the container-based workflow, but hey, it could actually be a pretty good system for the users I support in my family.

I got an opportunity to try it out some time ago. I visited my mom and decided to upgrade her laptop to Fedora 32. Everything would have gone well if my son hadn’t pulled the power cord out during the process. The laptop is old and has a dead battery, so it resulted in an immediate shutdown. And that’s never good during a system upgrade. Instead of manually fixing broken packages which is a lengthy process I decided to install Silverblue.

The fruits of it came a week later. My mom called me that she was experiencing some graphical glitches and hangs with Fedora 32. Probably some regression in drivers/mesa. It’s a T400s from 2009 and neither Intel nor anyone else does any thorough regression testing on such old models I suppose. On the standard Fedora Workstation my mom would have been screwed because there is no easy way back.

But it’s a different story on Silverblue. I just sent her one command over Telegram:

rpm-ostree rebase fedora/31/x86_64/silverblue

She copy-pasted it to Terminal, pressed Enter and 5 minutes later she was booting into Fedora 31.

And the best thing about it is not that it’s so easy to upgrade and rollback in Silverblue, but that the apps are not affected by that. I know that if I let my mom rollback to Fedora 31 she will find her applications there just like she left them in Fedora 32. The same versions, same settings…

P.S. my mom’s laptop is from 2009, but Fedora Workstation/Silverblue flies on it. Who would have thought that GNOME Shell animations could be smooth on an 11-year-old laptop. Kudos to everyone who helped to land all the performance optimizations in the last several releases!

Mozilla makes Firefox Beta available on Flathub

I’m glad to see that Mozilla has made a significant process with offering Firefox as a flatpak. Having Firefox as a flatpak was one of our long-term goals.

Three years ago we started a testing flatpak repo with Firefox Developer Edition and soon after that we added Firefox Nightly. For a long time it was the only source of Firefox for Flatpak out there. The user base grew into thousands, a level our hosting could barely deal with. Lately we haven’t had much time for its maintenance and at least the nightly builds were often broken.

That’s why from the very beginning we worked with Mozilla to make official Firefox builds available as flatpaks. The effort was later on picked up by Endless.

Now it brings first fruits, Mozilla is already shipping Firefox Beta in the beta channel on Flathub. You just need to enable it by installing this file: https://flathub.org/beta-repo/appstream/org.mozilla.firefox.flatpakref

I think it may already be useful for Silverblue users who have relied on our testing repo if they didn’t want to use package overlay.

There are still a few things to polish before making the stable Firefox available in the stable channel. One of them is localization files which has always been a difficult thing with Firefox. Mozilla has traditionally provided official localized builds for each language. This is not how localizations are typically handled in Flatpak or even in Linux distro packages.

For Fedora Firefox RPM we had to write a script that on startup automatically loads a particular localization file in the form of an addon. I suppose the official Firefox flatpak will work in a similar way.

Last year we also started providing Firefox flatpak built from Fedora packages. That has been the only stable Firefox for Flatpak around. And even after Mozilla releases their official Firefox for Flatpak, we will stay committed to it because there is demand among Fedora users for Firefox that is verified and built by the Fedora Project and it’s also a requirement for software included in Fedora anyway. So if we want to have Firefox as a default, pre-installed browser e.g. in Silverblue we’ll have to build the flatpak ourselves. It also gives us flexibility to ship crucial fixes and features important to our users faster than upstream (e.g. Firefox in Fedora already runs natively on Wayland by default, not on XWayland like upstream Firefox).

In the future, users will have a choice. They can either stick with the default Firefox provided by us, or switch to the one provided directly by Mozilla in a more convenient and secure way than the current tarballs with binaries are. We will also keep maintaining our testing repo for those who are interested in Nightly and Developer Edition. And we’ll see if there is sufficient interest in it to continue.

 

Get notified of new upstream releases

If you maintain a distribution package, or app bundle, or container image, it’s handy to know when components you use in it have new releases.

There is a really useful service Anitya that resides on release-monitoring.org. It watches almost 20 thousand projects for new releases and notify about them.

I maintain several packages in Fedora and the Fedora Project already makes it really convenient for me. It uses Anitya and opens a new bug against your package every time there is a new upstream release which I close once I update the package. But not every project gives you this service.

I also maintain several apps on Flathub which doesn’t provide such a service (yet). And it’s even more important to know about new upstream releases because besides the apps themselves I also have to maintain their dependencies which are not available in runtimes. Especially Evolution has quite a few of them.

Anitya gives you an API, so you can write your own service that will be checking with Anitya. But I’m a lazy person and why to write something that already exists, right? The Fedora Project has a web app called Fedora Notifications which can send notifications of all sorts of events to your email or IRC. You don’t have to be a Fedora package maintainer or anyhow involved in the Project. All you need is a Fedora account (FAS) to log in.

You pick Email Notifications (or IRC, whichever you prefer), click Create New Filter, then you pick Anything regarding a particular “upstream project” rule, and add projects you want to monitor separated by commas. Then click Add this rule and you’re good to go. This is what my rule for components I maitain in Flathub looks like:

You’ll be getting notifications of new upstream releases to your email and it’s up to you how you’ll act on them. I typically check what kind of release it is, if there are any security or important bug fixes. I try to release those ASAP. Otherwise I plan them for future app releases and test with them beforehand.

I can imagine that there can be a nice automation built on the top of the Anitya API. If it detects a new release, it updates the manifest, triggers a build, and sends you info how it went. Maybe one day 🙂

Fedora at LinuxDays 2018 in Prague

LinuxDays, the biggest Linux event in the Czech Republic, took place at the Faculty of Information Technology of Czech Technical University in Prague. The number of registered attendees was a bit lower this year, it could be caused by municipality and senate elections happening on Fri and Sat, but the number got almost to the 1300 mark anyway.

Besides a busy schedule of talks and workshops the conference also has a pretty large booth area and as every year I organized the Fedora one. I drove by car to Prague with Carlos Soriano and Felipe Borges from the Red Hat desktop team on Saturday morning and we were joined by František Zatloukal (Fedora QA) at the booth.

František and me at the booth.

Our focus for this year was Silverblue and modularity. I prepared one laptop with an external monitor to showcase Silverblue, the atomic version of Fedora Workstation. I must say that the interest of people in Silverblue surprised me. There were even some coming next day saying: “It sounded so cool yesterday and I couldn’t resist and install it when I got home and played with it in the night…” With Silverblue comes distribution of applications in Flatpak and there was a lot of user interest in this direction as well.

Reasons to use Fedora.

I was hoping for more interest in modularity, but people don’t seem to be so aware of it. It doesn’t have the same reach outside the Fedora Project as Flatpak does, it’s not so easy to explain its benefits and use cases. We as a project have to do a better job selling it.

The highlight of Saturday was when one of the sysadmins at National Library of Technology, which is on the same campus, took us to the library to show us public computers where they run Fedora Workstation. It’s 120 computers with over 1000 users (in the last 90 days). Those computers serve a very diverse group of users, from elderly people to computer science students. And they have received very few complaints since they switched from Windows to Fedora. Also they’ve hit almost no problems as sysadmins. They only mentioned one corner case bug in GDM which we promised to look into.

Carlos and Felipe checking out Fedora in the library.

It was also interesting to see the setup. They authenticate users against AD using the SSSD client, mount /home from a remote server using NFS. They enable several GNOME Shell extensions by default: AlternateTab (because of Windows users), Places (to show the Places menu next to Activities)… They also created one custom extension that replaces the “Power Off” button with “Log Out” button in the user menu because users are not supposed to power the computers off. They also create very useful stats of application usage based on “recently-used” XML files that GNOME creates to generate the menu of frequently used applications. All computers are administrated using Ansible scripts.

Default wallpaper with instructions.

The only talk I attended on Saturday was “Why and How I Switched to Flatpak for App Distribution and Development in Sandbox” by Jiří Janoušek who develops Nuvola apps. It was an interesting talk and due to his experience developing and distributing apps on Linux Jiří was able to name and describe all the problems with app distribution on Linux and why Flatpak helps here.

On Sunday, we organized a workshop to teach to build flatpaks. It was the only disappointment of the weekend. Only 3 ppl showed up and none of them didn’t really need to learn to build flatpaks. We’ll have the same workshop at OpenAlt in Brno and if the attendance is also low, we’ll know that workshop primarily for app developers is not a good fit for such conferences.
But it was not a complete waste of time, we discussed some questions around Flatpak and worked on flatpaking applications. The result is GNOME Recorder already available in Flathub and Datovka in the review process.

The event is also a great opportunity to talk to many people from the Czech community and other global FLOSS projects. SUSE has traditionally a lot of people there, there was Xfce, FFMPEG, FreeBSD, Vim, LibreOffice…

 

Why I use Flatpak for 3rd party apps

There are more reasons why to run applications as flatpaks. Someone wants to have the latest versions as soon as possible. For me as a user of Fedora which provides up-to-date versions of apps this is not a big motivation. Someone wants to run apps more securely. Again I usually trust software provided by Fedora and Flatpak sandbox is still not as strictly enforced as it should ideally be.

But where I really prefer using Flatpak to RPM packages are 3rd party applications. I’m usually running development versions of Fedora. Pre-releases on my work machine and Rawhide on my home laptop. They have been pretty stable for me, including applications in Fedora repositories. Unfortunately it’s not the case for 3rd party applications. Their authors usually don’t follow distro development closely and although many of them bundle as much as possible to avoid problems with changing dependencies, things break.

I used to use the Spotify client as a package from Negativo17 repos. But when I upgraded to F27, something broke and it stopped working. I’m pretty sure it was fixed later on after people started reporting it, but I didn’t want to stop using Spotify for the time being and didn’t have time to debug and report the issue. So I switched to Spotify flatpak and it has worked for me ever since.

The problem I had with very early stages of pre-released Fedora and Rawhide is that dependecies of GStreamer plugins in RPMFusion were usually broken. So I ended up without system multimedia codecs. It was often the case for VLC from the same repository, too. Then I switched to VLC and GNOME MPV flatpaks. Problem solved.

The last example is Telegram. Until recently I was using the official version. It’s not even provided as an RPM package. You have to download an archive, unpack its content to your home, run the binary which creates a desktop file… not very elegant in 2018, but once you do it, it just works. Well… until it doesn’t. I upgraded to F28 and Telegram suddenly took a lot of time to start up. It hung on some font config error until it timeouted and finally started. It easily took 1 minute. So I switched to Telegram flatpak as well. Works like a charm.

So what I really appreciate about Flatpak is that the apps don’t rely on the underlying system, so if the system changes e.g. due to upgrade to a newer major version apps don’t break. As I said it’s not such a major issue for distro-provided apps, but it’s certainly an issue for 3rd apps and Flatpak solved it for me.

Flathub Experience: Adding an App

Flathub is a new distribution channel for Linux desktop apps. Truly distro-agnostic, unifying across abundance of Linux distributions. I was planning for a long time to add an application to Flathub and see what the experience is, especially compared to traditional distro packaging (I’m a Fedora packager). And I finally got to it this last week.

In Fedora I maintain PhotoQt, a very fast image viewer with very unorthodox UI. Its developer is very responsive and open to feedback. Already back in 2016 I suggested he provides PhotoQt as a flatpak. He did so and found making a flatpak really easy. However it was in the time before Flathub, so he had to host its own repo.

Last week I was notified about a new release of PhotoQt, so I prepared updates for Fedora and noticed that the Flatpak support became “Coming soon” again. So I was like “hey, let’s get it back and to Flathub”. I picked up the two-year-old flatpak manifest, and started rewriting it to successfully build with the latest Flatpak and meet Flathub requirements.

First I updated dependencies. You add dependencies to the manifest in a pretty elegant way, but what’s really time consuming is getting checksums of official archives. Most projects don’t offer them at all, so you have to download the archive and generate it yourself. And you have to do it with every update of that dependency. I’d love to see some repository of modules. Many apps share the same dependencies, so why to do the same work again and again with every manifest?

Need to bundle the latest LibRaw? Go to the repository and pick the module info for your manifest:

{
"name": "libraw",
"cmake": false,
"builddir": true,
"sources": [ { "type": "archive", "url": "https://www.libraw.org/data/LibRaw-0.18.8.tar.gz", "sha256":"56aca4fd97038923d57d2d17d90aa11d827f1f3d3f1d97e9f5a0d52ff87420e2" } ]
}

And on the top of such a repo you can actually build a really nice tooling. You can let the authors of apps add dependencies simply by picking them from the list and you can generate the starting manifest for them. And you could also check for dependency updates for them. LibRaw has a new version, wanna bundle it, and see how your app builds with it? And the LibRaw module section of your manifest would be replaced by the new one and a build triggered.

Of course such a repo of modules would have to be curated because one could easily sneak in a malicious module. But it would make writing manifests even easier.

Besides updating dependencies I also had to change the required runtime. Back in 2016 KDE only had a testing runtime without any versioning. Flathub now includes KDE runtime 5.10, so I used it. PhotoQt also uses “photoqt” in all file names and Flatpak/Flathub now requires it in the reverse-DNS format: org.qt.photoqt. Fortunately flatpak-builder can rename it for you, you just need to state it in the manifest:

"rename-desktop-file": "photoqt.desktop",
"rename-appdata-file": "photoqt.appdata.xml",
"rename-icon": "photoqt",

Once I was done with the manifest, I looked at the appdata file. PhotoQt has it in a pretty good shape. It was submitted by me when I packaged it for Fedora. But there were still a couple of things missing which are required by Flathub: OASR and release info. So I added it.

I proposed all the changes upstream and at this point PhotoQt was pretty much ready for submitting to Flathub. I never intended to maintain PhotoQt in Flathub myself. There should be a direct line between the app author and users, so apps should be maintained by app authors if possible. I knew that upstream was interested in adding PhotoQt to Flathub, so I contacted the upstream maintainer and asked him whether he wanted to pick it up and go through the Flathub review process himself or whether I should do it and then hand over the maintainership to him. He preferred the former.

The review was pretty quick and it only took 2 days between submitting the app and accepting it to Flathub. There were three minor issues: 1. the reviewer asked if it’s really necessary to give the app access to the whole host, 2. app-id didn’t match the app name in the manifest (case sensitivity), 3. by copy-pasting I added some spaces which broke the appdata file and of course I was too lazy to run validation before submitting it.

And that was it. Now PhotoQt is available in Flathub. I don’t remember how much time exactly it took me to get PhotoQt to Fedora, but I think it was definitely more and also the spec file is more complex than the flatpak manifest although I prefer the format of spec files to json.

Is not your favorite app available in Flathub? Just go ahead, flatpak it, and then talk to upstream, and try to hand the maintainership over to them.

We’re Organizing Flatpak Workshop

Several years ago, we organized two workshops focused on packaging software for Fedora. It was a success. Both workshops were full and several participants became package maintainers in Fedora. Packaging for Flatpak is becoming popular lately, so we decided to offer a workshop which is focused on flatpaking apps.

Flatpak and Endless OS at InstallFest Prague

I spent the last weekend in Prague attending InstallFest 2017. The event is called InstallFest because many, many years ago it started as an event where students could come and get help with installations of various Linux distributions. Times of installfests are gone and this event has transitioned into an open source conference with more practical focus.

The event has moved to a new venue – Faculty of Electrical Engineering of Czech University of Technology. It’s where Red Hat recently started a new open source lab. The venue was larger than the one in previous years and hosted 3 tracks + a small booth area.

I came to talk on two things – Flatpak and Endless OS. My Flatpak talk was on Saturday and got a 55-minute slot which seemed like a lot of time, but if you want to cover all the specifics of the technology, even 55 minutes is not much. The room was pretty full and the topic apparently stirred some attention. There was even one person interested in porting Flatpak to another distribution.

My talk on Flatpak

My talk on Endless OS was the first one of the second day. I only asked for a 25-minute slot which was just enough to make a brief introduction of the system. I also brought with me both Endless devices I have in possession – Endless One and Endless Mini. There were not as many people as at my Flatpak talk, but those who came seemed pretty interested. Almost none of them had ever heard of the OS and PCs before. They asked if they’d ever be available in Europe (which I couldn’t answer because I have no idea) or if you can connect extending hardware to the PCs just like to Raspberry.

As a side note, I was positively surprised how many people wore Fedora t-shirts at the conference.

Myself with the Endless PCs

 

Nightly and Wayland Builds of Firefox for Flatpak

When I announced Firefox Developer Edition for Flatpak over a month ago, I also promised that we would not stop there and bring more options in the future. Now I can proudly announce that we provide two more variants of Firefox – Firefox Nightly and Firefox Nightly for Wayland.

With Nightly, you can closely follow the development of Firefox. Due to Flatpak you can easily install it and keep getting daily updates via our flatpak repo.

As a bonus, we’re also bringing a Firefox build that runs natively on Wayland. We used to provide a Copr repository, but with Flatpak it’s open to users of other distros, too. When running this version, keep in mind it’s still WIP and very experimental. Firefox seems to run just fine on Wayland at first glance, but there is still some basic functionality missing (copy-paste for example) and it’s not so stable either (it crashed the whole Wayland session for me once). But once it’s done, it will be a big improvement in security for Firefox on Linux because Wayland isolates the application on the display server level. Together with other pieces of Flatpak sandboxing, it will provide a full sandbox for the browser in the future.

When adding more Firefox variants to the repo, we first considered using branches, but you have to switch between them manually to start different variants of Firefox which we didn’t find very user friendly. In the end, we’re using one branch and multiple applications with different names in it. This way, you can install and run multiple variants in parallel.

You can find the instructions to install Firefox for Flatpak on the repository webpage. We’re also constantly improving how Firefox runs in Flatpak. If you have any Flatpak-specific problems with Firefox, please report it to our bug tracker on Github. If you hit problems that are not Flatpak-specific, please report them directly to Mozilla.

And again kudos to Jan Hořák from our team who made all this happen!