A long time ago (exactly 10 years ago) it was decided that the the shell for GNOME would be written in JavaScript. GNOME 3 was still looking for its new face, a lot of UI experimentation was taking place, and JavaScript looked like the best candidate for it. Moreover it was a popular language on the web, so barriers to entry for new contributors would be significantly lowered.
When you have the shell written in JavaScript you can very easily patch it and alter its look and behaviour. And that’s what people started doing. Upstream was not very keen to officially support extensions due to their nature: they’re just hot patching the GNOME Shell code. They have virtually unlimited possibilities in changing look and behaviour, but also in introducing instability.
But tweaking the shell became really popular. Why wouldn’t it? You can tweak your desktop by simply clicking buttons in your browser. No recompilations, no restarts. So extensions.gnome.org was introduced.
The number of available extensions grew to hundreds and instability some of them occasionally introduced seemed like a fair price for the unlimited tweakability. In the end when the Shell crashed it was just a blink. Xorg held up the session with opened clients, the Shell/Mutter was restarted and the show could go on.
In 2016 GNOME switches to Wayland by default. No Xorg and also nothing to hold up the session with opened clients when the Shell crashes. There is only Mutter as a Wayland compositor, but unfortunately it runs in the same process as GNOME Shell (a decision also made 10 years ago when it also looked like a good idea). If the Shell goes down, so does Mutter. Suddenly harmless blinks became desktop crashes with losing all unsaved data in opened applications.
I read user feedback and problems users are having with Fedora Workstation (and Linux desktop in general) a lot on the Internet. And desktop crashes caused by GS extensions are by far the most frequent problem I’ve seen recently. I read stories like “I upgraded my Fedora to 28 and suddenly my desktop crashes 5 times a day. I can’t take it any more and I’m out of ideas” on daily basis. If someone doesn’t step in and say: “Hey, do you have any GS extensions installed? If so, disable them and see if it keeps crashing. The extensions are not harmless, any error in them or incompatibility between them and the current version of GS can take the whole desktop down”, users usually leave with the experience of unstable Linux desktop. It hurts our reputation really badly.
Are there any ways to fix or at least improve the situation? Certainly:
- Extensions used to be disabled when the Shell crashed hard (couldn’t be restarted). Since on Wayland it’s the result of every crash, we should do that after every GS crash. And when the user goes back to GNOME Tweak Tool to enable the extensions again, she/he should be told that it was mostly likely one of the 3rd party extensions that made the desktop crash, and she/he should be careful when enabling them.
- Decoupling GNOME Shell and Mutter or/and other steps that would bring back the same behaviour like on Xorg: GS crash would not take everything down. This would require major changes in the architecture and a lot of work and GNOME Shell and Mutter developer community has already a lot on their plates.
- Discontinuing the unlimited extensions, introducing a limited API they can use instead of hot patching the GS code itself. This would be a very unpopular step because it’d mean that many of the existing extensions would be impossible to implement again. But it may become inevitable in the future.
Brno hat 
Firefox recently solved similar problem with something like option no. 3. They replaced previous extension capabilities with Web extensions. That change wasn’t that bad, because many of current extensions could be easily ported into new system. Also Chrome extensions became compatible (or required not much changes), which was a good for developers, as now they have 1 API for all browser extensions.
However, in Gnome case loosing current extensions won’t be good.
I believe all of 3 solutions from post could be used as solution for the problem.
Creating limited API from option 3, will allow developers to create “safe” extensions. They won’t be as powerful, but because they use only limited functionalities they shouldn’t also cause crashes. If they do, then probably it’s an API issue, not extension.
However if the Shell crashes, the option number 1, will come into place. All extensions not using API will be disabled and user will be asked to be careful when enabling non-API extensions.
While this is not as difficult to implement, it may increase stability of GS, as well as give time to develop solution number 2.
Also by allowing 2 types of extensions gnome devs will gain more time to polish API and make it more powerful.
Extensions API seems to be also providing more solutions:
– It will be a good step forward into making extensions compatible when GS is getting updated, as the compability will be based on API version rather than Gnome. Changing Shell internals may only require updating API, not all extensions.
– Easier extensions developing – as API will have documentation
– cross-desktop extensions maybe in far future
> Firefox recently solved similar problem with something like option no. 3.
> They replaced previous extension capabilities with Web extensions. That
> change wasn’t that bad, because many of current extensions could be easily
> ported into new system.
Whilst I understand that’s not your point, I wanted to note it *was* very bad, this decision made a big technical auditory to leave, and led to a lot of bad for Firefox’s reputation posts. I certainly know, I was one of them. And, maybe because Mozilla didn’t have resources, but lack of communication from their side made the situation even worse. Now there’s a bunch of people who would never recommend Firefox to anyone.
Although I still wish Mozilla luck — but it’s for irrelevant reason, it’s because they’re the developers and supporters of Rust.
—–
But it’s a bit offtopic, and I don’t know the situation with extensions on Gnome to make any valuable comment here.
Yeah I guess some people would not want to give up their developer freedoms they once enjoyed. Therefore *all* three options should be pursued, while extensions not using the limited API would require review, hopefully limiting them to good ones.
Firefox breaking plugins *was* painful… but there’s no question it was vital for them to do so. Having no defined API and allowing plugin developers to consider *everything* as usable… well, that was never going to be sustainable. And now Shell have the same problem for exactly the same reasons… they need more control over their internals, but can’t gain it without breaking compatibility.
The lesson is simple – extension APIs *must* be constrained by design, right from the start.
I’m a big fan of option number 1 and option 4 (detailed below). When I updated my desktop to Ubuntu 18.04 I was unable to login due to an extension crashing the desktop. The problem ended up being one of my extensions doing a gsettings get on an option that didn’t exist anymore. Number 1 would have made it so that I could login without digging through logs (which would not have been an option for a large portion of the userbase).
An even better solution would be to
4. Improve the code quality and make it handle errors in a more robust way. There is no need to have the desktop crash if a gsetting is missing. This was just my one use-case but I would bet that this approach would help a lot of desktop crashes. I’m guessing it’s not just the gsettings interface that needs to be updated to handle errors. Errors are a lot of work to handle correctly so I know this isn’t an easy approach but taking some of the burden* for these desktop crashes would help.
* As someone who experienced the bug and did not report it I know that I neglected my part in this.
A very stupid question by me: is it not possible to wrap extensions’ with a try/catch statement (or whatever) so to handle errors at higher level? As soon a crashing extension is identified, it may be disabled and even removed (or marked as “unstable” until an update is installed).
Hi Bob,
No, that’s not possible, extensions literally overwrite the code of GNOME Shell, akin to a runtime light fork. It’s called monkey patching in case you want to look for more on internet.
Cheers
At first I thought, it should definitely be option 1 and 2, because that leaves the most freedom to developers. But after a bit of thought, it seems option 3 is better. Limiting the available API to View stuff and trimmed down data API would aid in the separation of concerns. Extensions would be rather dumb consumers of data API and be responsible for display. Templating brought to the desktop in a sense.
Apart from GNOMEs data API, consumption of own API should be possible, e.g. via web service running either remote or locally. That allows you to do the heavy stuff in other languages than JS. GNOME should integrate the operation of local web servers, i.e. a daemon control.
Option no 5: fix stupid decision made 10 years ago and drop JavaScript. Replace it with gtk4, qt or rust.
Having all those extensions means nothing if only handful are used. Don’t be the people who made measure (number of extensions) to be a target. Check goodharth’s law for consequences.
Changing the language in which the extensions are written would not change anything. They still would be able to crash your entire desktop.
What is needed instead is a fault tolerant API which can be used by the extensions, like normal applications are handled by the kernel. If the extension makes a mistake (violating the rules of the API, like an illegal memory access), it should be shut down and disabled.
PS: As an intermediate option you could allow for safe and unsafe extensions. Safe extensions would only have access to the fixed API (as limited as it might be at the start) and unsafe extensions could still be used in the meantime, discouraging their use as API is improving.
Fair point.
Still, having JavaScript a selected language, you get shitty result as we can see. Not to mention performance problem with shell. And now, solutions will be invented for problems that shouldn’t exist in the first place.
I feel that selecting JavaScript as a shell language was a wrong decision based on this article. If this is so, I completely agree.
If we know that wrong decisions have been made, it needs go be fixed. Popularity of JavaScript does not make it a good choice for everything just because there will be more development. We should aim for quality and ‘quality’ and ‘JavaScript’ can’t be used at the same time.
Having api would make it much better choice, that’s for sure, but dropping this abomination of a language from the desktop would have its benefits for sure.
Agree.
Agree to option 5: just remove JS extension.
As a user of several GS extensions I have experienced several crashes caused by extensions, I’ve experienced what the change from Xorg to Wayland meant from this point (and still feel it due to gnome-shell restart with Alt+F2 -> R doesn’t work on wayland) I understand the risks of having the extension system, BUT I am always showing off people Gnome Shell, and I’ve not seen a single one (although have successfully helped 10+ users work on Fedora instead of *****) using Gnome Shell with defaults. It might work for some users, lightweight usage, and I see that usability for normal users is kept in mind, but this seems to affect default behaviour for users e.g. with multiple (e.g. at work everyone is using 3 ) displays. So for the options:
1. is too agressive in my opinion, as the user still would have to manually figure out which is the problematic extension.
2. means too much work, and I understand that GNOME does not have the resources to do it
3. seems the best option to me, but carefully choosing the proper extension points (e.g. based on extension installations count from extensions.gnome.org – if that’s possible from privacy-related views) and providing proper documentation for migration/implementation.
In any case, I volunteer to work on any of the areas if there’s something planned, anything else but dropping the extension system to increase GS stability. GS is fairly stable, with carefully chosen extensions, so unless the maintenance burden is not accepted even with people volunteering, it should not be dropped IMO.
I wonder whether Gnome Shell will be ported to GTK4/GSK and how it influences extentions.
1) Learn from the past and don’t make the window manager and the X provider the same thing. One of the reasons for the longevity of X is just that (along with laziness/”don’t fix what ain’t broke” and many other reasons). This just has to be good thing – a crash in one program should not crash another (especially from the user(s) viewpoint). So point 2 I guess.
2) API is a good thing – until (at some point in the future) it’s not – if it lasts as long as X then it’s probably worth the cost albeit storing up technical debt all the time. In any case you can provide a “basic” API and add to it over time (and possibly, in the mean time pee, off a lot of developers). Option 3 I guess.
3) Blame is always a hard sell. It’s actually an attack on someone’s code. “My code is the best code ever” – even if it’s not. There are often emotions involved (especially in open source). In any case, if the user likes or (at least thinks they) need the extension, they’ll keep using it. No gain in effect. M$ did try this in the early/mid 90s with the graphics side of things (so Windows/DOS with Windows on top) – and ended up defining a API. They’re were in a different position at the time – “this is my software that you pay me for and f**k you.” I don’t think Gnome would benefit from such an attitude. I think I’m not in favour of blame.
Speaking as someone who had a laptop without Gnome extensions that switched to XWayland (was Fedora 21/2/3/4/5????) and experienced crashes (yes uploads and bugs were opened). It was irritating in the extreme and I could see people, (even knowledgeable people) saying things like Gnome is buggy/full of sh*t, etc..
Gnome used to have a lot of “nerd knobs” back in the day, then Gnome decide to scale that back. Now the nerd knobs take the form of extensions (sort of). What do we want? A customizable desktop? Something that somebody (even with a bucket load of studies behind them) dictates to us? I suspect the former.
There’s no easy answer here.
You mean compositor and shell. Window Manager is something completely different as it can reside in either of the two and its sole responsibility is painting borders and having that in either won’t solve absolutely anything
The only real long term solution is option 2. It needs to happen anyway as it’s not only extensions that are a problem, but all that Javascript in the compositor just makes it too brittle to handle.
We have the Enlightment option too, we can make the session survive a crash of the compositor. That means fixing GTK to make it able to reconnect to the Wayland compositor after it restarts. It also means having something else (gnome-session or systemd) start programs and kill them when the session exits.
Option 3 is not really a fix, because even with a limited API, as long as extensions are running inside the same main loop as the sheet, they can always run in a loop forever or do something else that can cause a freeze or a crash. Firefox is very different because they already have a complex system in place to protect the browser from the web content that they could re-use. In gnome-shell, we don’t have this. So there is no option 3.
Or maybe just make GNOME shell more configurable out-of-the-box? Who would need an extension to change the date and time format if GNOME Settings actually had settings for this? Or an extension to move the launcher panel from the left to the bottom of the screen if there was a native setting for it?
Or example an extension allows to set different wallpaper per workspace? Hmm, this is the 17-year users’ request ( https://bugzilla.gnome.org/show_bug.cgi?id=48004 ). Never implemented…
From engineering point of view, 2) is the most sane thing to do. Together with 3). The latter would increase testability of the extensions. I’m (sort of) maintaining one extension for GS and testing is currently really hell. First of all, I’d need to test very old shell versions and provide work-arounds to support them. And then, since using Wayland, the debug/run cycle is nightmare. Which is the main reason, I don’t really have time to test and provide updates for new GS releases.
Having said that, I’d gladly port the extension to a new API, say for GS 4.0.
Mozilla did a bad job communicating the drop of XUL, since at least some developers and most users were unprepared. Nevertheless, getting rid of XUL made a lot of sense and I’d support GNOME if they are doing the same. GS 4.0 should be a clean cut, switch back to GTK+4.0, getting rid of Clutter and St and have a hopefully rich extension API which is stable across GS releases.
That says more about the add-on maintainers than Mozilla though. Mozilla were publicly discussing the removal of XUL many years ago, and by the time the official deprecation was announced, it should have come as no surprise to anybody. And the current Web Extensions framework was added in 2015, so while it took a while to build the necessary APIs, add-on developers had plenty of time to figure out what they had to do…
No, Mozilla *was not discussing*, that was the main problem, even more than API-drop per se. They simply notified “we gonna drop XUL”, and went silent. Developers and users started creating bugreports for the missing API, but Mozilla for the most part ignored them.
And there was no time since 2015 to port extensions — because there was no API to port to. It’s not there even now — more than year into XUL-less world.
You see the main problem: it would be okay if they had to remove API that’s holding them back — but first there have to be something that extensions could be ported to. In ideal world there would also be some time for porting. But neither time, nor API was given.
The first blog post about dropping XUL appeared in August 2015, see Add-ons/developer/communication on the Mozilla wiki. The Web Extensions page on Mozilla Wiki together with “Porting a legacy Firefox extension” on MDN both appeared in July 2016. You could experiment with Web Extensions in Nightly at that point, and I think in Firefox 51 released January 2017 you could wrap an add-on using Web Extensions and ship it in both old and new formats.
I think that you’re complaining that Mozilla didn’t provide *specific* APIs needed by many extensions in a timely fashion. Well, that’s the whole point of option 3 (a limited API). It takes a lot of time and expertise to craft a single good programming interface; completing a full-coverage API surface takes years and you won’t get it right without feedback which means forcing developers to transition to it before you have the complete API. GNOME Shell will have the same problem if it adopts option 3.
Why do extensions exist and why are they useful?
It means GNOME doesn’t have to implement a lot of things that the community may want. This means a distribution like Ubuntu doesn’t have to fork GNOME Shell just to add the UI customizations it wants like a dock or custom panel applets. This is a huge win for GNOME and a very good thing to prevent endless fragmentation.
One of the major issues with extensions isn’t listed in the main post.
4. Lack of feedback loop from the extensions community back upstream.
This is the biggest problem. When extensions become extremely popular to the point of becoming “must haves” there should be a process where they can become part of the GNOME Shell codebase upstream. The option to enable them should be in tweak tool as an “advanced” feature.
With regards to the other points 1, 2, 3…
Extensions should never crash the shell ever. The shell and mutter should be part of a multiprocess architecture like Chromium. If one component of the shell crashes it shouldn’t take down the whole system, that part should just crash and restart. Session state should always be stored somewhere in an external server and restored when needed if the entire desktop crashes. Extensions should have access to a low level API that enables much of the functionality found in extensions today. If an extension loads and causes issues the API should gracefully disable just that extension and report it to the user.
It should be very very difficult to crash the shell entirely and take something really catastrophic to do so.
It’s a lot of work but it’s essential that the shell is robust as a basic requirement.
Except that if you’re monkey patching the gnome code, then you *are* forking the code. It’s just that now your fork is being forcibly rebased onto the latest gnome code.
This means that zero qa is being done for any code/extension pairing.
The question that needs to come before considering a new extension system, is why do they exist, and can popular extensions be brought back into the main system.
I spent far too much time trying to get a good cpu meter working, and then have up, due to either visual or functional bugs.
You can reduce the pain caused by faulty extensions, by removing, as much as possible, the need for them.
I’m a big fan of Gnome, but I feel like it needs some serious improvements in performance and stability. Can we get a Gnome Shell 4 please? Let’s rewrite it in Rust, make it multithreaded and multiprocess, make it stable, and make it fast.
I also like the idea someone else posted, of having two ways to do extensions: safe, with the limited API; and unsafe, with full access. That seems like the best of both worlds.
I think option 2 will need to be done anyway, as crashes are not only caused by extensions.
In addition to that, how about a combination of options 1 and 3: create an Extension API. When the shell crashes, keep the extensions that use the API enabled and disable the others.
If you’re going to support extensions at all, option 2 is non-negotiable, really… running everything in a single process is poor design by any modern standards… and doing it with a platform that allows monkey-patching is somewhere on the far side of outright insanity.
I think option 3 is also essential. The comparison with Firefox is obvious – while the versatility of unrestricted extensibility was a big selling point in the early days, the downsides of that feature came very close to killing them, proving a huge impediment to their ability to deliver the architectural changes needed to keep them competitive.
Sooner or later, Shell will find themselves forced into the same decision… and it’s always better to address problems before they become critical…
Incidentally, it’s interesting reading that “ten years ago” link you posted… where a significant number of the comments back then were raising concerns over the single-process model, and how it would have exactly the same problems you’re now talking about…
In long term option 2 is a real fix, this is means resilience! Because “cannot happen” happens in real world. Probably a ugly wrapper, a more elegant layer, a library function or a daemon like GDM can provide this.
Maybe we should look further on Gtk4 and a possible GNOME 4, which could be GNOME 3 but with internal modifications. This allows for replacing JavaScript by a typed, compiled and native language:
* No more problems with garbage collection, interference and other weird problems, either RIAA or Reference-Counting works (this just depends on the implementation language, like C and GObject, C++, maybe Rust..)
* Possibility of reduce memory consumption and increase overall performance
* Can provide a defined API for Extensions, especially because GNOME avoids beloved configuration options
* Ahead-Of-Time compilers and modern tools like address-sanitizer give us more safety
Clearly nothing for a quick fix. I’m afraid GNOME was trapped in an usual “hype”. Everybody in 2010 believed to use JavaScript everywhere, but it is just made for the web and not for desktop, reliability and performance.
Hi,
Why don’t mutter implement arcan (https://arcan-fe.com/2017/12/24/crash-resilient-wayland-compositing/) ?
Best,
I think option #3 is the best. Totally I think extensions for desktop are bullshit! I don’t use any extension on my GNOME Shell, maybe it was better to implement GS and some extensions as official in the C language and totally ignore JavaScript!! What you think?!
Use an API to allow DOM changes and limited functionalities, rest of the application should run in a worker or separated process with i/o support and the communication with the main thread with a standard limited protocol that ensures no heavy process can be executed and there are memory limitations.
This way mostly if an extension crashes it won’t affect the GNOME Shell, however, this will create limitations on how much customizable it can be. A good trade-off, especially if those applications can have their own rendering window like chrome extensions.
Jiri, thanks for the insights. Interesting reading your thoughts and the comments at this time, since I just dealt with a crashing classic apps-menu extension in a fresh Antergos install. I’m not a super savvy dev – just a mildy savvy power user, and it took some digging to figure out a package providing GMenu support was left out by default in this particular distro.
Right now I have a total of 19 shell extensions installed and most of them provide a real benefit to my workflow in terms of features and helping the DE “get out of the way”. Honestly if I lose a few of them I might stick with Gnome, but at some point I’ll just look elsewhere. In the early days of Gnome 3, I bristled at the zeal with which configuration options were denied or removed, but I could look past it because at least the DE came with sane defaults, looked good, and was very extensible. But ultimately there seems to be an internal philosophical conflict with the “simple, everything just works” aesthetic and “well if you need more features go find an extension” mindset. That conflict arises for precisely the issues Jiri raises again today — broken extensions break the experience, but the Gnome devs can simply write them off even if one or more of them are critical to a user.
I really hope in the long term a refactoring of Gnome that makes it crash resilient and recoverable, while also fast and extensible can happen. It’s a shame that limited resources are a major factor when Redhat, Canonical, and others are shipping Gnome as their default. I realize they make their money on the server, but Linux wouldn’t be in the enterprise without Linus Torvalds and crew wanting a good unix experience on a PC.
Sorry for the long winded comment without putting forth a real solution, but I do care where Gnome goes in the future, and I’m happy to contribute and I hope others are too.
Is is interesting to read the link Jiri has provided, you should really read it 🙂
> Javascript doesn’t pull in another complicated platform, almost everyone is familiar with it to some extent another, it offers good possibilities for sandboxing applets, it’s pretty light-weight for memory, and there is a lot of work going on to make it fast.
1.) It has pulled in another complicated platform
2.) Neither everyone is familiar with it
3.) Nothing is sandboxed, especially our extensions
4.) It’s memory consumption is a massive problem now and GNOME is working hard on fixing some of them
5.) And it’s damn slow compared to native
That hurts.
It took a decade but finally I feel justified!
I was the one of those people who had the obvious concerns about gnomeshell’s architecture but no one listened to us.
it’s just too hard for me to believe this piece of software was created by professional developers and took them a whole decade to admit their mistakes. But also can’t believe they intentionally wanted to be defective by design. Something feels off here.
Multithreading/multiprocessing paradigms of course weren’t something new a decade ago and even chrome, the most popular web browser, which predates gnome 3, proved that as a viable way but gnome didn’t follow for some reason we will never know.
The technical debt they caused is by far the largest ever happened in the history of linux desktop.
I’m not even sure if this is funny or sad.
Please drop extensions and JS from gnome. Thanks.
The point 2 is a fix for an architecture bug and need to be fixed not matter if the point 1 and 3 are apply or not. The point 1 is essentially an improve and is also recommendable to be apply (but optional). The point 3 is the critical one because respond of a philosophic interpretation of developers and users of what they want to have in his desktop. I can only provide my opinion in that point and this is that i prefer flexibility more than the security, because I can restrict what i install always if this is what i want. To me, anything that removes or hides the user’s power of decision, is not an advantage but an inconvenience instead.
I run Ubuntu 18.10. GS extensions are a must and I cannot leave without many of those. I still use Xorg.
But they crash Gnome Shell a lot. Currently when my computer turns off the screen, for power saving, I am left with a dark gray background and the top right system menu, after waking it up. The workaround I found is to make the computer sleep, then on wake up, I get the lock screen login and can access my desktop. Except Firefox that I find in a freezed state and that I need to force close and reopen.
I need a clear way to identify the extensions causing the crash and to report the error to the developer.