What Apple gets right and why Linux keeps slipping behind

I'm a huge Free Software enthusiast. I founded the Open Graphics Project because I want to promote Free Software. I want people, users of computers, to have freedom and control over their computers. I don't want greedy corporations to limit our freedoms and hold us hostage to their DRM, unreliable licensing mechanisms, and a tendency to leave us holding the bag when they go out of business or decide that we haven't paid them enough money. I believe that if you have a piece of technology in your possession, then you have every right to take it apart, understand it, reverse engineer it, and use it however you like, until you decide you no longer want to. Of course, there are limits. Other people have rights too, and rampant piracy is not a good thing, but that is beyond the scope of this blog entry.

So, I think Free Software should rule the world, and I believe that it is an ethical issue, not merely a practical one. I feel that I am a minor activist in this regard. The more Free Software that people use, the more freedom we'll all have, and I am trying to do my part in my own community project.

At the same time, I'm also an engineer and a scientist. I think about the systems I use, critique them, and consider how they can be improved. As part of my Ph.D. studies at OSU, I'm minoring in Cognitive Engineering. Putting Freedom into people's hands is useless when those people can't figure out how to use what they have. Good design and usability are very important. I haven't paid enough attention to the recent discussions between Linus Torvalds and GNOME developers, so I can't address it directly. But what I can say is that a learning curve is not a bad thing. While it's good to think about the total novice, it's even more important to have consistent and logical mechanisms. This way, if someone has to learn something new to use the computer, they have to learn it only once. This is why I think it's good that Apple and Microsoft have UI development guides that encourage developers to make their apps act consistently with other apps in areas where their functionalities conceptually overlap. And this is where I start to get disappointed with GNU/X11/Linux systems.

This is a blog, not a book, and I have work to do, so you'll forgive me if my points of discussion aren't very well organized. If this were a professional article, I would have researched things more carefully, providing detailed citations. I provided a long intro to set context and explain why I think it's important that Linux systems should make some very radical catch-up changes to the organizations of their systems. What follows are some ideas regarding areas that, once fixed, would bring Linux systems into the 21st centrury and make them compete as well-engineered systems against the likes of Apple. Microsoft has copied a great deal from Apple, but they clearly don't grok the underlying concepts that Apple tries to employ. Alas, I fear that Linux developers also don't grok these ideas either, and also like Microsoft, they're too afraid to change, sticking to out-dated methods in the name of backward-compatibility or inertia or both.

Application executables and resources

Apple didn't invent the idea storing all application-related files in one place. I think I read somewhere that an early version of RISCOS did it this way, but Apple has adopted this idea very nicely, and Windows isn't too far behind. The basic idea is that most files for an application are stored in one single place. The application NeoOffice, for instance, has a directory (anywhere you want to put it, really) called NeoOffice.app. Under that directory are all of the binaries and data files associated with the base install of that app. Some structure is imposed so that MacOS knows where to look to find things, but the main idea is that if you want to run or manage this app, there is this one place to look. Installing or uninstalling is a matter of drag-and-drop. Running the application is simple too--double-click on the directory, and the right thing happens, with the folder acting as a representative, to the user, of whatever is in the folder. For the end user, this keeps things painfully simple.

Windows isn't too bad on this front. Under the Program Files directory, you find most of your major apps. The only thing they're missing is the directory-as-proxy mechanism. You have to know a little more in order to find the executable and run it, if you haven't already got a link to it somewhere.

Linux, unfortunately, sticks to its UNIX heritage of spreading things out all over the file system. Executables to in /bin, /sbin, or /usr/bin or /usr/local/bin. Shared objects go into /usr/lib or somewhere under /usr/local. Configuration files (all plaintext; see below) go somewhere under /etc or somewhere under /usr/local. There is a standard that describes in detail how things should be structured, but it's much too brittle. In theory, conventions, even complex ones, can be strictly followed, and everything works out well. But this is an example of a convention that adds just enough complexity and confusion that it doesn't get followed consistently, and the one left holding the bag is the end-user who can't figure out where their files are when something goes wrong. I've been using Linux a lot longer than MacOS, but while I can easily find whatever I want pertaining to an app in MacOS, I am clueless about Linux. And I've even installed a good number of Gentoo systems. The distinction is that the UNIX way is logical but arbitrary. The Apple way, on the other hand, is simply intuitive. That's why it's easier to remember and use.

Here, like for most of this article, I'm going to be pragmatic and suggest that Linux people adopt a Microsoft-like strategy: If you see someone doing something in a way that works better, adopt it. So, the solution for Linux systems is to gradually deprecate all of this /bin, /usr, /etc confusion (except perhaps for the most basic of system tools like find and ifconfig) and adopt a system that collects all files of each app into its own directory. And this should be done even if there is some redundancy! I think one of the ideas behind the UNIX way is that many apps will share resources. This was a good thing in the 1970's when resources were scarce. Today, however, this sharing often results in version conflicts that break apps and make life hard for users. Think of the users and make things intuitive, even if it results in some minor increase in complexity or redundancy for software developers.

Configuration data

Microsoft almost got this right. With the introduction of Windows 95 came the introduction of a centralized registry for all system and application configuration data. It's a hierarchy of key/value pairs accessible from any application. In principle, this is a rather nice way of doing things. The OS provides a simple mechanism for apps to keep settings in a database.

Users of Windows and other systems panned this approach for one good, practical reason: It's unreliable. Although things have improved, registry corruption is still the bane of the Windows user. Install a new app, and it or the OS does something weird, and bang!, the whole registry is hosed, requiring a complete reinstall of the OS. Single points of failure are bad.

Because of a strangely black-and-white sort of thinking, GNU/Linux systems have avoided anything remotely like a registry and stayed stuck in the dark ages with respect to config files. Stored in seemingly random places, every application keeps their own config files in their own special formats, in plain-text ASCII. Despite being littered with comments, most of them, when you can find them, are basically impossible to figure out. Some apps provide GUI tools to edit them, which is great, but they're not always available.

But an even bigger problem, the one thing that made me ditch Gentoo completely and get frequently very annoyed at Ubuntu, is the nightmare of application upgrades. If you have made any changes at all to an application's config data, and then you want to upgrade that app, you have three choices: (1) Keep the old config file, not benefitting from any new options or defaults provided by the upgrade; (2) use the new config file, tossing out all of your changes, requiring you to redo them all; or (3) go through a painful largely-manual merge process, often guessing about the meanings of any new options you find and whether or not your old change is appropriate for the new version. From first-hand experience, I can tell you that this is a very tedious and error-prone process that can leave applications completely broken. For instance, every minor KDE upgrade under Gentoo would leave me finding that some standard KDE app was completely missing, and no one in the Gentoo forums could help me figure out what happened. (And that was after manually sifting through a few hundred config files that were updated.)

This total lack of a standard is a huge embarrassment for Linux.

Apple, as always, struck an elegant balance. In two intuitively predictable locations (one system-wide, the other per-user), you can find a big directory full of property list files. Each .plist file is associated with an application. This approach solves a number of problems. First, an application editing its own private registry won't corrupt anyone else's, because they're all in separate files. Secondly, if you want to find the config settings for an app and edit them directly (does happen now and then), you can isolate them, back them up, delete them, or whatever, without affecting any other apps. Bad things happen in MacOS just like in Windows. But while removing the registry entries for a Windows app can be difficult and dangerous, dragging a .plist file to the trash under MacOS is a simple and safe way of fixing many application problems. On top of all of that, upgrades are always easy, because structured config files are easy to merge. The format of the .plist files is either ASCII XML or a binary encoding of the same data (faster and smaller but not editable in vi).

My suggestion for Linux: Switch to using ASCII XML config files for all apps, highly restricted to store only key/value pairs, and organize them all in a single central location. Provide a shared library that all apps can link to that manage these mini-registries. Make virtual merging of system-wide and per-user config files seamless and automatic in the shared library, often making upgrade merges unnecessary. With this approach, management becomes as simple as it is for MacOS. Upgrade merges become easy, because a generalized system can be used on any config files, regardless of which app they're for. We could also take it one step further and provide meta-data to the generalized merger that tells it how to resolve conflicts between upgrade values and user-supplied changes.

Clipboard

X11 basically lacks a clipboard. Instead, the basic X11 cut/paste mechanism is based on a convention of storing information in atoms on the root window. This is another example of a lack of standards. Due to the severe limitations of the basic convention, the desktop managers have devised their own extensions to it in a valiant attempt to add some usability. Unfortunately, they all do if in different, incompatible ways. As a result, GNOME, KDE, and Motif apps don't interoperate well. For instance, one of my favorite editors, Nedit, won't work correctly with the clipboard under KDE or GNOME. The problem here isn't a technical one, however. Like so many other problems with Linux, it's all about ego. Everyone thinks their way is the best and has no interest in adapting to other conventions. The developer of Klipper (KDE's clipboard manager) has even gone so far as to publicly refuse to work with other developers to resolve these problems. He wrote a long article, describing the challenges and his solution (a very sensible one), but then goes on to basically say that it's everyone else's responsibility to adapt to his way. I and others have tried to talk to him about the problems of Motif/Lesstif, but he just blows them off. The fundamental problem is not that he won't work on the other apps (that would be unreasonable, because he's a busy person), but that he won't even talk to other developers and try to come to a working compromise. GNOME has similar problems. Of course, they're not totally to blame either; I've also filed bug reports on Lesstif, just to be completely ignored. No one wants to work together.

Some links on this topic: Article by the Klipper developer, Ubuntu bug report, Nedit list discussion, Lesstif bug report, KDE bug report, Another KDE bug report.

Ego is the problem

Here we are exposing one of the fundamental problems of Linux systems. It's all about ego. Windows users are elitist snobs because they use the dominant platform. Apple users are elitist snobs because they use the "cool" computers. And Linux users are elitist snobs because they use Free Software. What sets Windows and MacOS apart from Linux is that Linux doesn't separate the developer community from the user community. This has obvious advantages. But it also has some major drawbacks. To a noticeable degree, Linux snobbery permeates the development process, while Apple and Microsoft keep a much cleaner separation. Most Linux developers do what they do because they have a passion for it, which is great, but that also means they are sometimes stubborn and unwilling to cooperate with others who think differently. At Microsoft or Apple, a committee decides on APIs and system conventions, and then some coder is paid to implement it that way; as a result, applications see a consistent organization and set of system services. Linux follows a completely distributed development model, where often, it is individual developers who make unilateral decisions about system organization and architecture, quite frequently eschewing established standards (when there are any). I don't know how to fix this problem, but I believe it is a major barrier to Linux's success on the desktop. If it can't be fixed, then Linux and Free Software won't dominate.

This reminds me of a discussion I observed when I was on the Linux Kernel Mailing List. Kernel messages through a user-space daemon to be logged. In general, because message strings take up space, we would prefer to not keep them in the kernel. Instead, if we could number them, we could then have the userspace daemon translate them on their way out to the log file. Many people agreed that this would be a good idea. But it was ultimately rejected. Why? Because there could not possibly be a central authority who could manage the numbering of these messages. Kernel development is just way too distributed. What a pity. As it turned out, however, the messages only took about 6% of the kernel space in the first place! So, really there was no big loss. But this is a good, clean, sad example of the sorts of challenges that Free Software developers face that are not problems for the commercial developers.

Perhaps someone will respond by saying that too many standards restrict creativity. But I agree. If the system architects try to dictate everything down to the finest detail, then you'll end up with a monoculture, and everything will be totally boring. But notice that MacOS isn't boring, despite having standards for things that Linux doesn't even address.

My suggestion to the Linux community: Put aside your egos and work together. Stop all of your egocentric bickering and let reason work out who is right and who is wrong. Develop good standards and stick to them. Of course, no one can develop the perfect system, so you eventually have to declare a version "good enough" and move on. But be willing to make drastic changes when a better idea comes along! Empirically, we can see that Linux does many things in a way that is inferior to other systems; adopt those ideas quickly. And most importantly, put some thought into how the end user (novice and power user) will interact with the system, at every level. Taking a lesson from Donald Norman, we should strive to put more knowledge into the world and reduce how much knowledge must be kept in our heads. If you have to make a choice, favor intuitive options over ones that require learning, and when something does have to be learned, be sure that your new convention applies as universally as possible, minimizing what has to be learned in the future.

Conclusion

One of the things I'm implying here is that the Linux community should form standards committees. I've been indirectly involved in standards committees, such as the VESA DPVL committee, and it can be a painful, glacial process. But this pain is necessary if we're to come together and develop elegant systems that work well in the enterprise and on the desktop. Indeed, there are already many Free Software community organizations that develop standards. DRI is one such example, providing a rendering infrastructure that fits into Linux, *BSD, and Solaris. Similarly, my friends with the Open Hardware Foundation are working hard to bring together hardware vendors and projects to develop hardware standards for the good of Free Software. It's been pointed out that Linux isn't any more of a bazaar than Microsoft is a cathedral, so let's not be afraid to develop hierarchy, particularly ones that make it easier for us to nail down what would otherwise be arbitrary choices anyhow. Linux distros and community projects can add representatives to committees that decide things that would constitute sweeping changes to the way we do things. Without a democratic body to do this, it may be impossible for us to identify, decide, and make the changes that we really need to make to keep up.


Update

[10:07pm] A very kind commenter was nice enough to point out GoboLinux. This takes the NeXT/MacOS approach even farther and packages even basic system tools, like the BASH shell, in their own application directories. Also check out the Wikipedia article on GoboLinux for an overview. I'm downloading it right now, and perhaps in a later blog post, I'll mention some of my impressions. Just because the files are organized better doesn't mean it's got the same surface polish as, say, Ubuntu which is great to use unless you have to tweak the underlying system directly. So we'll see. Thanks!

[10:48pm] Well, I downloaded the GoboLinux ISO and tried to install it under Parallels. Parallels instantly pops up a bug report dialog, telling me that there's been a fatal error in the VM. So I submit the bug report. Speaking of Parallels, I've tried emailing them on a number of occasions regarding technical problems, only to be completely ignored. One time, I emailed them just to ask if they actually ever respond to customer emails. They ignored that too. The email is getting through, because I do get the automated response. It seems that Parallels are not too interested in customer service. Especially considering that they charge you the full price every time you upgrade, I have no incentive not to consider something like VMware instead next time. It's probably rude, unprofessional, and inappropriate for me to make a comment like this in this context, but it really makes me angry when I pay $80 for a piece of software, and the vendor treats me like I don't exist. It's one thing if you don't get support from projects done by unpaid volunteers, but it's entirely another when you pony up money for a broken piece of closed-source software.

[11:06pm, March 31] Still haven't tried GoboLinux. I'll have to try a live CD on real hardware some time soon. Some people mentioned also "klik", which I think is also a brilliant solution. It looks to me like I am not suggesting anything that hasn't already been tried. Well, it was already tried on MacOS and whatever it inherited from, but what I mean is that there are Linux people out there who are actively working on these ideas too. Some combination of GoboLinux and klik, I think, is exactly what we need to get out from under some of the architectural cruft that holds Linux back and start becoming competitive. Ubuntu does a great job of putting a nice face over a disorganized mess, but that's all it is -- a bandaid. We need to design systems to be intuitive from the ground up, not paste make up on top of blemishes. Once all of that is fixed, we can start looking at what another commenter mentioned: How the Mac does code reuse brilliantly.