You’ve just downloaded a feature-packed update to your favorite open source app. Everything is working well, and you use it on your other devices — so it is time to roll it out to those, too.
Except your shiny new Linux laptop isn’t compatible with your Windows installation package. How about your Android tablet? iPhone? PS4? Why can’t you just take that piece of software and use it wherever you like? Let’s explore some different barriers to the dream of “buy once, run anywhere.”
Software Development and OS Architecture
Understanding why software doesn’t work across operating systems requires a little (just a little, I promise) knowledge of how software is made.
The Software Development Process
In a very basic software development flow for desktop, server, and mobile (i.e. not web), a programmer will:
- Type some code into one or more files.
- Compile the code into something the computer can execute.
- Test to make sure the program works as expected.
- Package and distribute/deploy the software.
It is a combination of the first and second steps that concerns us here. The process of compiling software, or turning it from code into the ones and zeroes that a computer understands (machine language) is complex. We won’t get in to it in great detail, but it is useful to understand at a high level what happens.
One important point to understand is that an operating system isn’t a single entity. Rather, it’s made up of layers of software.
Operating System Kernels
An operating system’s kernel is responsible for communicating with the hardware of the computer. Software communicates its commands to the kernel, which in turn issues commands to the hardware to (for example) read a file from the hard disk, or draw a window on the screen. It basically coordinates all the information (whether it’s stored data, calculations, or user input) between hardware and various pieces of software. The kernel makes all this functionality available to software via system calls.
Each operating system’s kernel will implement system calls differently, in terms of which ones are available, what they’re called, or what options they take. As a result, software needs to take into account the system calls supported by the kernel of each OS it targets. The system call you use to send data to the GPU in Linux may have a different name, list of information you need to provide, or both in Windows. That exact call may not even be there at all.
In many cases software doesn’t call directly to the kernel. Instead, it calls to system libraries, or collections of basic functions. Libraries exist so (for example) each and every program that saves files to the hard drive doesn’t need to write a function to do so. Instead, it simply links to a system library and uses an existing function. The GLibC library for Linux is a prime example, as are the .DLL files in the Win32 API or the contents of a Mac’s /System/Library directory.
System libraries act as a kind of translator between applications and the kernel for routine tasks. Applications make function calls to these libraries, which handle a lot of the low-level details. They may also make system calls to the kernel for convenience. As you might have guessed, this means these libraries are written for a particular kernel, and therefore can’t be used across operating systems with different kernels.
Operating System Execution Headers
The last roadblock to universal software is the format of executable files for the operating systems. An OS expects the files it runs to follow a particular binary file format. For example, the Executable and Linkable Format (ELF) files that run on operating systems such as Linux and FreeBSD must specify certain properties of the file in certain bytes, as shown in the below image.
The application binary interface (ABI) shown able is of particular importance. A combination of the calls available from processor, kernel, and system libararies, an ABI is similar to an application programming interface (API) in that it defines how two programs communicate with each other. But the API is something used by programmers (humans) in source code to indicate two pieces of software should talk to each other. The ABI is what actually allows them to do so once the software is compiled and run. Each operating system implements a particular ABI, which may or may not change between versions of that same OS.
In general, operating systems implement their own ABI, determined by a combination of the type of processor, the kernel, and any standard system libraries. But sometimes an OS will implement more than one. FreeBSD has support for Linux binaries, for example, because it provides a Linux ABI as an add-on to the FreeBSD kernel (instead of the Linux kernel). This is different from virtualizatiton programs such as VMWare or VirtualBox, which use software to simulate an entire machine (hardware and all). As a result this type of ABI-compatibility is faster, but much more effort to maintain. This is why it’s rare, although Microsoft recently saw the value in doing it.
Exception: Interpreted Software
Based on the above we’ve learned that developers write software for one, and only one, type of target system. Except when they don’t. There are many applications that you can download and run on a Mac, then copy and run on Windows, and maybe even copy again and run on Linux with no issues. How is this possible?
Was I lying up until now?
As it turns out, there’s a category of software that looks on the surface like it just “runs everywhere.” You can download and run it on any supported platform — the key word being “supported.” In fact, you’re downloading the source code for the application, while another application (the interpreter) is sort of running the source code directly in real time. This is something of an oversimplification, so let’s look at exactly how this works with a couple languages.
When Java first released, it’s promise was (literally) “write once, run anywhere.” The idea was to create applications by using Java functions for how to save files, make calculations, or create an application window. Then a Java Runtime Enviornment (JRE) for each supported computer platform would run the code, and translate these to native OS functions. The trick to Java, then, is that it doesn’t run “directly” on the operating system. It runs in a part of the JRE called the Java Virtual Machine and that’s what runs on the operating system.
By inserting this additional software layer between the application and the OS, Java allows you to focus on a set of functions that’s the same across operating systems. You tell Java what you want to do, and let the JVM for your system worry about how to actually do it. The below picture shows this in action, where JIDE Software’s Java Desktop Application Framework displays the same application for Mac (top), Windows (middle-left), “pure Java” (middle-right), and Linux (bottom).
Java programs don’t precisely “compile” themselves in realtime. Rather, the Java compiler will render them into “bytecode.” You can think of bytecode as a half-baked program. When the developer releases the application it’s compiled as much as it can be without knowing which OS it’s going to run on. When you actually launch it, the JVM will “bake it the rest of the way” to fit the functions particular of the host OS.
A popular interpreted language is Python. When you run a Python script, the Python interpreter will translate code into instructions for the OS. It can also function similarly to Java: when you “import” code from outside your application it’s compiled to bytecode the first time it’s run. Then the interpreter will know if, on subsequent runs, the original code has changed, at which point it will re-compile it to new bytecode.
A cool byproduct of this “on-demand” running is that you can use the interpreter to develop your scripts interactively. By simply typing “python” at the command line you’ll start up the interpreter, and you can run code and see the results immediately.
This means developers can play around and tweak things “live.” Then, once a line of code does what they want, copy and paste it into a script file (which is much more efficient than the “code-compile-test” cycle that non-interpreted language programmers have to do).
Even When Software Is the Same, It’s Probably Not
Unfortunately for users, the tech industry hasn’t developed a truly “universal” format. And it may never do so. Introducing these types of standards often result in a “least common denominator” solution, with concessions in the interest of getting everyone’s approval.
What do you think? Would you rather have universally-compatible software, even if it meant it wasn’t as good? Or are you OK with the operating system you’re using, and have no interest in the apps from other platforms? Let us know below in the comments!
Image Credits: Masterchief_Productions/Shutterstock