The Right Tool for the Job (Shared Objects)

For any task, using the right tool for the job is always a crucial matter. This applies to driving a nail into a wall as much as developing software. And while nobody would ever try to use a glass bottle for the nail, the tools of the trade of the software developer are a bit more abstract (and sometimes, more brittle too).

Shared != Shared

On AmigaOS the word “shared” is used in two major contexts: Shared Library, and Shared Object. Both are tools for sharing code between applications. However, they have very different methods for doing this, and with that comes a very different approach to using them.

Let’s first look at what they are.

Shared Libraries

Since the early days of AmigaOS, shared libraries have been a means of sharing code and, to a certain degree, data between multiple users. A shared library is, essentially, a structure in memory called the Library Base, and one or more jump tables to functions that are to be shared. Since Version 4.0 of the OS, these jump tables are called Interface, and although their use differs slightly from their setup in AmigaOS 3.9 and earlier, the principles are the same. A program intending to use a library has to do two steps in order to perform any calls into that library:

• It has to open the library by calling Exec’s OpenLibrary call.
• It has to obtain at least one interface from the library by calling GetInterface.

The latter step was not needed on the classic AmigaOS 3.x, but has opened up a host of new possibilities on AmigaOS 4.0 (we’ll talk more about that in a later article).

Interfaces, like the classic AmigaOS 3.x jump tables, are a collection of function pointers in a structure. Calling a function in an interface usually involves knowing the offset of that function. We typically call a function like this:

struct Library *library = IExec->OpenLibrary("foo.library", 0);

The variable IExec contains the interface. The OpenLibrary call is a member of the interface. During compile time, the compiler will calculate the offset of the member and generate appropriate code for that. The code will load the IExec interface pointer into a register, load the CTR register with the address at the specified offset, and branch into the routine using the bctrl mnemonic.

The process relies on a few factors. First of all, it requires to have the library open and have the interface ready. It also relies on the fact that an interface, once written, will at most be extended at the end. It will never be possible to remove functions from the interface (at least it will always have to have a `dummy' entry) nor will it be possible to re-order functions.

Data access is done via the library’s base pointer. The implementer might chose to store user-accessible data within the library base and document (at least part) of it for public access. Since this is a compile-time decision to make, again, the organization of data, just like the organization of functions, must not change once it has been published (unless the library base data is private).

Shared Objects

Shared Objects are a relatively recent addition to AmigaOS 4. They work radically different from the traditional shared libraries. A shared object is, as its name implies, somewhat reminiscent of an object file that is used during compilation of a program. In essence, a shared object allows a program to defer complete and final linking of the program until such time as the program is actually executed on the target user’s machine. This means that some symbols in the program remain unresolved until such time as the program is run.

As a matter of fact, even then the symbols might still remain undefined if a feature called Lazy Binding is active. In essence, Lazy Binding delays the linkage of a function until the very moment it is called for the first time. So, suppose you have a function MyGreatFunc that you call. If Lazy Binding is active, the call will jump into a routine called .resolv within elf.library which performs a symbol look-up on the name of the function and, if it finds it, overwrites the call to .resolv with a call to MyGreatFunc which is located somewhere in the shared objects that are bound to this program.

From the programmer’s point of view, though, there is no difference in whether the program was linked completely (statically) during the development cycle, or dynamically during load time or run time. In essence, the programmer can use a shared object like he uses any other object file in his program. This includes referencing data in the shared object file. Even more, the shared object file can reference data and code in any other shared object file bound to the program, and in the program itself.

In fact, this is a common case. If a program uses a shared object implementation of the standard C library, the start-up code in the shared object file will usually reference and call the program’s main function. This works completely transparent for both; there is no need to open the shared object, it just needs to be specified during linking.

Shared Objects can do even more. There is a set of functions to dynamically load shared objects during runtime of the program, and to look up symbols in the program or any of the loaded shared objects, including those that have been loaded at runtime.

Don’t use the Bottle!

Looking at the above, it seems natural to want to use shared objects instead of shared libraries. The shared objects seem to have so many advantages over the traditional AmigaOS shared libraries, so why would you want to use shared libraries in the first place?

The answer is simple: The flexibility and power of shared objects comes at a cost. Depending on what you do, these costs can be quite substantial:

1. Shared Objects, in their current implementation as of AmigaOS 4.1 Update 3, are not really shared. Each program using them will load the file again into memory, completely, including all code and data associated with the file. This means that e.g. a shared object version of libc will load all math functions, all system functions, in fact, everything, even if the program only ever uses printf.
2. Shared Objects are not versioned. While you can specify a minimum library version when you open a shared library to make sure that you get a guaranteed set of functionality, shared objects are provided “as-is” and do not have this kind of information.
3. Shared Objects require a certain amount of infrastructure up and running. Therefore they cannot be used for Kickstart modules, or device drivers.

Especially point 2 is an issue, in theory and in practice. Some libraries that are implemented as shared objects carry their own versioning information, or provide a call to find out what version a program is using, but this is not standardized in any way, and therefore a shared object might or might not implement such a mechanism. A lot of systems using shared object exclusively end up with something commonly referred to as “Dll Hell” – meaning the system has to provide a wide variety of different versions of the same library, mostly encoded in their file name. If you look at the average Linux distribution, you will see what this means. This here is an example from my local network server:

-rw-r--r-- 1 root root  9156 2009-04-10 18:29 libgmodule-2.0.a
lrwxrwxrwx 1 root root    26 2010-05-04 13:05 libgmodule-2.0.so -> libgmodule-2.0.so.0.2000.1
lrwxrwxrwx 1 root root    26 2009-06-02 12:48 libgmodule-2.0.so.0 -> libgmodule-2.0.so.0.2000.1
-rw-r--r-- 1 root root 13644 2009-04-10 18:29 libgmodule-2.0.so.0.2000.1

As you can see, there are four “versions” of the libgmodule library, two of which are soft links to provide somewhat generic names. If this were AmigaOS, you would simply have one gmodule.library file, and nothing more.

Another issue is different “builds” of the same shared object library. For example, suppose a library that can be used to load image files. Depending on the facilities of the original compiler of the library, he might have decided to compile the library to include support for PNG and JPEG but not GIF. They end up with a shared object file called libImageLoad.so which, for all intend and purpose, looks like libImageLoad.so from another programmer. The other programmer might however have decided to use GIF as an image format and has compiled his copy of libImageLoad.so with GIF support enabled. A user trying to run this program on a system that only has the first version installed will not work (at best, the program will not load because of the missing symbols). The decision on which libImageLoad.so to install/keep is left to the user – not a good idea.

Of course, this is still somewhat possible for AmigaOS shared libraries as well. A bad practice in the past has been to always install libraries in to the LIBS: drawer, even if the library was only ever used by a specific program. That lead to situations where two programs would install a library of identical name into LIBS:, and only one program would still work.

Glass Hammers and Steel Bottles

So, this leaves the question as to what kind of tool to use in what circumstance. In spite of their individual shortcomings, both shared libraries and shared objects provide a lot of power if used right.

Shared Objects are primary intended for porting of programs from other systems. Any sufficiently large or complex program from Linux or Windows will, in one way or the other, rely on shared code, for various reasons. Usually, this type of code sharing does not translate very well (if at all) to the AmigaOS shared library model. Using shared objects for such projects is almost mandatory.

Shared Objects have other possible uses on AmigaOS, even for new, native development. As an example, a concept like plugins or overlays can be easily and efficiently implemented using shared objects. Anything that is specific to an application and does not install on a global scope is probably easier to do with shared objects, more so since the compiler has intrinsic support for them (as opposed to AmigaOS shared libraries that need to be more or less hand-crafted).

For almost any other purpose, it is recommended to not use shared objects. If you are developing a library, chances are good that you should use a library for that purpose. AmigaOS shared libraries have limitations that are easy to work around in newly developed code. They offer a solid framework for versioning and an escape from other systems’ “Dll Hell”. They are independent on the compiler used, and as a rule, their limitation will force the programmer into a better designed interface. Since AmigaOS 4.0, the Interface concept allows for a strict versioning, meaning that changing the layout of an interface is even possible if they have a different version number, and GetInterface can retrieve a specific version of an interface if the library still implements it.

Closing Words

This post has tried to shed some light on a question that has confused some developers for a while. It tried to give a broad overview of how the different mechanisms work, and when to chose which method for implementing a task. AmigaOS 4.x and beyond tries to give you, the programmer, the tools required to build, but choosing the right tool is still your job.

Oh yeah, and we’re not responsible if you hit your thumb.

Author

Hans-Jörg Frieden