Exec Libraries

This page is not yet fully updated to AmigaOS 4.x some of the information contained here may not be applicable in part or totally.

Exec Libraries

Exec maintains lists of libraries and devices. An Amiga library consists of a collection of related functions which can be anywhere in system memory (RAM or ROM). An Amiga device is very similar to an Amiga library, except that a device normally controls some sort of I/O hardware, and generally contains a limited set of standard functions which receive commands for controlling I/O. For more information on how to use devices for I/O, see the "Exec Device I/O".

Unlike AmigaOS 3.9 and previous, AmigaOS 4.x libraries can export more than one function table and usually do. Function tables under AmigaOS 4.x are called "interfaces". An interface is a simple structure with a few predefined fields at the front and function pointers thereafter, see What is an Interface? below for more details.

Not for Beginners
This article concentrates on the internal workings of Exec libraries (and devices). Most application programmers will not to know the internals workings of libraries to program the Amiga. For an introduction to libraries and how to use them, see Programming in the Amiga Environment

What is a Library?

A library consists of a group of functions somewhere in memory (ROM or RAM), a vector table, and a Library structure which can be followed by an optional private data area for the library. The library's base pointer (as returned by OpenLibrary()) points to the library's Library data structure:

struct Library
{
    struct  Node lib_Node;
    UBYTE   lib_Flags;
    UBYTE   lib_pad;
    UWORD   lib_NegSize;            /* number of bytes before library */
    UWORD   lib_PosSize;            /* number of bytes after library */
    UWORD   lib_Version;
    UWORD   lib_Revision;
    APTR    lib_IdString;
    ULONG   lib_Sum;                /* the checksum itself */
    UWORD   lib_OpenCnt;            /* number of current opens */
};

/* Meaning of the flag bits: */
                               /* A task is currently running a checksum on */
#define LIBF_SUMMING (1 << 0)  /* this library (system maintains this flag) */
#define LIBF_CHANGED (1 << 1)  /* One or more entries have been changed in the library
                                  code vectors used by SumLibrary (system maintains
                                  this flag)  */

#define LIBF_SUMUSED (1 << 2)  /* A checksum fault should cause a system panic
                                  (library flag) */
#define LIBF_DELEXP (1 << 3)   /* A user has requested expunge but another user still
                                  has the library open (this is maintained by library) */

Using a Library to Reference Data

Most libraries (such as Intuition, graphics and Exec) have other data that follows the Library data structure in memory. Although it is not normally necessary, a program can use the library base pointer to access the Library structure and any custom library data.

In general, the system's library base data is read-only, and should be directly accessed as little as possible, primarily because the format of the data may change in future revisions of the library. If the library provides functions to allow access to library data, use those instead.

Relationship of Libraries to Devices

A device is a software specification for hardware control based on the Library structure. The structures of libraries and devices are so similar that the routine CreateLibrary() is used to construct both.

Devices require the same initial four code vectors as a library, but must have two additional code vectors for beginning and terminating special device I/O commands. The I/O commands that devices are expected to perform, at minimum, are shown in Exec Device I/O. An example device is listed in Example Device.

Minimum Subset of Library Vectors

The first four code vectors of a library must be the following entries:

OPEN

is the entry point called by the function OpenLibrary(). In most libraries, OPEN increments the library variable lib_OpenCnt. This variable is also used by CLOSE and EXPUNGE.

CLOSE

is the entry point called by the function CloseLibrary(). It decrements the library variable lib_OpenCnt and may do a delayed EXPUNGE.

EXPUNGE

prepares the library for removal from the system. This often includes deallocating memory resources that were reserved during initialization. EXPUNGE not only frees the memory allocated for data structures, but also the areas reserved for the library node itself.

RESERVED

is a fourth function vector reserved for future use. It must always return zero.

Changing the Contents of a Library

The way in which an Amiga library is organized allows a programmer to change where the system looks for a library routine. Exec provides a function to do this: SetFunction(). The SetFunction() routine redirects a library function call to an application-supplied function. (Although it's not addressed here, SetFunction() can also be used on Exec devices.) For instance, the AmigaDOS command SetPatch uses SetFunction() to replace some OS routines with improved ones, primarily to fix bugs in ROM libraries.

The format of the SetFunction() routine is as follows:

SetFunction( struct Library *lib, LONG funcOffset, APTR funcEntry)

The lib argument is a pointer to the library containing the function entry to be changed. The funcOffset is the Library Vector Offset (negative) of the function and funcEntry is the address of the new function you want to replace it with. The SetFunction() routine replaces the entry in the library's vector table at the given Library Vector Offset with a new address that points to the new routine and returns the old vector address. The old address can be used in the new routine to call the original library function.

Normally, programs should not attempt to "improve" library functions. Because most programmers do not know exactly what system library functions do internally, OS patches can do more harm than good. However, a legitimate use for SetFunction() is in a debugger utility. Using SetFunction(), a debugger could reroute system calls to a debugging routine. The debugging routine can inspect the arguments to a library function call before calling the original library function (if everything is OK). Such a debugger doesn't do any OS patching, it merely inspects.

SetFunction() is also useful for testing an application under conditions it does not encounter normally. For example, a debugging program can force a program's memory allocations to fail or prevent a program's window from opening in order to test the program's error handling code.

SetFunction() is for Advanced Users Only
It is very difficult to cleanly exit after performing SetFunction() because other tasks may be executing your code and also because additional SetFunction()'s may have occurred on the same function. Also note that certain libraries (for example the V33 version of DOS library) and some individual library function vectors are of non-standard format and cannot be replaced via SetFunction().

Although useful, performing SetFunction() on a library routines poses several problems. If a second task performs SetFunction() on the same library entry, SetFunction() returns the address of the new routine to the second task, not the original system vector. In that case, the first task can no longer exit cleanly since that would leave the second task with an invalid pointer to a function which it could be relying on.

You also need to know when it is safe to unload your replacement function. Removing it while another task is executing it will quickly lead to a crashed system. Also, the replacement function will have to be re-entrant, like all Exec library functions.

Don't Do This!
For those of you who might be thinking about writing down the ROM addresses returned by SetFunction() and using them in some other programs: Forget It. The address returned by SetFunction() is only good on the current system at the current time.

Adding a Library

Exec provides several ways to add your own libraries to the system library list. One rarely used way is to call LoadSeg() (a DOS library function) to load your library and then use the Exec CreateLibrary() and AddLibrary() functions to initialize your library and add it to the system.

CreateLibrary() allocates space for the code vectors and data area, initializes the library node, and initializes the data area according to your specifications, returning to you a library base pointer. The base pointer may then be passed to AddLibrary() to add your library to the system.

Another way to initialize and add a library or device to the system is through the use of a Resident structure or romtag (see <exec/resident.h>). A romtag allows you to place your library or device in a directory (default "LIBS:" for libraries, "DEVS:" for devices) and have the OS automatically load and initialize it when an application tries to open it with OpenLibrary() or OpenDevice().

Two additional initialization methods exist for a library or device which is bound to a particular Amiga expansion board. The library or device (containing a romtag) may be placed in the "SYS:Expansion" drawer, along with an icon containing the Manufacturer and Product number of the board it requires. If the "startup-sequence" BindDrivers command finds that board in the system, it will load and initialize the matching "Expansion" drawer device or library. In addition, since 1.3, the Amiga system software supports ROM drivers on expansion boards. See Expansion Library for additional information on drivers and Expansion drawer drivers. The sample device code in Example Device may be conditionally compiled as an "Expansion" drawer driver.

Library Initialization

Need to show RomTag structure here.

Initialization via init tables is no longer supported. To initialize a library you need to provide a ROMTAG structure like with previous versions of the OS although the interpretation of some of the fields are a bit different. Most notably, the rt_Flags field must have the RTF_NATIVE flag set to indicate that the code in this library is native PowerPC and not 68k. If the RTF_AUTOINIT flag is not set, the rt_Init field is assumed to point to a function that is called up on initialization (depending on the RTF_NATIVE flag, it might be called under emulator control).

If the RTF_AUTOINIT flag is set, then the rt_Init must point to a taglist that will be passed to a function - the function again depends on RTF_NATIVE ROMTAGs without the RTF_NATIVE flag will be handled identically to the old way - their rt_Init field points to a init table and is passed to MakeLibrary.

Native libraries expect to find a taglist at rt_Init that is passed to CreateLibrary(). This creates a new-style PowerPC-native library with an optional 68k interface part.

A typical taglist for this looks somewhat like this:

struct TagItem fubar_libCreateTags[] =
{
 {CLT_DataSize, sizeof(struct FubarBase)},
 {CLT_InitFunc, (uint32)fubar_libInit},
 {CLT_Interfaces, (uint32)fubar_libInterfaces},
 {TAG_END, 0}
};

In this case, the first tag specifies the size of the base structure, the second tag specifies the libinit function and the last tag specifies the interfaces array. The interfaces array is a null-terminated array of pointers to tag lists for the MakeInterface() call. Typically you will have at least two interfaces here, the library manager interface and an exported interface (which will most likely be "main"). The layout of the library manager interface has already been discussed, we are going to cover the methods in a few moments.

The interfaces array might look something like this:

uint32 fubar_libInterfaces[] =
{
 (uint32)fubar_managerTags,
 (uint32)fubar_mainTags,
 0
};

The first would be the taglist for creating the manager interface, the second for the main interface. We will only cover the manager interface here, since the main interface looks exactly the same, with different functions only. Note that both start with the four default vectors.

For our library manager interface, the tag list looks like this:

struct TagItem foobar_managerTags[] =
{
 {MIT_Name, (uint32)"__library"},
 {MIT_VectorTable, (uint32)foobar_manager_vectors},
 {MIT_Version, 1},
 {TAG_END, 0}
};

Here, the name is fixed (in any other interface the name could be chosen freely) and the version number is also fixed to "1". you may use different version numbers in "normal" interfaces, but an interface of this type must be present.

The foobar_manager_vectors variable is the vector table. This is simply a null-terminated array of methods, in the order in which they appear in the interface. For the library manager interface, we need to have the methods as outlined earlier in this document. The table should look somewhat like this:

void *fubar_manager_vectors[] =
{
 (void *)generic_Obtain,
 (void *)generic_Release,
 (void *)0,
 (void *)0,
 // In general, start the "normal" methods here
 (void *)fubar_libOpen,
 (void *)fubar_libClose,
 (void *)fubar_libExpunge,
 (void *)0,
 (void *)-1,
};

As per usual the interface starts with the default methods. We use the generic obtain and release methods, and leave both expunge and clone unimplemented (if you leave expunge unimplemented, the interface will simply be freed).

The following three vectors are the same as with classic libraries, only that they are assumed to be native code as opposed to 68k code.

The last (in this case unimplemented) vector is the GetInterface call. With all probability you will leave this empty for just about everything. Exec already offers a GetInterface() function that should be sufficient for just about every purpose. Only very special cases require this vector, so we will skip its discussion for the purpose of this document.

The idltool provides a good sample implementation of the three above-mentioned library manager methods and should be referenced for more details.

We now need to look at the libInit function. This was specified using the CLT_InitFunc tag in the ROMTAG-anchored taglist for CreateLibrary(). The function is responsible for setting up one-shot resource stuff and things that are not handled in the open code. Generally, you will almost always be able to use the generic implementation provided by the idltool.

What is an Interface?

In order to use an interface, the appropriate library needs to be opened first. This is done with the usual call to OpenLibrary(), which returns a library base pointer like under AmigaOS 3.x. It is not possible, however, to directly call function from that library. The exception being for "classic" libraries.

To call library functions, you need to retrieve an interface pointer first. This is done using the GetInterface() function (see the autodoc). The method returns an already Obtain()'ed interface that must be disposed with DropInterface() after usage. Interfaces are retrieved by name. Most libraries export at least the "main" interface which is the representation of their basic functionality, especially for classic libraries.

If an interface pointer is passed along to another entity, for example another task, process or thread, then that entity must call Obtain() before using the interface. Likewise, when the entity is done, it needs to call Release().

In summary, new interfaces must always be created using GetInterface/DropInterface. Existing interfaces must be referenced/dereferenced with Obtain/Release.

Anatomy of an Interface

The first four function pointers are fixed and need to be present in every interface. After that, any number of function pointers may follow.

Each interface has a version number. Any change in the interface will keep the version number if and only if function pointers are added to the interface. That is, if any prototype or location of an interface function changes, the version number needs to be incremented. Programs using an interface can request a specific version number; if that version is available, it will be returned, otherwise an error is generated.

The include file <exec/interfaces.h> introduces the overall structure of an interface:

struct InterfaceData
{
 struct Node Link; /* Node for linking several interfaces */
 struct Library *LibBase; /* Library this interface belongs to */
 ULONG RefCount; /* Reference count */
 ULONG Version; /* Version number of the interface */
 ULONG Flags; /* Various flags (see below) */
 ULONG CheckSum; /* Checksum of the interface */
 ULONG PositiveSize; /* Size of the function pointer part */
 ULONG NegativeSize; /* Size of the data area */
 // ...
};
 
struct Interface
{
 struct InterfaceData Data; /* Interface data area */
 /* Increment reference count */
 ULONG (*Obtain)(struct Interface *Self) APICALL;
 /* Decrement reference count */
 ULONG (*Release)(struct Interface *Self) APICALL;
 /* Destroy interface. May be NULL */
 void (*Expunge)(struct Interface *Self) APICALL;
 /* Clone interface. May be NULL */
 struct Interface * (*Clone)(struct Interface *Self) APICALL;
};

Let's first have a look at the embedded InterfaceData structure. RefCount is a counter that works like the OpenCnt in the library base. Only Interfaces with a RefCount of zero can be deleted. The Version number is stored in the Version field. Flags contains a number of bitdefs that specify certain behaviour principles of the interface. A number of flags have been currently defined:

• IFLF_PROTECTED indicates that this is a protected interface, that is, an interface that must not be modified by exec/SetMethod() (exec/SetMethod is to interfaces what SetFunction() is to AmigaOS 3.9 libraries). An attempt to patch a protected interface fails. This protection may or may not be enforced by hardware. Potential usage for this is an SSL interface, Password/Login interface, or any interface that is security-sensitive.

• IFLF_NOT_NATIVE indicates that this is an interface that contains emulator stubs for calling a classic 68k jump table. This flag doesn't serve any particular purpose other than to mark this interface as potentially slow.

• IFLF_PRIVATE indicates that this is a private interface. Private interfaces are non-sharable, and usually carry contextual information that are local to a process or task.

• IFLF_CHANGED and IFLF_UNMODIFIED are used by the system. The Set-Method method sets the IFLF_CHANGED flag, and the SumInterface method recalculates the checksum and sets IFLF_UNMODIFIED again.

• IFLF_CLONED marks a cloned interface. See below in the description of the Clone method.

The CheckSum field contains a checksum over the interface, and is used for sanity checking and for security, much the same way as the library checksum.

The PositiveSize and NegativeSize fields specify, as the name implies the positive and negative size of an interface. The positive size is the size from the interface start to the end, stretching across the fields at the start of the interface and all function pointers. The negative size is usually zero; if it is non-zero, it specifies the size of the interface's private data area, which is located NegativeSize bytes before the Interface data. Contrary to the AmigaOS 3.9 library, data is always stored in front of the interface. Most interfaces don't contain their private data, and therefore the NegativeSize is zero.

The interface itself start with the embedded InterfaceData structure, followed by the four standard interface functions (we will call interface functions "methods" from now on). The four standard methods are:

• ULONG Obtain() This method adds a reference to the interface. It increments the RefCount field by one and returns the current RefCount. In order to use an interface, the Obtain method must be called first. The exec library method "GetInterface" does this implicitly.

• ULONG Release() This method removes a reference from the interface. After releasing the interface it must not be used again.

• void Expunge() This method deletes the interface and frees all associated resources. This function is usually not called directly.

• struct Interface *Clone() This method clones an interface. Usually it is sufficient to use Obtain() to be able to use the interface. However, some interfaces might contain data that need to be cloned. For example, consider an interface that can load an image from disk, and store the image data within the interface. That interface would have methods for reading/writing pixels within the picture. Now your paint package want to apply a filter and offer a preview function. For this, the picture would need to be modified, but the original needs to stay around in case the user pressed cancel. For this reason, the interface would be cloned (and hence the picture with it), and the preview could be done on the clone.

Using Interfaces

We assume that the global variable "IExec" contains the main interface pointer of exec.library. Therefore, to open "test.library", we would write

struct Library *testLib = IExec->OpenLibrary("test.library", 0);

The above will yield a struct Library pointer in testLib that we can use to retrieve interfaces from test.library. Since this library isn't a classic library (it has no parallel version under AmigaOS 3.9) it doesn't automatically export any methods. To get the "main" interface of this library we would write:

struct TestMainIF *IMain = (struct TestMainIF *)
 IExec->GetInterface(testLib, "main", 1, NULL);

The struct TestMainIF is a structure that contains an embedded struct Interface along with additional function pointers - the methods that the interface offers. After this code sequence we can use the IMain interface. Note that by convention, all interface pointers are written as a capital 'I' followed by a capitalized name. IExec and IIntuition would be examples. A test for a NULL pointer should always be performed before using any interface.

While this at first glance looks a bit intimidating, keep in mind that you usually only do this once and for libraries like Intuition. Note the IExec, IDOS and IUtility (when using newlib) interfaces are pre-initialized by the C startup code.

Calling a function from IMain would look something like

int32 test = IMain->DoSomething(10, 100);
IDOS->Printf("IMain->DoSomething(10, 100) = %ld\n", test);

To dispose of the interface, one would call

IExec->DropInterface((struct Interface *)IMain);

Finally, to close the library call

IExec->CloseLibrary((struct Library*)testLib);

Classic libraries

"Classic" libraries pose a special case since any source code from the classic OS expects to open e.g. intuition.library and directly call OpenWindow() on it. These libraries have special inline functions that call the appropriate method in the "main" interface.

Library Internals

Internally, libraries and devices look very similar. They both can export any number of interfaces, making it possible for e.g. timer.device to export its time arithmetic functions in its main interface (or under any other name). The only difference between libraries and devices is that a library must export at least one interface called "__library" of type LibraryManagerInterface, and a device must export an interface called "__device" of type DeviceManagerInterface.

Both interfaces are defined in <exec/interfaces.h>.

struct LibraryManagerInterface
{
 struct InterfaceData Data;
 
 uint32 APICALL (*Obtain) (struct LibraryManagerInterface *Self);
 uint32 APICALL (*Release)(struct LibraryManagerInterface *Self);
 VOID APICALL (*Expunge)(struct LibraryManagerInterface *Self);
 struct Interface * APICALL (*Clone)(struct LibraryManagerInterface *Self);
 struct Library * APICALL (*Open)(struct LibraryManagerInterface *Self, uint32 version);
 APTR APICALL (*Close)(struct LibraryManagerInterface *Self);
 APTR APICALL (*LibExpunge)(struct LibraryManagerInterface *Self);
 struct Interface * APICALL (*GetInterface)(struct LibraryManagerInterface *Self, STRPTR name, uint32 version, struct TagItem *taglist);
};
 
struct DeviceManagerInterface
{
 struct InterfaceData Data;
 uint32 APICALL (*Obtain) (struct DeviceManagerInterface *Self);
 uint32 APICALL (*Release)(struct DeviceManagerInterface *Self);
 VOID APICALL (*Expunge)(struct DeviceManagerInterface *Self);
 struct Interface * APICALL (*Clone)(struct DeviceManagerInterface *Self);
 VOID APICALL (*Open)(struct DeviceManagerInterface *Self, struct IORequest *ior, uint32 unit, uint32 flags);
 APTR APICALL (*Close)(struct DeviceManagerInterface *Self, struct IORequest *ior);
 APTR APICALL (*LibExpunge)(struct DeviceManagerInterface *Self);
 struct Interface * APICALL (*GetInterface)(struct DeviceManagerInterface *Self, STRPTR name, uint32 version, struct TagItem *taglist);
 VOID APICALL (*BeginIO)(struct DeviceManagerInterface *Self, struct IORequest *ior);
 VOID APICALL (*AbortIO)(struct DeviceManagerInterface *Self, struct IORequest *ior);
};

As you can see, a library has four and a device six additional function pointers. The four library function pointers are also represented in the device manager interface. The Open/Close/Expunge vectors serve the exact same purpose as under AmigaOS 3.9, and their implementation can more or less follow their AmigaOS 3.9 counterpart. The fourth vector, GetInterface, is usually empty, because exec.library already implements that functionality. However, some libraries might want to override the functionality and implement their own GetInterface function.

A device manager interface has two additional vectors, BeginIO and AbortIO whose functionality mirrors that of the AmigaOS 3.9 device.

Using inline stubs

Note that we could write

struct Library *testLib = OpenLibrary("test.library", 0);

because there is a stub function OpenLibrary() for backwards compatibility with classic programs. The OpenLibrary stub function assumes that the global variable IExec contains the "main" interface pointer of exec. The first version is the preferred version.