Copyright (c) Hyperion Entertainment and contributors.

How to Build Stubs for 68k Libraries

From AmigaOS Documentation Wiki
Jump to navigation Jump to search

Author

Original written by Roman Kargin

Introduction

The idea for this tutorial came from when I was playing with Dopus5 code and want to make a one single function from the 68k version of dopus5.library from native AmigaOS 4 code. While we have all kinds of auto generation tools to build all kinds of protos/interfaces/inlines/stubs/etc. it still wasn't clear to me how to do that what I need.

From the beginning a very helpful article was Thomas Rapp's How to create PPC stubs for M68K libraries where he shows how with using of fd2pragma and IDLTool to build a jpeg.l.main stub from a 68k jpeg.library, as well as helpful answers from Fredrik Wikstrom. In the end, all of it seemed important for me to note so I just put everything into this tutorial so that I hope others will find useful.

68k->PowerPC Glue Stubs

While 68k libraries are old and on AmigaOS today almost all of the libraries are in native PowerPC, there are still some old 68k libraries which are necessary and have no native PowerPC replacement. 68k->PowerPC glue stubs are necessary to be able to call 68k routines from PowerPC code in AmigaOS.

If you have created native code yourself which will be calling some old 68k library (i.e native aos4-ppc binary with available source code which will use 68k library), you not only can use 68k-PowerPC glue stubs but you can also use EmulateTags() and call your library functions directly. This makes it easy for your code to call both PowerPC and 68k native libraries with only a small bit of small overhead; just a few instructions. This greatly reduces the amount of work and will support both PowerPC and 68k modules transparently with no stub-files needed. But this only works when the PowerPC native code is available and there are just a few functions in the library and so on. Quite often, you just do not want to deal with all of this because of the large number of functions in a library and the different ways of using them from the code. In this case, using 68k-PowerPC stubs are a simpler way.

For a better explanation of what is 68k-PowerPC stub is, refer to the Migration Guide. In brief, if you have let's say 68k version of dopus5.library and want to use it from native AmigaOS 4 binaries then you need to create dopus5.l.main. When code in your AmigaOS binary needs to interface to a 68k library, the system (ramlib to be more precise) will automatically scan its library search path for the file "dopus5.l.main" and if found, will try to open it and obtain the interface. That l.main file is pure non-startup code which contains the necessary stubs. You do not need to write the stubs from scratch manually. That is the purpose of different kinds of tools (fd2pragma, fdtrans and IDLTool). The library.l.main code consists of 2 files if you auto generate them: PowerPC->68k crosscall stubs and vectors. You of course can write them from scratch manually, but its a pain and there are already tools which do it for you.

The name of the file in general, does not matter. A library does not necessarily have to be named ".library" and could be named ".module" or whatever. Ramlib will search for the first "." in the library name, starting from the end, the first part up to and including the dot plus one more character, usually the 'l' in library, is used, another dot is appended, then the interface name is added. It then calls the exec function.

For example, if the interface name is 'main' and the library name is 'foobar' then this is what it looks for: 

foobar.library  -> foobar.l.main
foobar.dohickey -> foobar.d.main
foobar.module   -> foobar.m.main

Things to note:

1. Because of the way RamLib parses glue-stubs names, you can't call your glue stubs say "foobar.library" as "foobar.library.main". It is understandable from previous description, but can drive to some misleading when you run for example "snoopy" and can see how it tries to search for "foobar.library.main", but it tries to search to for a file, but for "resident". File for search by ramlib are only foobar.l.main if main library called as foobar.library.

2. You can't put your glue stub in directories which are not PROGDIR:, PROGDIR:Libs and LIBS: or RamLib will not find them. If you have let's say directory "modules" which have some of your plugins which in reality are libraries, just in different directory and called different, so, then you can't put to the same "modules" directory your glue stubs, because system will still try to find out those glue files in PROGDIR: PROGDIR:Libs or LIBS: paths.

pragmas/fd/sfd/protos

Old 68k libraries its not something which done on GCC most of time: on old 68k machines SASC was the popular one for C, and pure assembler was even more popular for every needs in compare with todays times. And so, many of those old 68k libraryes, provide for developers only includes for sasc, or only asm, or mix or so. So, when you have by hands such a library, where let's say only pragma file is present, and protos let's say done in diffeent includes like a mess: you do need to understand all of this and create your own proto files, build that stub lib.l.main, make a os4 includes from and so on.

All what we need to know now, is that libraries have a jump table. Jump table have a list of library functions by offsets. Offsets can't be messed, because by offsets in jump table your code can call necessary function from library. Be it old sasc/assembler, or new gcc on os4: if we works with 68k library and create any fd/proto files from scratch, then offsets should be always be the same.

For example you have dopus_pragmas.h file and dopus.fd file (fd files is standard way to describe the interface for libraries) and dopus_pragmas entryes looks like this: 

#pragma libcall DOpusBase RemovedFunc0 1e 0
#pragma libcall DOpusBase Random 24 001
#pragma libcall DOpusBase Atoh 2a 0802
....

It mean that first "user" library function in jump table is started on offset 0x1e (30), and then multiply by 6 for every function. So first function on 30(0x1e), second on 36 (0x24), next on 42 (0x2a) and so on for all the functions. That mean, that if for example function Atoh() in the jump table placed by offset 0x2a, then it mean that every single code and includes which we will generate for os4 should respect jump-table offsets.

If we will look at dopus_lib.fd file, we will see: 

* "dopus.library"                                             
##base _DOpusBase
##bias 30
*                                                             
* Support routines for Directory Opus and associated programs 
* (c) Copyright 1995 Jonathan Potter                          
*                                                             
##public
RemovedFunc0()()
Random(limit)(d0)
*
Atoh(str,len)(a0,d0)
....

What mean that public functions starts from offset 30 (bias 30), and they placed in the same necessary order as they placed in the pragma file. I.e. if you for example will just put Atoh() from FD file at first place, and RemovedFunc0() at place of Atoh(), generate all your files for os4, and build native binary to use library: then instead of Atoh() you will have RemovedFunc0() and instead of RemovedFunc0() you will have Atoh(), what of course mean crashes, bugs and problems.

So, respect offsets in jump table is must.

Need to add, that also .sfd kind of files present (you will not find them usually in old 68k dev archives of libraryes as its something fresh in compare with .fd), and they hold a lot more infomation in compare with .fd and easy to operate. To see full description of .fd and .sfd formats check fd2pragma guide. But in general they more or less the same by logic, the same respect for offsets, etc.

Now What?

We have 2 ways to build lib.l.main and necessary includes. But for both of them you need .fd file as minimum (if you have .sfd its even better), and proto file which will describe all (or not all) your functions. While .fd/.sfd its something which should be done by developers initally, you still can make them from scratch just if you have pragma file with lib calls (you make that files in the same order by the same offsets). If you didn't have one proto file, but many in different includes, then you just can make your own one, just string by string grab from all the include files all the protos, while all your FD entrys will not have necessary proto descriptions in proto file (btw, if you will not have proto for some function, then entry from fd/sfd file will be reserved when you will auto generate os4 code/includes, so respect offsets).

Let's say we have dopus_lib.fd and nothing more nowhere. Only protos in different includes in diffrent places. We then string by string create proto file from all those includes (in refering to the dopus_lib.fd), and then auto-tools time coming. As i say we have 2 ways:

First way: fdtrans + sfd

1. generate .sfd file: 

ram:> fd2pragma dopus5_lib.fd clib our_proto.h special 112
SourceFile: dopus5_lib.fd
ResultFile: dopus5_lib.sfd

2. generate xml and ppc->68k crosscall stubs via fdtrans: 

ram:> fdtrans dopus5_lib.sfd -x -s

Change manually in the crosscall-stubs file (dopus5.c) "main_vectors" on "main_v1_vectors", as it seems some typo beetwen different tools. No other changes need it, fdtrans automatically insert including of interface (while fd2pragma will skip it, see later).

3. generate proto/interface/inline (i.e. includes) and vectors: 

ram:> idltool -p -i -n -c dopus5.xml

4. build dopus5.l.main 

ram:> gcc -Iinclude -nostartfiles dopus5.c -o dopus5.l.main

5. build test case which will use one of functions from dopus5.library called StrConcat() 

#include <proto/exec.h>
#include <proto/dopus5.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
 
struct Library *DOpusBase;
struct DOpusIFace *IDOpus;
 
void main(int argc,char **argv)
{
	char buffer[256] = "First part-";
	char *concat = "Second part";
 
	// Need dopus library
	if (!(DOpusBase=IExec->OpenLibrary("PROGDIR:dopus5.library",1L))) {
		printf("cant library");
		exit(10);
	};
	if (!(IDOpus=IExec->GetInterface((struct Library *)DOpusBase,"main",1L,NULL))) {
		printf("cant interface");
	   exit(10);
   };
 
	IDOpus->StrConcat(buffer, concat, 36);
	printf("Result=%s\n", buffer);
 
	IExec->CloseLibrary(DOpusBase);
}
ram:> gcc -Iinclude testcase-native-fdtrans.c 
ram:> a.out
ram:> Result=First part-Second part

It works !

Second way: fd2pragma + fd

1. Create xml: 

ram:> fd2pragma dopus5_lib.fd clib proto.h special 140
SourceFile: dopus5_lib.fd
Resultfile: dopus5.xml

2. generate ppc->68k crosscall stubs: 

ram:> fd2pragma dopus5_lib.fd clib proto.h special 141
SourceFile: dopus5_lib.fd
Resultfile: dopus5.c

In result file (dopus5.c) manually add including of interfaces/dopus5.h and replace value "main_vectors" on "main_v1_vectors"

3. generate proto/interface/inline (i.e. includes) and vectors: 

ram:> idltool -p -i -n -c dopus5.xml

4. build dopus5.l.main 

ram:> gcc -Iinclude -nostartfiles dopus5.c -o dopus5.l.main

5. build test case (its a bit different in compare with one from fdtrans, because names of iface a bit different (big/small letters, 5 at end, etc) : 

#include <proto/exec.h>
#include <proto/dopus5.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
 
struct Library *DOpusBase;
struct Dopus5IFace *IDopus5;
 
void main(int argc,char **argv)
{
	char buffer[256] = "First part-";
	char *concat = "Second part";
 
	// Need dopus library
	if (!(DOpusBase=IExec->OpenLibrary("PROGDIR:dopus5.library",1L))) {
		printf("cant library");
		exit(10);
	};
	if (!(IDopus5=IExec->GetInterface((struct Library *)DOpusBase,"main",1L,NULL))) {
		printf("cant interface");
	   exit(10);
   };
 
	IDopus5->StrConcat(buffer, concat, 36);
	printf("Result=%s\n", buffer);
 
	IExec->CloseLibrary(DOpusBase);
}

Then hard-reboot is need it in case you already run previous test case on dopus5.l.main done from fd2trans and which still in memory (as it can freeze the OS hard because of different code of the same stub libs), and then: 

ram:> gcc -Iinclude testcase-native-fd2pragma.c 
ram:> a.out
ram:> Result=First part-Second part

It works as well !

Conclusion

What way better ? Imho fdtrans one. Because:

  1. you need to change less (no need to manually include interfaces/dopus5.h file).
  2. the code of stubs from fdtrans looks better and cleaner in compare with code of stubs from fd2pragma. For example such a even small moment as comments saying what value in hex mean your offset in jump table is pretty nice (expectually when you works manually with pragmas, where those hex offsets is present).
  3. size of dopus.l.main done from fd2pragma output bigger in compare with size of dopus.l.main done from fdtrans output.

I want one single function to work

Now something intersting : you for example want to just check if it all will works at all for you from os4 and with your new lib.l.main and native os4 includes, code and co. You then choice any "simply" function from library and want to make a simple os4 native test case for it. Let's say it will be StrConcat() function from dopus5.library, which is described in pragma file as: 

#pragma libcall DOpusBase StrConcat 47a 09803

and in dopus_lib.fd like: 

StrConcat(s1,s2,len)(a0/a1,d0)

We then simply calc:

1. 0x47a = 1146 2. 1146 / 6 = 191 3. 191 - 4 (i.e. 30/6 from begining 5, but 30 is our first one, so -4) = 187

I.e. entry in jump table for StrConcat is 187. And entry in the dopus_lib.fd file for Strconcat also 187 in the list of functions (+ header of .fd)

In other words you can build your .fd file from scratch, which will looks like this: 

* "dopus.library"                                             
##base _DOpusBase
##bias 30
*                                                             
* Support routines for Directory Opus and associated programs 
* (c) Copyright 1995 Jonathan Potter                          
*                                                             
##public
aa()()
aa()()
aa()()
aa()()
.... 186 entrys to fill the gap in jump table ....
StrConcat(s1,s2,len)(a0/a1,d0)
....
##end

You of coures can go more elegant way and use .sfd instead, which have "reserved" keyword, so no need to put all those empty-funct. I.e.: 

==id $Id: dopus5_lib.sfd,v 1.0 2013/03/29 09:24:08 noname Exp $ 
* "dopus5.library" 
==base _DopusBase 
==basetype struct Library * 
==libname dopus5.library 
==bias 30 
==public 
==include <exec/types.h> 
==reserve 186 
BOOL StrConcat(char * s1, char * s2, int len) (a0,a1,d0) 
==end

And when you will generate all necessary stuff via fd2grama/fdtrans/idltool from .fr or from .sfd, offsets will be respected, and when you will call from your code IDopus->StrConcat(blablab);, it will works because code will find out StrConcat in jump-table by right offset and will works.

Final words

As usual, there is nothing hardcore in here. This is just a short article about the problems which I encountered. I hope that it can be any help for new developers. In end thanks to Fredrik Wikstrom for hints, and many others for proofreading and grammar corrections.

Links

[1] OS4 libraries and devices: http://wiki.amigaos.net/index.php/Libraries_and_Devices

[2] RamLib autodoc.

[3] fd2pragma guide from: http://aminet.net/dev/misc/fd2pragma.lha

[4] How to create PPC stubs for M68K libraries: http://thomas-rapp.homepage.t-online.de/ppclib.html

Retrieved from "http://wiki.amigaos.net/w/index.php?title=How_to_Build_Stubs_for_68k_Libraries&oldid=5520"