Advanced Serial Debugging Guide

Author

Roman Kargin and Lyle Hazelwood
Copyright © 2013 Roman Kargin and Lyle Hazelwood
Proofreading and grammar corrections by the AmigaOS Wiki team.

Introduction

While AmigaOS has "hi-level" capabilities to help debug problems (e.g. redirecting debug outputs to memory and displaying the log with DumpDebugBuffer or using serial redirection tools like Sashimi) it is sometimes still not enough. For example, there is no protection for areas like input and Intuition so we can get into a situation where the GUI is not operational but the OS is still working. Intuition, input and any other necessary components can crash and then the GrimReaper utility has no way to spawn itself. Meanwhile, the kernel, still can output debug information about the crash on serial.

To begin with, you should know that AmigaOS has 2 reapers. One is a user friendly utility from SYS:System called GrimReaper. The GrimReaper can be called a "hi-level" one. Another reaper is called Reaper which is a part of the kernel itself. The kernel's Reaper actually launches the user-friendly GrimReaper when a required. So, you can imagine that when things get nasty, the kernel's reaper not all the time can spawn a GrimReaper, but still can catch some information and output it to the serial. Visually, when such a nasty things happens and input and/or intuition die, you just see a "freeze" of the OS and may think that everything is freezes, while it not: you just can't operate with OS anymore as necessary parts of OS crashes, but OS itself still working and kernel's reaper already throw all the info into the serial.

That means that when you want to do some serious work and you need any way to catch your bugs and/or debug any of your problematic code, then serial debugging is the best and only one way. Of course it's still possible to hope that the system friendly GrimReaper will popup for you and that there will be nothing so hardcore which it will crash Intuition and "freeze" the OS but that's just not enough when it come to doing real work.

You may ask now why the serial port is used at all? Why not parallel or something else? Why does everyone keep telling you to use serial debugging as if there is nothing else you can connect a cable to? The answer is in simplicity and reliability. Serial was chosen because it is simple to transfer data, bit by bit, not like for example done with a parallel port. There are different interfaces which also can transfer information bit by bit (like Ethernet, Firewire and USB) but the standard serial port is common to all platforms and far less complex. Because of that it should come as no surprise that the old AmigaOS 3 which works on classic Amigas throw all the debug information via serial port as well. Today, we have different kinds of hardware, where for example USB is pervasive, but USB is more complex and still not as reliable for debugging purposes.

Serial and Null Modem

Serial

Serial ports (often called RS-232 but it's really RS-232C) have disappeared as a standard option on many user level computers and have been replaced by USB. You can still find many business level x86 motherboards with serial ports. There are solutions for PC hardware without a serial port like USB to serial converters but we will talk about that later. Right now, all we need to know is that all our PowerPC based Amigas, classic Amigas and even the AmigaOne-X1000 have a serial port and the AmigaOS kernel's reaper outputs debug information over that serial port.

Serial ports most often come in two pin outs: 9 pins and 25 pins.

9 Pin Connector

25 Pin Connector

The 25 pin one, which is usually called DB25, was the first one and is used on the classic Amigas. After some time, software was simplified to use only half of the pins and then 9 pin connector (usually called DB9 but in reality it is DE9) was created. While classic Amigas have the older serial connectors with 25 pins, the Pegasos2, X1000 and Sams all have 9 pin connectors. In general, it makes no differences for our purposes. This is just information in case you decide to build your cable from scratch.

While the information given there is enough for our article, you still can check out Wikipedia for more information about serial ports and the RS232C standard itself at http://en.wikipedia.org/wiki/Serial_port.

Null Modem

In the good old days, when Modems/Teletypes were used to connect over the serial port, it was expected that one side is the Data Terminal Equipment (DTE) (usually a computer/terminal) and the other side is Data Circuit-terminating Equipment (DCE) (usually a modem). One side which is an output on a DTE is an input on a DCE and vice versa. Because of that DCE can be connected to a DTE with a straight wired cable.

Today things are different. Terminals and modems have mostly disappeared and computers have the same kind of serial port that terminals used to have: DTE. As a result, we can't connect them with a straight, pin to pin cable because it will connect the data transfer pins (TX to TX and RX to RX) like it was when we connect a serial port with a modem and that will not work. The solution is to swap the data transfer wires: TX to RX. That's what is simply called a Null Modem Cable (though without ground connected which is always better to do).

Usually when you go into a shop and find a modem cable it is most often not a null modem cable. You need to ask for a null modem cable where the data pins have been crossed over. Visually, the cables look exactly the same. Only the wires inside are not cross over and thus it will not transfer any serial debug output.

A null modem cable can one of 4 types:

1. No hardware handshaking
   This cable has only the data and signal ground wires connected. I.e. RX, TX and ground.
2. Loop back handshaking
   Just some "cheat" cable to create fake hardware flow control.
3. Partial handshaking
   Advanced "cheat" cable.
4. Full handshaking
   No cheats, just everything that can be in null modem cable.

In general, to create a basic null modem cable (that one, with "no hardware handshaking") it is enough to cross over pins 2 and 3 (RxD with TxD) and connect the ground pin. Even with such a simple null modem cable of just 3 wires, you will be able to output debug information and everything will work. You can of course build a cable with full handshaking and it will also work but the hardware may or may support it. For example, the X1000 does not support hardware handshaking and even if you have such a cable, the terminal software where you will capture your debug output you still need to choose the "no handshaking" option. Classic Amiga serial ports support hardware h handshaking. For example, AmigaExplorer from AmigaForever transfers data between Amiga and Windows computers via serial and handshaking helps in that case. So it's better to have a full-handshaking cable and to test it with your hardware. But if you need a null modem cable just for capturing debug output, a simple 3 wire null modem cable will allow up to 115200 and it's more than enough.

Note
If you have 2 computers which you want to connect via serial ports you need a null modem cable and not a straight serial cable.

Since serial ports can be only of 2 types (DB25 and DB9) there are 3 possible pin outs for null modem cables with full handshaking: DB25 to DB25, DB9 to DB9 and DB9 to DB25.

DB9 to DB9 serial cables are fairly easy to find in stores today but they are not likely to be null modem cables and you will need to work on them. DB25 are more rare and you may need to build a cable yourself. Just buy connectors and connect the necessary wires as shown above.

You can also build a cable which has 2 connectors on either side (i.e. DB9 and DB25 on one side and DB9 and DB25 on the other) so you can use either connector type as needed. This can be handy when you have a classic Amiga and modern PowerPC hardware and connected to a PC for example. To build such a cable just follow the same pin logic as shown below and just double them where needed.

Another way to solve the problem would be to buy a straight serial cable and add a small connector-adapter which will change the wiring from straight to null modem like in the screenshot below:

You can find even more information about null modem cables on Wikipedia.

Connect them now!

Serial<->Serial

There is nothing else you need except a working null modem cable. It must be a real null modem cable or a straight serial cable with a null modem adapter; doesn't matter which one.

Serial<->USB

This section is important and interesting for anyone who works with AmigaOS. The reason is that it is more difficult to find x86 hardware (which most of us have use as a necessary part of life) with built-in serial ports. It is even more difficult to find a notebook with a serial port. The solution is to use commonly available USB to serial adapters. There are plenty of them from all sorts of different companies. Despite the fact that they are common, some adapters can be better than others. Although most will work without problems and in practice all of the tested ones works fine, you still may found some adapters which can give you some problems. The reason is that USB does not directly translate to serial and data must be buffered between the two while at the same time keeping accurate timing. There is software inside the adapters and more than one way to implement the necessary hardware buffering thus not all adapters are equal and some are better than others. For serious serial work, nothing beats an actual UART which is why they are still an available option on industrial, laboratory and business PCs.

If you in interested in building a Serial to USB adapter yourself, you can find plans on the Internet. One example is this USB to serial adapter.

The look of the adapter can be different of course since it depends on what connectors are used when building a cable. I use one from Gembird based on the PS2303 chip on my notebook with Windows XP, where after installing of drivers it shows ups as COM17:

After you have the USB to serial adapter connected in your system, everything will be the same as if you connected Serial with Serial. A null modem cable is still required or a "straight" cable with a null modem adapter.

Serial<->Anything

You can connect serial to any port in a computer. You just need the correct adapter and cable. There are serial to USB (either purchased or made yourself), serial to Firewire adapters, serial to Ethernet, etc.

What I am trying to point here is that it does not matter how but you still need to make a real serial port in your system. USB to serial adapters are most popular because they are readily available, easy to find, inexpensive and well tested.

Terminal Software and port settings

Now, after you have your cable connected between 2 machines, you need software which will catch the data from one side. The "receiver" side can be an x86 system with Windows, Linux, Mac OS or whatever else. You could also use an Amiga with AmigaOS, pOS, Unix or whatever else runs on classic Amiga hardware or any kind of another hardware. The software is called "Terminal" software and since many of us use x86 systems to capture debug output from our Amigas, we will start with x86.

One of the best terminal software packages for x86 is PuTTY. You can get PuTTY from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html. Be sure that you get latest/full version because there are some stripped versions around with no support of serial connections. The normal version of PuTTY with working serial support will looks like this:

On Amiga (if you connect 2 Amigas by serial) you can use the old 68k Term 4.8:

On Linux you can also use PuTTY or the well-known minicom.

Whatever software is used, it is all about getting the debug data from your Amiga. Debug output is ASCII text data, so anything which can just grab information from your serial port and redirect it where you need it (console or file) will work. You can even write your own simple code which will receive data without the need for any terminal programs. But that's in theory, in practice it is better to use something which is already done.

The settings which you need to use and which always works on any Amiga hardware from which you want to get your serial output are:

baud: 115200 
data bits: 8 
stop bits: 1 
parity: none 
flow control: none

It may work with other settings (like different parity and flow control), but as it was pointed out before, on some hardware it may not work at all, on another it will not work as intended, so just these setting to do your first test. After all, you can always change your settings later and experiment after everything works.

How to redirect debug output to serial

By default, AmigaOS debug output is sent to a memory debug buffer and not to serial. This is done for simplicity. If something happens, especially bugs, then just type "DumpDebugBuffer" in a shell and it shows you what the debug output was. This is similar to using the Sashimi tool which redirects serial output to the console.

For developers, it is usually best to output everything to the serial port so you need to switch the output to serial. There are 2 ways to do this: boot option in the firmware (U-Boot for Sams/Eyetech AmigaOnes, OpenFirmware for Pegasos2 and CFE for X1000) and the "KDebug" utility.

Firmware method

Given there are so many different hardware platforms on which to run AmigaOS today, it should come as no surprise that they use different boot firmwares. AmigaOne-XE/MicroA1-C/Sam uses U-Boot, Pegasos2 uses OpenFirmware and the AmigaOne X1000 uses CFE. For all of them the boot menu options can be a bit different and you need to refer to the documentation which comes with the hardware. But in all cases you have a common variable called os4_commandline which you need to set. The name of that variable is the same for all the hardware platforms which support AmigaOS. Just the way the variable is setup varies.

For example on Pegasos2 OpenFirmware, to redirect output to serial with the "munge" debug kernel option and debug level 7 in OF you do:

Pegasos BIOS Extensions Copyright 2001-2004 by bplan GmbH.
All rights Reserved.
Auto-boot in 1 seconds - press ESC to abort, ENTER to boot: aborted
ok#
ok# setenv os4_commandline serial munge debuglevel=7

and then boot AmigaOS.

On AmigaOne X1000/CFE set munge, debuglevel to 4 and output to serial:

setenv -p os4_commandline "serial munge debuglevel=4"

and then boot AmigaOS. For CFE if you are not using -p, then the variable is only saved for that session, not permanently.

On Sam/AmigaOne U-Boot with the same munge, debuglevel=7 and serial redirect:

setenv os4_commandline "serial munge debuglevel=7"
saveenv

On X5000 it's the same U-Boot as with Sam/AmigaOne, so as example set serial, munge and debuglevel to 5:

setenv os4_commandline serial munge debuglevel=5
saveenv

In other words, use SetEnv to set the os4_commandline variable and then boot the OS. You can also verify what the variable is using the "printenv" command.

Just so you are not confused, "amigaboot.of" (the code which first boots) is the component which parses the os4_commandline firmware variable and didn't parse any command line arguments. It only parses firmware variables and then passes them on to the kernel. If you try something like amigaboot.of os4_commandline = "serial", it will not work and will be ignored, the same as any other extra parameters you provide. So be sure that you use "setenv" and check via "printenv" that you set the variable correctly.

KDebug method

KDebug is command line utility found in SYS:C which communicates with the AmigaOS kernel. With this utility you can redirect your output to serial or memory. It will override whatever you do with the os4_commandline firmware variable. This is very handy for setting different debug levels at the time you need it. For example, you may also put a command in S:User-Startup

run >NIL: kdebug "debuglevel 5"
run >NIL: kdebug "console serial"

This will set the debug level to 5 and output via the serial port.

To note ,that in firmware method we use '=' for debuglevel, i.e. 'debuglevel=5'. But when we use 'kdebug", then instead of '=' we put space, i.e. 'debuglevel 5'.

Note
Quotation marks are mandatory when using the KDebug tool. What KDebug does is send whatever string you enter directly to the AmigaOS kernel for interpretation. The KDebug command itself does not parse the string.

Note
KDebug method have higher priority. I.e. if you set your environment variable to some values, and then after os is boots use KDebug to change the settings, then they will be taken in account and previous one will be forgotten. The only thing that you can't enable/disable by KDebug is 'munge'.

Getting the Most out of Serial Debugging

To start with, you need to read the following articles:

1. The Debug Kernel
2. Exec Debug

To get the most out of serial debugging, developers should use what is called the debug kernel (named kernel.debug). The debug kernel provides you with more functionality compared with the release kernel at the cost of some execution speed. One thing provided is the ability to play with debug levels where you can choose how much "system" debug output you want to see. It's pretty easy to miss the fact that debug levels only work on debug kernels. You can play with levels, see no change, and you then wonder "Why doesn't it work?". It works, it is just that the kernel should be the debug one.

And while the debug kernel does give you some debug functionality, it still may not cover everything. For example, tools like the old MungWall tool can do a bit more: debug kernel has only the "munge" feature but not the "wall" feature. Another example is MemGuard (which can be found on OS4Depot) adds additional debug capabilities on top of what the debug kernel provides and catches more difficult to find bugs.

So, to get most of serial debugging, you need to use a combination: debug kernel with "munge" option enabled, MemGuard running in the background and set debug level to the value you find most informative. Usually 5 or 7 is enough but sometimes, when a bug is behaving very strangely, you can raise the value in the hope that you see something interesting.

If nothing works

Now on to troubleshooting. If you build/buy a cable, adapters and all the stuff, connect everything, fire up the terminal software, set up everything correctly and still nothing works, then:

1. Check the cable
2. Check the ports.
3. Check the cable again :)
4. Check program's settings and settings of the port if it USB to serial one.

Be sure that for you have right settings and in the terminal software (especially the baud, flow control and parity).

If you have a few spare connectors that you can use for troubleshooting, just solder a jumper between TX and RX (pins 2 and 3 on DB9). Then open a terminal program on the computer, set it up as described above, and turn "Local Echo" OFF. When Local Echo is OFF it does NOT show what you are typing, it only shows what comes in over the serial port.

The "loopback" connector has TX and RX connected, so it will loop whatever you send back to you. Plug a female loopback directly into the serial port connector and you should see whatever you type, because it is being "looped back" through your test plug. Now take the loopback off, connect your cable, and attach a male loopback at the other end of the cable. If you can see your typing, then you are successfully going out the serial port, down your cable, then looping back up the cable and back into your port.. You get the idea. Having one male and one female "loopback" will let you test ALMOST every possible cable, connection, and software configuration.

Desperate times: If you don't have any "spare" connectors or a soldering iron, a small paper clip can be used instead of a male loopback connector.. Just bend the clip and insert the ends into pins 2 and 3 of the cable end, and connect the other end to the serial port being tested. If the clip does not fit into the cable end, don't force it, find a smaller paper clip.

The one thing that a loopback can NOT test is whether your cable is a "Null Modem" cable or not. The best way to check a cable is with a multimeter. Set it so that when you touch wires on different sides you will heard a "beep" if it connected. That setting usually looks like a little speaker. Check that all of the connections are as they should be. If you use null modem adapter on top of "straight" serial cable, then check it anyway, just in case, with the same meter. The meter lead probably won't fit inside the female connector pins. Another small paper clip helps with that.

To work properly, (describe a null modem cable, assuming 9 pin to 9 pin):

• Pin 2 on end A should connect to pin 3 on end B.
• Pin 3 on end A should connect to pin 2 on end B.
• Pin 5 on end A should connect to pin 5 on end B.

If these three connections work, it should be enough to get you going.

Final words

While this topic is more or less an easy one, there wasn't a guide, until now, which describes everything from the beginning. What kind of cable you can use, how to build it, how to use USB to serial adapters, what settings should be used and what to do with it all. We hope that this guide will at least cover all the gaps and any developers told to "use serial debugging" will just read that article and will have no questions. Anyway, if you have any more info to add, or just found a wrong part somewhere, just send an email to kas1e@yandex.ru or use the AmigaOS Contact Form and we can update article.

Links

[1.] http://en.wikipedia.org/wiki/Serial_port
[2.] http://en.wikipedia.org/wiki/Null_modem
[3.] Debug Kernel
[4.] Exec Debug