Programming AmigaOS 4: DOS - The Data Administrator

This article was adapted from Amiga Future magazine's series on developing for AmigaOS....

The file names that have become long thanks to AmigaOS 4 can make some of the old software/data administration sweat. We would like to show you this time what has changed in DOS.

The interface name for the dos.library is "IDOS" and its type is "struct DOSIFase *". Unlike other interface names all letters are in upper case and not only the first one. However, you don't need to open the interface itself. That's the job of the startup code. Only if you program libraries/devices or BOOPSI gadgets and data types must you take care of it yourself. So much about the pre-conditions.

Long names

Thanks to the new FFS2 (with the internal identifier DOS\7) both data names and directory names can have up to 107 characters. Hence, absolute path statements can also be longer than the so far maximum possible 254 characters (limit from BSTR). When in the past only names with 30 characters were allowed, you could create many directory depths, before you hit the limit. Now this is possible very quickly, at least in theory. You should keep this in mind, when you create local buffers in order to fill them, for example by using IDOS->NameFromLock(). This function would cause the error code ERROR_LINE_TOO_LONG (from dos/errors.h), if the buffer is too small. Old file managements such as Opus or MaxonTools reach their limits when they get to deal with longer names. They are then incompletely displayed and you may face some problems when you want to rename or edit them. The latest file processing has already been prepared for this.

The developer of the dos.library suggests a buffer length of 1024 characters. This length will probably never be reached in reality. Who on Earth would enter a path statement in the Shell window that stretches for 10 lines (80 column mode)?

Normal file operations

Nothing has changed regarding the normal file operations apart from the possibility to use long names. Nevertheless we would like to broach this subject, as misunderstandings keep happening in reality. Initially AmigaDOS offered only "LowLevel" file operations such as IDOS->Read() and IDOS->Write(). With Kickstart 2.0 the buffered file operations were also added. You can recognise them on the "F" in front: IDOS->FRead(), IDOS->FWrite() etc. Be careful not to switch between direct reading/writing and buffered reading/writing. DOS creates here an IDOS->FFlush(), if buffered data must be saved, but the whole thing is not really nice. The speed is also affected when reading as well as when buffering content that has already been read as it must always be discarded and positioned back into the file. An IDOS->Seek() is a very "expensive" file operation and therefore ideally entirely avoided, at least backward. In practice it is therefore better to simply read over data that is not needed instead of skipping it with "seek"! In the first examples "CopyFile.c" different DOS commands are used to read and write files.

/* CopyFile.c
 *
 *  gcc -o CopyFile CopyFile.c
 */
 
#include <exec/types.h>
#include <dos/dos.h>
#include <dos/rdargs.h>
#include <proto/dos.h>
#include <proto/utility.h>
 
int main()
{
  int res = RETURN_ERROR;
  struct RDArgs *readargs;
  int32          rargs[3];
 
  IUtility->ClearMem(rargs, sizeof(rargs));
 
  if((readargs = IDOS->ReadArgs("FROM/A,TO/A", rargs, NULL)))
  {
    uint8 *fromfile = (UBYTE *) (rargs[0]);
    uint8 *tofile   = (UBYTE *) (rargs[1]);
    BPTR fhin, fhout;
    uint8 buffer[1024];
    int32 readbyte;
 
    if((fhin = IDOS->Open(fromfile, MODE_OLDFILE)))
    {
      if((fhout = IDOS->Open(tofile, MODE_NEWFILE)))
      {
        res = RETURN_OK;
        while((readbyte = IDOS->Read(fhin, buffer, sizeof(buffer))))
        {
          if(IDOS->Write(fhout, buffer, readbyte) != readbyte)
          {
            IDOS->PrintFault(IDOS->IoErr(), tofile);
            res = RETURN_FAIL;
            break;
          }
        }
        IDOS->Close(fhout);
      }
      else IDOS->PrintFault(IDOS->IoErr(), tofile);
      IDOS->Close(fhin);
    }
    else IDOS->PrintFault(IDOS->IoErr(), fromfile);
 
    IDOS->FreeArgs(readargs);
  }
  else IDOS->PrintFault(IDOS->IoErr(), NULL);
 
  if(res == RETURN_OK) IDOS->Printf("Copy successful\n");
 
  return res;
}

Create directories

So far it has been quite arduous to create whole hierarchies of directories and sub-directories. Backup programs or the like are the usual candidates here. Now it is possible to create a complete path with a single command. In reference to IDOS->CreateDir() the new function is called IDOS->CreateDirTree(). The relative or absolute path entry is expected. You should keep in mind here that the last directory name must be entered without the concluding slash "/". This is possible in the shell-command "MakeDir", because the command makes sure that it is removed. If parts of the path already exist, there will be no error. Only if a directory of the path can't be created, the function will return with an error code. A possible error source is, for instance, a file with the name of the drawer that is to be created. A FileLock is returned to the last directory, which is cleared with IDOS->UnLock() by the program. As usual, we've created a short sample program "CreateDir.c". IDOS->ReadArgs() is also used for parsing the command line in the source code and IDOS->PrintFault() for error reports.

/* CreateDir.c
 *
 *  gcc -o CreateDir CreateDir.c
 */
 
#include <exec/types.h>
#include <dos/dos.h>
#include <dos/rdargs.h>
 
#include <proto/dos.h>
#include <proto/utility.h>
 
 
int main()
{
  int res = RETURN_ERROR;
  struct RDArgs *readargs;
  int32          rargs[2];
  BPTR           fh;
 
  IUtility->ClearMem(rargs, sizeof(rargs));
 
  if((readargs = IDOS->ReadArgs("DRAWER/A/M", rargs, NULL)))
  {
    if(rargs[0])
    {
      /* run through the list of arguments, */
      CONST_STRPTR *drawer = (CONST_STRPTR *) rargs[0];
      while(*drawer)
      {
        /* create directory with full path when not exists */
        if((fh = IDOS->CreateDirTree(*drawer)))
        {
          IDOS->Printf("Path <%s> created successfully\n",*drawer);
          IDOS->UnLock(fh);
        }
        else IDOS->PrintFault(IDOS->IoErr(), *drawer);
 
        drawer++;
      }
    }
 
    IDOS->FreeArgs(readargs);
  }
  else IDOS->PrintFault(IDOS->IoErr(),NULL);
 
  return res;
}

Date

So far there has been no direct possibility to get from the second-based timestamp (as used by ITimer->GetSysTime()) to the AmigaDOS timestamp (based on days/minutes/ticks). Both new functions in charge of this task are IDOS->DateStampToSeconds() and IDOS->SecondsToDateStamp(). The result is a uint32-value with the seconds or a TimeStamp structure with the Amiga-values. However, you must keep in mind that the ANSI-C Timer functions can't be mixed with the AmigaDOS-Timer functions! ANSI-C has been delivering the seconds since 1.1.1970, while DOS uses the 1.1.1978 as its basis. So it becomes clear that if you mix them you will get the wrong results. We've created a practical example under "CurrentTime.c", which also uses the timer.device, but we'll get back to it in another part of this workshop. You should also keep in mind that the "struct timerequest" has been renamed in "TimeRequest" as "devices/timer.h" so there won't be any collisions/mixups with the structure with the same name in the C-libraries. If you want to use the old names (e.g. when porting), you can turn them on per Define __USE_OLD_TIMEVAL__.

/* CurrentTime.c
 *
 *  gcc -o CurrentTime CurrentTime.c
 */
 
#include <exec/types.h>
#include <exec/io.h>
#include <exec/memory.h>
#include <devices/timer.h>
#include <dos/datetime.h>
 
#include <proto/exec.h>
#include <proto/dos.h>
 
uint32 GetSysTime()
{
  uint32 res = 0;
  struct TimeRequest *timerio;
  struct MsgPort     *timerport;
 
  if((timerport = IExec->AllocSysObjectTags(ASOT_PORT, TAG_END)))
  {
    if((timerio = IExec->AllocSysObjectTags(ASOT_IOREQUEST,
                           ASOIOR_ReplyPort, timerport,
                           ASOIOR_Size, sizeof(struct TimeRequest), TAG_END)))
    {
      if(IExec->OpenDevice("timer.device", UNIT_MICROHZ, (struct IORequest *)timerio, 0) == 0)
      {
        /* send synchron command to device */
        timerio->Request.io_Command = TR_GETSYSTIME;
        IExec->DoIO((struct IORequest *) timerio);
 
        /* we will retunr only seconds, microseconds are ignored */
        res = timerio->Time.Seconds;
 
        IExec->CloseDevice((struct IORequest *) timerio);
      }
      else IDOS->Printf("Error: cannot open timer.device\n");
 
      IExec->FreeSysObject(ASOT_IOREQUEST, timerio);
    }
    else IDOS->Printf("Erro: cannot create IORequest\n");
 
    IExec->FreeSysObject(ASOT_PORT, timerport);
  }
  else IDOS->Printf("Error: cannot create MsgPort\n");
 
  return( res );
}
 
 
int main()
{
  const ULONG act = GetSysTime();
  struct DateStamp ds = {0};
 
  IDOS->SecondsToDateStamp(act, &ds);
  IDOS->Printf("%ld seconds have elapsed since 1.1.1978\n", act);
  IDOS->Printf("or %ld days, %ld minutes and %ld ticks\n", ds.ds_Days, ds.ds_Minute, ds.ds_Tick);
 
  TEXT daystr[LEN_DATSTRING], timestr[LEN_DATSTRING];
  struct DateTime dt = { ds.ds_Days, ds.ds_Minute, ds.ds_Tick, FORMAT_DEF, 0, NULL, daystr, timestr };
  IDOS->DateToStr(&dt);
  IDOS->Printf("Today is %s at %s o'clock\n", daystr, timestr);
 
  return 0;
}

The function to calculate the second timestamp is new: IDOS->AddDates() and IDOS->SubtractDates(). The first one, adds two dates and returns the result in the first argument. In the second one an Argument1 ? Argument2 is calculated and the result is stored again in Argument1. Both functions return DOSTRUE if valid parameters are used.

Notification

By using notifications you can get informed about changes in the files or directories. Interestingly enough you can monitor files that still don't exist. But as soon as you create them, you will get a change notification. You can select between Signal or Message Type, but the latter is recommended. Then you will get the message type 'struct NotifyMessage'. With nm_NReq.nr_Name you can determine the name of the monitored file/directory. The short example "SignalNotification.c" shows you how the function IDOS->StartNotify() is used in practice.

/* SignalNotification.c
 *
 *  gcc -o SignalNotification SignalNotification.c
 */
 
#include <exec/types.h>
#include <exec/memory.h>
#include <dos/dos.h>
#include <dos/dosasl.h>
#include <dos/notify.h>
#include <dos/rdargs.h>
 
#include <proto/exec.h>
#include <proto/dos.h>
#include <proto/utility.h>
 
int main()
{
 
  struct RDArgs *readargs;
  int32          rargs[2];
  uint32         signr;
 
  IUtility->ClearMem(rargs, sizeof(rargs));
 
  if((readargs = IDOS->ReadArgs("FILENAME/A", rargs, NULL)))
  {
    STRPTR filename = (STRPTR) (rargs[0]);
 
    if((signr = IExec->AllocSignal(-1)) != -1)
    {
      struct NotifyRequest notifyrequest = {0};
 
      /* Send a signal to our task when file/directory are changed */
      /* NRF_NOTIFY_INITIAL = send immediately a signal when file/directory are exist */
      notifyrequest.nr_Name = filename;
      notifyrequest.nr_Flags = NRF_SEND_SIGNAL | NRF_NOTIFY_INITIAL;
      notifyrequest.nr_stuff.nr_Signal.nr_Task = (struct Task *) IExec->FindTask(NULL);
      notifyrequest.nr_stuff.nr_Signal.nr_SignalNum = signr;
 
      if((IDOS->StartNotify(&notifyrequest)) == DOSTRUE)
      {
        for(;;)
        {
          const ULONG signal = IExec->Wait(1L << signr | SIGBREAKF_CTRL_C);
 
          if(signal & (1L << signr))
          {
            /* we have notification */
            IDOS->Printf("Notification from '%s' !\n",notifyrequest.nr_FullName);
          }
 
          if(signal & SIGBREAKF_CTRL_C)
          {
            /* CTRL-C = quit program */
            IDOS->EndNotify(&notifyrequest);
            IDOS->PrintFault(ERROR_BREAK,NULL);
            break;
          }
        }
      }
      else IDOS->PrintFault(ERROR_NOT_IMPLEMENTED,NULL);
 
      IExec->FreeSignal(signr);
    }
    else IDOS->Printf("Error: there is no signal available\n");
 
    IDOS->FreeArgs(readargs);
  }
  else IDOS->PrintFault(IDOS->IoErr(), NULL);
 
  return 0;
}

That's for the theory that was applicable to the old FastFile system. But some issues arise with AmigaOS 4 and the new FastFileSystem2. But let me make a digression and write about the file system. Let's assume that we have a dir1, then a dir2 and in it a file3 in the RAMDisk. If the file3 is changed or something new is created in the dir2, the file3 (i.e. the newly created one) will get the current timestamp. Furthermore, the superordinate directory dir2 will also get the current timestamp. The novelty with AmigaOS 4 (and in theory also the correct thing) is that this process goes all the way to the root directory. So the dir1 gets the current timestamp too. The advantage is obvious; you must only take a look at the root directory of the data storage medium to realise when and in what directory something has been change. Whether this has happened one level down or five levels down is irrelevant at this point. The user can click himself through the directories to get to the changed content. Also backup programs with the rule "everything since '.'" can work more expeditiously and must not search all directories.

The whole things becomes problematic in practice with the notifications. Let's suppose, we're monitoring the dir1 and change the file3. Before AmigaOS 4 we wouldn't have got a notification about the change. Since AmigaOS 4, however, we get the change notification, although strictly speaking nothing has changed in the monitored directory. At the moment is not known how this dilemma will be solved. The developers presume that it's better if one notification too many occurs than one notification too few. But the user might be confused, if, for example, the list in the file viewer keeps being re-configured without seeing a change afterwards.

Virtual drives

Various virtual drives have been integrated with AmigaOS 4. This way you can address a CDROM-Image (ISO file) over ICD0: as if it were a burned CD. Using a comfortable GUI (DiskImageGUI) you can 'create' and delete the images. The whole thing works also with ADF files that are addressed as IDF0:. Also ENV: is no longer re-copied in the RAM-Disk, but administered with its own handler and synchronised with ENVARC: on the hard disk. Another drive is URL: with two different options. You can basically start a web-browser with it and display an Internet site (HTTP). Alternatively the displayed file can also lie locally on the hard disk (FILE). The second option is opening the mail program. You can configure the whole thing by using the "URLOpen" Prefs-program.

  BPTR handle = IDOS->Open("URL:http://www.amigafuture.de", MODE_OLDFILE);
  BPTR handle = IDOS->Open("URL:file=Work:docs/demo.html", MODE_OLDFILE);
  BPTR handle = IDOS->Open("URL:mailto.name@amigafuture.de", MODE_OLDFILE);
  if(handle) IDOS->Close(handle).

To check whether the URL device is registered you can use the following Block Code:

  APTR oldwin = IDOS->SetProcWindow((APTR)-1); 
  handle = IDOS->Open("URL:NIL:",MODE_OLDFILE);
  IDOS->SetProcWindow(oldwin);
 
  if( handle )
  {
    IDOS->Close(handle);
    /* URL-Device can be used */
  }

Obsolete

With AmigaOS 4 functions from the dos.library have been dropped out or are reported as "DEPRECATED" when compiling. Some of them are, for example, Examine() or Seek(). The reason for this is that these functions can handle files with a maximum of 2 GByte size (because of the data type int). Completely new functions have been created here as supplement. Let's take a brief look at them. With Examine() you can request information about a file or a directory. It has been replaced with ExamineObject(). Its counterpart is FreeDosObject(DOS_EXAMINEDATA) that you use to deallocate the object. The structure ExamineData contains comparable values like the old FileInfoBlock, but values about the file size are available as 64 bit Integer. In order to go through directories using ExAll/ExNext/ExAllEnd you can now use the ExamineDir() function that gives you the next entry with each request. You need an administration block created with ObtainDirContext() which helps you define the result data and which you can deallocate with ReleaseDirContext(). Instead, with Seek() you position with ChangeFilePosition() in the file and with GetFilePosition() you request the current position. To change the file size you need ChangeFileSize() instead of SetFileSize(). Keep in mind that the function of the physical data blocks must be set up on the hard disk and hence it can take a while in very large areas until the function returns. The SetOwnerInfo() function for file rights supplements the so far used SetOwner() function if you need it at all. With Execute() you have several options to replace it. System() or RunCommand() are the candidates to use. System() is thanks to its tag list far easier to handle. By using CreateProc() you can create new processes; this function must be replaced with CreateNewProc() or you can resort directly to System(). "NewDosFunctions.c" shows an example with the new functions.

/* NewDosFunctions.c
 *
 *  gcc -o NewDosFunctions NewDosFunctions.c
 */
 
#include <exec/types.h>
#include <proto/dos.h>
#include <stdlib.h>
 
int main(int argc, char *argv[])
{
 
  BPTR lock;
  BPTR fh;
  struct ExamineData *ed;
  struct ExamineData *edata;
  APTR context;
 
  if((lock = IDOS->Lock("",SHARED_LOCK)))
  {
    if((ed = IDOS->ExamineObjectTags(EX_FileLockInput, lock, TAG_END)))
    {
      IDOS->Printf("Directory <%s> ...\n", ed->Name);
 
      if((context = IDOS->ObtainDirContextTags(EX_FileLockInput,lock,
                            EX_DoCurrentDir,TRUE,
                            EX_DataFields,EXF_ALL,
                            TAG_END)))
      {
        while((edata = IDOS->ExamineDir(context)))
        {
          if( EXD_IS_LINK(edata) )
          {
            IDOS->Printf("%s [link]\n",edata->Name);
            IDOS->Printf("   File is linked to <%s>\n", edata->Link);
          }
          else if ( EXD_IS_DIRECTORY(edata) )
          {
            IDOS->Printf("%s [directory]\n", edata->Name);
          }
          else if( EXD_IS_FILE(edata) )
          {
            IDOS->Printf("%s [file]\n", edata->Name);
          }
        }
 
        if(IDOS->IoErr() != ERROR_NO_MORE_ENTRIES)
        {
          IDOS->Printf("Directory scan error\n");
        }
      }
      else IDOS->Printf("Can't create context\n");
 
      IDOS->ReleaseDirContext(context);
    }
    else IDOS->Printf("Can't examine object\n");
 
    IDOS->FreeDosObject(DOS_EXAMINEDATA, ed);
 
    IDOS->UnLock(lock);
  }
  else IDOS->Printf("Can't lock current dir\n");
 
 
  if((fh = IDOS->Open("datei", MODE_NEWFILE)))
  {
    IDOS->ChangeFilePosition(fh, 100, OFFSET_BEGINNING);
 
    IDOS->ChangeFileSize(fh, 1024, OFFSET_BEGINNING);
 
    IDOS->Close(fh);
  }
  else IDOS->Printf("Can't open file <datei>\n");
 
 
  return 0;
}

The description of the old functions has been stored in "dos.obsolete.doc". You can still compile old sources with these functions without any problems. At some point in the future they certainly won't be used any more; at the latest then you must adjust the code. Above all, when creating new sources or revising existing sources you should switch to the new functions.

Forbidden

We would also like to mention the Exit() functions that can lead to a compiling error, as they no longer exist. You can simply replace them with exit() for the Runtime library.

You must relocate formatted file outputs via VFWritef() to VFPrintf(). The greater difference lies in the format string, as it was coded in the old function as a BCPL string and is therefore not directly compatible with the RawDoFmt() placeholders.

End

Now we've come to the end of this part with DOS. Next time we will talk about Intuition. A variety of new functions support the programmer in his work. You will get to see something more visual then, as the last two parts deal with the background.

New (selected) functions in the dos.library:

int32 AddDates(struct DateStamp *to, CONST struct DateStamp *from) : Adds dates
BPTR CreateDirTree(STRPTR path) : creates old directory trees
uint32 DateStampToSeconds(struct DateStamp *ds) : Converts a DateStamp structure into seconds
int32 DevNameFromLock(BPTR fh, STRPTR buffer, int32 buffersize) : Gives the device name (not the data carrier name !) to the assigned lock
int32 DosControl(struct TagItem *taglist) : Setting and reading DOS internals
BPTR ErrorOutput(void) : gives the error output stream (stderr)
int32 FixDateStamp(struct DateStamp *ds) : if necessary, it corrects the datestamp entries
BPTR GetCurrentDir(VOID) : gives a lock for the current directory
int32 GetSegListInfo(BPTR seglist, struct TagItem *taglist) : gives information about the loaded program segment
int32 HexToLong(STRPTR buffer, uint32 *val) : Transforms a hexadecimal number into a long value
int32 PutErrStr(STRPTR txt) : Text output in the error output stream
struct DateStamp * SecondsToDateStamp(uint32 sec, struct DateStamp *ds) : Converts seconds into a datestamp structure
BPTR SelectErrorOutput(BPTR fh) : Changes the error output stream
APTR SetProcWindow(APTR ) : sets the pr_WindowPtr
int32 SubtractDates(struct DateStamp *ds1, CONST struct DateStamp *ds2) : Subtracts dates
int32 WaitForData(BPTR stream, int32 data_direction, int32 timeout) : waits until the stream is ready to send/read (especially Pipes)

What has become deprecated:

LONG Seek(BPTR file, LONG position, LONG offset)
-> ChangeFilePosition()
LONG SetFileSize(BPTR fh, LONG pos, LONG mode)
->ChangeFileSize()
LONG SetOwner(CONST_STRPTR name, ULONG owner_info)
-> SetOwnerInfo()
LONG Examine(BPTR lock, struct FileInfoBlock *fib)
-> ExamineObject()
LONG ExamineFH(BPTR fh, struct FileInfoBlock *fib)
-> ExamineObject()
LONG ExAll(BPTR lock, struct ExAllData *buffer, LONG size, LONG data, struct ExAllControl *control)
-> ExamineDir()
LONG ExNext(BPTR lock, struct FileInfoBlock *fib)
-> ExamineDir()
VOID ExAllEnd(BPTR lock, struct ExAllData *buffer, LONG size, LONG data, struct ExAllControl *control)
-> ExamineDir()
struct MsgPort *CreateProc(CONST_STRPTR name, LONG pri, BPTR segList, LONG stackSize)
-> CreateNewProc()
struct MsgPort *DeviceProc(CONST_STRPTR name)
-> GetDeviceProc()
LONG ReadLink(struct MsgPort *port, BPTR lock, CONST_STRPTR path, STRPTR buffer, ULONG size)
-> ExamineObject()
LONG SetVBuf(BPTR fh, STRPTR buff, LONG type, LONG size);
-> SetFileHandleAttr
LONG Execute(CONST_STRPTR string, BPTR file, BPTR file2);
-> RunCommand()

What has become obsolete:

VOID Exit(LONG returnCode);
-> exit()
VOID VFWritef(BPTR fh, CONST_STRPTR format, LONG *argarray)
-> VFPrintf()
VOID FWritef(BPTR fh, CONST_STRPTR format, ...)
-> VFPrintf()
BPTR InternalLoadSeg(BPTR fh, BPTR table, const LONG *funcarray)
-> LoadHunk()
LONG InternalUnLoadSeg(BPTR seglist, VOID (*freefunc )())
-> UnLoadHunk()
BPTR LoadSeg(CONST_STRPTR name, BPTR hunktab, BPTR stream)
-> RunCommand

Authors

Written by Michael Christoph and Aleksandra Schmidt-Pendarovska
Copyright (c) 2013 Michael Christoph