Copyright (c) Hyperion Entertainment and contributors.

Difference between revisions of "AmigaDOS Vector-Port"

From AmigaOS Documentation Wiki
Jump to navigation Jump to search
 
(31 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
== Rationale ==
  
== Rationale ==
   
−
During development of file systems, developers often need to implement many parts to interface with DOS library rather than concentrate on implementing the actual file system functionality. The new Vector-port interface simplifies this by delegating most of this work to the DOS Library. This leaves only the function implementations up to the file system implementer.
 +
During development of file systems, the developer often needs to write a lot of code just to interface with DOS library. The new Vector-port interface simplifies this by delegating most of this interface work to the DOS Library. This means that the developer writing the file system can concentrate on implementing the functions required.
   
 
== Introduction ==
  
== Introduction ==
   
−
In the past, DOS library used to communicate with file systems and handlers exclusively using [[AmigaDOS_Packets|DOS packets]]. A DOS packet is merely an [[Exec_Messages_and_Ports#Messages|Exec Message]] with a structure appended in order to pass parameters and results. The aim being to remain as generic as possible with abstracted parameters. This pushes much of the work to the developer when they have to handle the DOS packet.
 +
In the past, DOS library used to communicate with file systems and handlers using DOS packets exclusively. A DOS packet is merely an Exec Message with parameters and results appended. The aim of this format is to remain as generic as possible with abstracted parameters. Unfortunately, this approach requires more overhead work for the developer to handle the packet, extract arguments from it and return results to it.
   
  +
[[File:dos-packet-based-file-system.png|frame|center|DOS Packet-Based File Systems]]
−
Starting with version 53.77, DOS Library offers the new "Vector-Port" API for file systems. Note this API is exclusively for use by file systems. It is basically a [[Exec_Messages_and_Ports#Message_Ports|Message port]] with an associated function list (also called "table of vectors" thus the API name). In this way, we are freed from many legacy things (notably BSTRs) and the functions let the developer know exactly what he is manipulating.
  
  +
  +
{{Note|text=The coloured boxes in this diagram and the ones below represent different task contexts and hence a task switch between them. The use of the vector-port API removes the task switch between the caller and the file system itself.}}
  +
  +
DOS packet-based file systems:
  +
* The server is asynchronous with the client.
  +
* Client can perform other tasks while waiting for the server.
  +
* Task switching is relatively slow.
  +
* Server must have access to client's address space.
  +
  +
Examples of DOS packet-based file systems include:
  +
* OFS (Old File System)
  +
* FFS, FFS2 (Fast File System)
  +
* PFS3 (Professional File System)
  +
  +
{{Note|text=The SFS, SFS2 and JXFS file systems use an early custom version of a vector-port like interface so they are actually not DOS packet-based. Keep that in mind when doing comparisons.}}
  +
  +
Starting with version 53.77, DOS Library offers the new ''Vector-Port'' API, exclusively for file systems. It is basically a [[Exec_Messages_and_Ports#Message_Ports|Message port]] with an associated function list (also called 'table of vectors' thus the API name). In this way, we are freed from many legacy constructs (notably BPTRs and BSTRs) and the arguments are clearly defined. The semantics of each function are now defined, so different file systems should behave identically when written to follow the API rules. The vector-port of a file system looks like a library and is called from DOS like a library (direct call).
  +
  +
[[File:dos-vector-port-based-file-system.png|frame|center|DOS Vector-Port-Based File Systems]]
  +
  +
DOS vector-port-based file systems:
  +
* Remove the need for task switching.
  +
* Server has access to client's address space.
  +
* Specific syntax and semantics of API to prevent variations.
  +
* Eliminates overlap of responsibilities between DOS and server.
  +
* Eliminates duplication of code in file systems and DOS.
  +
  +
Examples of DOS vector-port-based file systems include:
  +
* NGFS (New Generation File System)
  +
* ENV: (env-handler)
  +
* RAM: (ram-handler)
  +
* APPDIR: (appdir-handler)
  +
  +
{{Note|title=Times Have Changed|text=In the 68K days, I/O speed took the vast majority of time when compared to the Task switching speed. Today, I/O speed has increased to the point where Task switching speed is now taking an increased fraction of the total time.}}
   
 
== Legacy Support ==
  
== Legacy Support ==
   
−
Since version 53.95, DOS Library provides DOS packet emulation for legacy applications that still send DOS packets directly to the file system's Message Port. This effectively keeps these old applications from breaking on a complete Vector-Port based file system. This special DOS packet emulation functionality allows these 'application-sent' DOS packets to be automatically converted to equivalent Vector-Port calls.
 +
Since version 53.95, DOS Library provides DOS packet emulation for legacy applications that still send DOS packets directly to the file system's Message Port. This allows these old applications to work seamlessly with a complete Vector-Port based file system. This special DOS packet emulation functionality converts these 'application-sent' DOS packets to equivalent Vector-Port calls.
   
 
== Existing file system conversion ==
  
== Existing file system conversion ==
Line 19: Line 53:
 
== Key points ==
  
== Key points ==
   
−
Formerly a file system was required to allocate a Message Port. Now it only needs to replace this creation by a call to DOS method AllocDosObject() requesting creation of a Vector-Port and providing it the list of implemented functions. Everything else is automatically handled. Any unimplemented functions DOS library will fallback to sending a DOS packet. One of the advantage of this solution is because it uses direct function calls, it is substantially faster than sending a message and waiting for its handling by the file system. It is also processed on the caller's context thus we benefit from the OS multitasking feature to parallelize calls and there is no expensive context switching.
 +
Formerly a file system was required to allocate a Message Port. Now it only needs to replace this creation by a call to DOS method AllocDosObject(), requesting creation of a Vector-Port and providing it the list of implemented functions. Everything else is automatically handled. If unimplemented functions are called, DOS library falls back to sending a DOS packet. The advantage of this solution is that it uses direct function calls, so is substantially faster than sending a message and waiting for its handling by the file system. It is also processed on the caller's context thus we benefit from the OS multitasking feature to parallelize calls and there is no expensive context switching.
   
  +
== Examples ==
−
Example for ACTION_LOCATE_OBJECT DOS packet and the corresponding "Vector-Port" function FSLock():
  
   
  +
Example for ACTION_LOCATE_OBJECT DOS packet and the corresponding "Vector-Port" function FSLock():
 
<pre>
  
<pre>
−
INPUTS (DosPacket method)
 +
INPUTS (DosPacket method)
−
dp_Type - (int32) ACTION_LOCATE_OBJECT
 +
dp_Type - (int32) ACTION_LOCATE_OBJECT
−
dp_Arg1 - (BPTR) lock.
 +
dp_Arg1 - (BPTR) rel - Name reference lock.
−
dp_Arg2 - (BSTR) object_name. (may also include path)
 +
dp_Arg2 - (BSTR) name - Object name. (may also include path)
−
dp_Arg3 - (int32) Mode. SHARED_LOCK or EXCLUSIVE_LOCK.
 +
dp_Arg3 - (int32) mode - SHARED_LOCK or EXCLUSIVE_LOCK.
−
dp_Arg4 - <unused> 0
 +
dp_Arg4 - <unused> 0
−
dp_Arg5 - (BSTR) nameformat - name format indicator value.
 +
dp_Arg5 - (BSTR) nameformat - name format indicator value.
−
only if(dp_Arg2==dp_Arg5) then BSTR's are guaranteed to be
 +
only if(dp_Arg2==dp_Arg5) then BSTR's are guaranteed
−
a nul-terminated string. (53.23)
 +
to be nul-terminated. (53.23)
−
RESULT1 - (BPTR) Lock - (ZERO on failure.)
 +
RESULT1 - (BPTR) Lock - (ZERO on failure.)
−
RESULT2 - (int32) Failure code if RESULT1 == ZERO
 +
RESULT2 - (int32) Failure code if RESULT1 == ZERO
  +
</pre>
   
  +
<pre>
−
INPUTS (FileSystemVectorPort method)
  
−
struct Lock * RESULT1 = FSLock(struct FileSystemVectorPort *fsvp,
+
INPUTS (FileSystemVectorPort method)
  +
struct Lock * RESULT1 = FSLock(struct FileSystemVectorPort *fsvp,
−
int32 *result2, struct Lock *rel,
  
−
CONST_STRPTR name, int32 mode);
 +
int32 *result2, struct Lock *rel,
  +
CONST_STRPTR name, int32 mode);
−
fsvp - (struct FileSystemVectorPort *) Pointer to the vector port.
  
−
result2 - (int32 *) Pointer to the storage area for RESULT2.
 +
fsvp - (struct FileSystemVectorPort *) Pointer to the vector port.
−
rel_dir - (struct Lock *) Pointer to the 'name' reference dir lock.
 +
result2 - (int32 *) Pointer to the storage area for RESULT2.
−
name - (CONST_STRPTR) Pointer to the object name.(no path component)
 +
rel - (struct Lock *) Pointer to the 'name' reference lock.
  +
name - (CONST_STRPTR) Pointer to the object name.(no path component)
−
mode - (int32) SHARED_LOCK, EXCLUSIVE_LOCK.
  
  +
mode - (int32) SHARED_LOCK or EXCLUSIVE_LOCK.
−
RESULT1 - (struct Lock *) Pointer to the lock, NULL on failure.
  
  +
RESULT1 - (struct Lock *) Pointer to the lock, NULL on failure.
−
RESULT2 - (int32) Failure code if RESULT1 == NULL
  
  +
RESULT2 - (int32) Failure code if RESULT1 == NULL
 
</pre>
  
</pre>
   
−
== Developer tool ==
 +
=== ENV-Handler ===
  +
  +
The ENV-Handler has been refactored to use a DOS vector-port.
  +
  +
[[File:dos-vector-port-env-handler.png|frame|center|DOS Vector-Port-Based ENV-Handler]]
  +
  +
* Caches contents of Prefs/Env-Archive.
  +
* Reads Env-Archive at initialization time.
  +
* Does not write to Env-Archive.
  +
  +
=== RAM-Handler ===
  +
  +
The RAM-Handler has been refactored to use a DOS vector-port.
  +
  +
[[File:dos-vector-port-ram-handler.png|frame|center|DOS Vector-Port-Based RAM-Handler]]
  +
  +
* Acts like a disk-based file system.
  +
* Supports complete DOS Vector-Port API.
  +
* Was used as the model for testing activities.
  +
* Can use memory in the 2 to 4 GB address range.
  +
* Different attributes for meta data and file data.
  +
  +
=== FileSysBox (FUSE) ===
  +
  +
FUSE is from the Linux world and stands for Filesystem in Userspace. There are many file systems available which use FUSE.
  +
  +
[[File:dos-linux-fuse.png|frame|center|Linux FUSE]]
  +
  +
FileSysBox is a reimplementation of FUSE originally developed by Leif Salomonsson. FileSysBox has since been transformed from an old DOS packet-based architecture to a vector-port-based architecture for maximum speed and ease of maintenance.
  +
  +
[[File:dos-filesysbox-fuse.png|frame|center|FileSysBox FUSE]]
  +
  +
=== APPDIR-Handler ===
  +
  +
The APPDIR-Handler has always used the DOS vector-port API.
  +
  +
[[File:dos-appdir-handler.png|frame|center|DOS APPDIR-Handler]]
  +
  +
* Caches contents of Prefs/Env-Archive/AppDir.
  +
* Reads Env-Archive/AppDir at initialization time.
  +
* Updates with every task loading operation.
  +
* Purges old unused entries.
  +
* Writes to Env-Archive/AppDir.
  +
  +
=== New Generation File System ===
  +
  +
The New Generation File System was designed to use the DOS vector-port API.
  +
  +
[[File:dos-ngfs-architecture.png|frame|center|New Generation File System Architecture]]
  +
  +
* Supports journalling
  +
* Supports disk/device partition sizes from 1 MiB to 100 EiB
  +
* Supports object names up to 255 characters
  +
* Supports UTF-8 object names
  +
* Max file size (2^64 – 1) bytes
  +
* Disk/device sector sizes 512 – 64 kiB
  +
* Virtual block sizes 512 – 64 kiB
  +
* Supports hard and soft links like FFS2
  +
* Comments can be of any length
  +
* No limit to number of objects in a directory
  +
* 32-bit attributes Protection/Owner/Group
  +
* Write protection with password at volume level
  +
* Bootable on AmigaOne X-1000 and later platforms
  +
* Separate command for file system checking and repair
  +
  +
== Developer tools ==
  +
  +
=== FSVPtool ===
  +
  +
FSVPtool creates a ready to use skeleton to help with creation of new file systems. This tool is available in the latest SDK.
  +
  +
=== FSTest ===
  +
  +
FSTest is a tool to test your file system for compliance. Even the most mature file systems are not 100% compliant as this tool will demonstrate.
  +
  +
== Disk I/O Library ==
   
  +
More assistance to the file system developer is available via the [[DiskIO_Library|DiskIO Library]].
−
FSVPtool creates a ready to use skeleton to help with creation of new file systems. This tool should be available in the latest SDK.
  
   
 
== More information ==
  
== More information ==

Latest revision as of 20:09, 27 October 2016

Rationale

During development of file systems, the developer often needs to write a lot of code just to interface with DOS library. The new Vector-port interface simplifies this by delegating most of this interface work to the DOS Library. This means that the developer writing the file system can concentrate on implementing the functions required.

Introduction

In the past, DOS library used to communicate with file systems and handlers using DOS packets exclusively. A DOS packet is merely an Exec Message with parameters and results appended. The aim of this format is to remain as generic as possible with abstracted parameters. Unfortunately, this approach requires more overhead work for the developer to handle the packet, extract arguments from it and return results to it.

DOS Packet-Based File Systems
Note
The coloured boxes in this diagram and the ones below represent different task contexts and hence a task switch between them. The use of the vector-port API removes the task switch between the caller and the file system itself.

DOS packet-based file systems:

  • The server is asynchronous with the client.
  • Client can perform other tasks while waiting for the server.
  • Task switching is relatively slow.
  • Server must have access to client's address space.

Examples of DOS packet-based file systems include:

  • OFS (Old File System)
  • FFS, FFS2 (Fast File System)
  • PFS3 (Professional File System)
Note
The SFS, SFS2 and JXFS file systems use an early custom version of a vector-port like interface so they are actually not DOS packet-based. Keep that in mind when doing comparisons.

Starting with version 53.77, DOS Library offers the new Vector-Port API, exclusively for file systems. It is basically a Message port with an associated function list (also called 'table of vectors' thus the API name). In this way, we are freed from many legacy constructs (notably BPTRs and BSTRs) and the arguments are clearly defined. The semantics of each function are now defined, so different file systems should behave identically when written to follow the API rules. The vector-port of a file system looks like a library and is called from DOS like a library (direct call).

DOS Vector-Port-Based File Systems

DOS vector-port-based file systems:

  • Remove the need for task switching.
  • Server has access to client's address space.
  • Specific syntax and semantics of API to prevent variations.
  • Eliminates overlap of responsibilities between DOS and server.
  • Eliminates duplication of code in file systems and DOS.

Examples of DOS vector-port-based file systems include:

  • NGFS (New Generation File System)
  • ENV: (env-handler)
  • RAM: (ram-handler)
  • APPDIR: (appdir-handler)
Times Have Changed
In the 68K days, I/O speed took the vast majority of time when compared to the Task switching speed. Today, I/O speed has increased to the point where Task switching speed is now taking an increased fraction of the total time.

Legacy Support

Since version 53.95, DOS Library provides DOS packet emulation for legacy applications that still send DOS packets directly to the file system's Message Port. This allows these old applications to work seamlessly with a complete Vector-Port based file system. This special DOS packet emulation functionality converts these 'application-sent' DOS packets to equivalent Vector-Port calls.

Existing file system conversion

DOS allows for a step by step conversion of existing file systems to the Vector-Port API by falling back to sending a DOS packet when any Vector-Port function is NULL or the Vector-Port itself does not validate. This means that one can convert single functions one at a time by simply adding or removing the function vector from the Vector-Port initialisation table.

Key points

Formerly a file system was required to allocate a Message Port. Now it only needs to replace this creation by a call to DOS method AllocDosObject(), requesting creation of a Vector-Port and providing it the list of implemented functions. Everything else is automatically handled. If unimplemented functions are called, DOS library falls back to sending a DOS packet. The advantage of this solution is that it uses direct function calls, so is substantially faster than sending a message and waiting for its handling by the file system. It is also processed on the caller's context thus we benefit from the OS multitasking feature to parallelize calls and there is no expensive context switching.

Examples

Example for ACTION_LOCATE_OBJECT DOS packet and the corresponding "Vector-Port" function FSLock(): 

 INPUTS      (DosPacket method)
   dp_Type - (int32) ACTION_LOCATE_OBJECT
   dp_Arg1 - (BPTR)  rel  - Name reference lock.
   dp_Arg2 - (BSTR)  name - Object name. (may also include path)
   dp_Arg3 - (int32) mode - SHARED_LOCK or EXCLUSIVE_LOCK.
   dp_Arg4 - <unused> 0
   dp_Arg5 - (BSTR)  nameformat - name format indicator value.
              only if(dp_Arg2==dp_Arg5) then BSTR's are guaranteed
              to be nul-terminated. (53.23)
   RESULT1 - (BPTR)  Lock - (ZERO on failure.)
   RESULT2 - (int32) Failure code if RESULT1 == ZERO

 INPUTS      (FileSystemVectorPort method)
   struct Lock * RESULT1 = FSLock(struct FileSystemVectorPort *fsvp,
                                  int32 *result2, struct Lock *rel,
                                  CONST_STRPTR name, int32 mode);
   fsvp    - (struct FileSystemVectorPort *) Pointer to the vector port.
   result2 - (int32 *) Pointer to the storage area for RESULT2.
   rel     - (struct Lock *) Pointer to the 'name' reference lock.
   name    - (CONST_STRPTR) Pointer to the object name.(no path component)
   mode    - (int32) SHARED_LOCK or EXCLUSIVE_LOCK.
   RESULT1 - (struct Lock *) Pointer to the lock, NULL on failure.
   RESULT2 - (int32) Failure code if RESULT1 == NULL

ENV-Handler

The ENV-Handler has been refactored to use a DOS vector-port.

DOS Vector-Port-Based ENV-Handler
  • Caches contents of Prefs/Env-Archive.
  • Reads Env-Archive at initialization time.
  • Does not write to Env-Archive.

RAM-Handler

The RAM-Handler has been refactored to use a DOS vector-port.

DOS Vector-Port-Based RAM-Handler
  • Acts like a disk-based file system.
  • Supports complete DOS Vector-Port API.
  • Was used as the model for testing activities.
  • Can use memory in the 2 to 4 GB address range.
  • Different attributes for meta data and file data.

FileSysBox (FUSE)

FUSE is from the Linux world and stands for Filesystem in Userspace. There are many file systems available which use FUSE.

Linux FUSE

FileSysBox is a reimplementation of FUSE originally developed by Leif Salomonsson. FileSysBox has since been transformed from an old DOS packet-based architecture to a vector-port-based architecture for maximum speed and ease of maintenance.

FileSysBox FUSE

APPDIR-Handler

The APPDIR-Handler has always used the DOS vector-port API.

DOS APPDIR-Handler
  • Caches contents of Prefs/Env-Archive/AppDir.
  • Reads Env-Archive/AppDir at initialization time.
  • Updates with every task loading operation.
  • Purges old unused entries.
  • Writes to Env-Archive/AppDir.

New Generation File System

The New Generation File System was designed to use the DOS vector-port API.

New Generation File System Architecture
  • Supports journalling
  • Supports disk/device partition sizes from 1 MiB to 100 EiB
  • Supports object names up to 255 characters
  • Supports UTF-8 object names
  • Max file size (2^64 – 1) bytes
  • Disk/device sector sizes 512 – 64 kiB
  • Virtual block sizes 512 – 64 kiB
  • Supports hard and soft links like FFS2
  • Comments can be of any length
  • No limit to number of objects in a directory
  • 32-bit attributes Protection/Owner/Group
  • Write protection with password at volume level
  • Bootable on AmigaOne X-1000 and later platforms
  • Separate command for file system checking and repair

Developer tools

FSVPtool

FSVPtool creates a ready to use skeleton to help with creation of new file systems. This tool is available in the latest SDK.

FSTest

FSTest is a tool to test your file system for compliance. Even the most mature file systems are not 100% compliant as this tool will demonstrate.

Disk I/O Library

More assistance to the file system developer is available via the DiskIO Library.

More information

For further details and example code, refer to the autodoc: dos.dospackets.doc

Retrieved from "http://wiki.amigaos.net/w/index.php?title=AmigaDOS_Vector-Port&oldid=8926"