DCFS and LNFS Low Level Data Structures - Revision history

Costel Mincea: Text replacement - "" to ""

2025-01-26T19:19:20Z

Text replacement - "<syntaxhighlight>" to "<syntaxhighlight lang="C" line>"

← Older revision		Revision as of 19:19, 26 January 2025
Line 29:		Line 29:

	Standard root and user directory blocks (512 bytes) for OFS and FFS would look like this:		Standard root and user directory blocks (512 bytes) for OFS and FFS would look like this:
	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct RootBlock {		struct RootBlock {
	LONG Type; /* This is used to mark the type of this		LONG Type; /* This is used to mark the type of this
Line 82:		Line 82:
	</syntaxhighlight>		</syntaxhighlight>

	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct UserDirectoryBlock {		struct UserDirectoryBlock {
	LONG Type; /* This is used to mark the type of this		LONG Type; /* This is used to mark the type of this
Line 123:		Line 123:
	Each directory list block begins with a header consisting of six fields, which is then followed by a variable number of list entries. The directory list block header looks as follows:		Each directory list block begins with a header consisting of six fields, which is then followed by a variable number of list entries. The directory list block header looks as follows:

	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct FileListBlock {		struct FileListBlock {
	LONG Type; /* This is used to mark the type of this		LONG Type; /* This is used to mark the type of this
Line 140:		Line 140:
	Each directory list entry begins with a header of constant size, which is followed by a variable number of bytes which contain the name and the comment text. The directory list entry is a "digest" of the directory/file/hard link/soft link block contents, which includes the name, comment and other metadata:		Each directory list entry begins with a header of constant size, which is followed by a variable number of bytes which contain the name and the comment text. The directory list entry is a "digest" of the directory/file/hard link/soft link block contents, which includes the name, comment and other metadata:

	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct FileListEntry {		struct FileListEntry {
	ULONG Key; /* Number of the block which this		ULONG Key; /* Number of the block which this
Line 195:		Line 195:
	If the FileListBlock.Type field is set to TYPE_DIRLIST_REV0 then the format of the directory list entries is slightly different:		If the FileListBlock.Type field is set to TYPE_DIRLIST_REV0 then the format of the directory list entries is slightly different:

	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct FileListEntry_Rev0 {		struct FileListEntry_Rev0 {
	ULONG Key; /* Number of the block which this		ULONG Key; /* Number of the block which this
Line 246:		Line 246:
	type. To illustrate, here is the default layout of a directory entry with 512 bytes per block; note the section enclosed in dashes:		type. To illustrate, here is the default layout of a directory entry with 512 bytes per block; note the section enclosed in dashes:

	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct UserDirectoryBlock {		struct UserDirectoryBlock {
	LONG Type; /* This is used to mark the type of this		LONG Type; /* This is used to mark the type of this
Line 296:		Line 296:
	the layout of the block was changed to allow for longer file names:		the layout of the block was changed to allow for longer file names:

	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct LongNameUserDirectoryBlock {		struct LongNameUserDirectoryBlock {
	LONG Type; /* This is used to mark the type of this		LONG Type; /* This is used to mark the type of this
Line 354:		Line 354:
	number is zero, then there is no comment block associated with this directory entry. The comment block's layout looks like this:		number is zero, then there is no comment block associated with this directory entry. The comment block's layout looks like this:

	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct CommentBlock {		struct CommentBlock {
	LONG Type; /* This is used to mark the type of this		LONG Type; /* This is used to mark the type of this
Line 383:		Line 383:
	the signature of the file system type. Here is how the root block would look like for a 512 byte block (note the sections enclosed in dashes):		the signature of the file system type. Here is how the root block would look like for a 512 byte block (note the sections enclosed in dashes):

	<syntaxhighlight>		<syntaxhighlight lang="C" line>
	struct RootBlock {		struct RootBlock {
	LONG Type; /* This is used to mark the type of this		LONG Type; /* This is used to mark the type of this

Steven Solie: /* Limitations and drawbacks */

2016-08-29T19:05:56Z

Limitations and drawbacks

← Older revision		Revision as of 19:05, 29 August 2016
Line 227:		Line 227:
	Due to the complexity of maintaining the directory cache, any update takes significantly longer than in the comparable OFS/FFS variants of the file system, and it modifies more data structures. During this operation the file system data structures are not entirely consistent at all times, and if the system should crash or the power should fail, then a validation may stall because of inconsistencies. The file system will have to be repaired. The DCFS mode makes this case more likely to appear and then adds insult to injury by making the validation process at least twice as long as with the comparable OFS/FFS variants of the file system.		Due to the complexity of maintaining the directory cache, any update takes significantly longer than in the comparable OFS/FFS variants of the file system, and it modifies more data structures. During this operation the file system data structures are not entirely consistent at all times, and if the system should crash or the power should fail, then a validation may stall because of inconsistencies. The file system will have to be repaired. The DCFS mode makes this case more likely to appear and then adds insult to injury by making the validation process at least twice as long as with the comparable OFS/FFS variants of the file system.

	There is no "garbage" collection process for the directory cache. If most of the entries are removed from a directory list block no merging		There is no "garbage collection" process for the directory cache. If most of the entries are removed from a directory list block no merging of neighbouring directory list blocks takes place. If all entries are removed from a directory list block, that block is not removed from the directory list and its storage space is not reclaimed. Only deleting the empty directory which contains the empty directory list chain will free that memory.
	of neighbouring directory list blocks takes place. If all entries are removed from a directory list block, that block is not removed from
	the directory list and its storage space is not reclaimed. Only deleting the empty directory which contains the empty directory list chain will free that memory.

	= Long Name File System (LNFS) =		= Long Name File System (LNFS) =

Steven Solie: /* Limitations and drawbacks */

2016-08-29T03:09:15Z

Limitations and drawbacks

← Older revision		Revision as of 03:09, 29 August 2016
Line 460:		Line 460:
	== Limitations and drawbacks ==		== Limitations and drawbacks ==

	The LNFS mode exists in two variants only, this being the OFS mode and		The LNFS mode exists in two variants only, this being the OFS mode and the FFS mode. Except for the extensions to the name length these modes operate like the traditional OFS and FFS modes, respectively. There is no DCFS variant of the LNFS mode.
	the FFS mode. Except for the extensions to the name length these modes
	operate like the traditional OFS and FFS modes, respectively. There is
	no DCFS variant of the LNFS mode.

	The LNFS mode always uses the "international" directory entry name		The LNFS mode always uses the "international" directory entry name hashing operation.
	hashing operation.

	Because the changes required in order to support the long file names		Because the changes required in order to support the long file names of the LNFS mode are minimal, all the limitations of the original
	of the LNFS mode are minimal, all the limitations of the original
	Amiga OFS and FFS modes are still present.		Amiga OFS and FFS modes are still present.

Steven Solie: /* Data structure changes and new data structures */

2016-08-29T03:08:32Z

Data structure changes and new data structures

Show changes

Steven Solie: /* Data structure changes and new data structures */

2016-08-29T03:06:31Z

Data structure changes and new data structures

← Older revision		Revision as of 03:06, 29 August 2016
Line 26:		Line 26:
	== Data structure changes and new data structures ==		== Data structure changes and new data structures ==

	The DCFS mode reuses the Amiga file system data structures which are utilized		The DCFS mode reuses the Amiga file system data structures which are utilized in the original file system (OFS) and the Fast File System (FFS) and builds upon them by adding directory lists blocks. The directory list blocks are attached to the root directory and all user directories through a field which was, up to 1992, unused.
	in the original file system (OFS) and the Fast File System (FFS) and builds
	upon them by adding directory lists blocks. The directory list blocks are
	attached to the root directory and all user directories through a field
	which was, up to 1992, unused.

	Standard root and user directory blocks (512 bytes) for OFS and FFS would look like this:		Standard root and user directory blocks (512 bytes) for OFS and FFS would look like this:

Steven Solie: /* History and purpose */

2016-08-29T03:05:37Z

History and purpose

← Older revision		Revision as of 03:05, 29 August 2016
Line 239:		Line 239:
	== History and purpose ==		== History and purpose ==

	The LNFS mode was introduced early during development of the FFS reimplementation which eventually shipped with AmigaOS 4~~.0 and MorphOS~~. These changes were introduced by Olaf Barthel, who created the FFS reimplementation in 2001.		The LNFS mode was introduced early during development of the FFS reimplementation which eventually shipped with AmigaOS 4. These changes were introduced by Olaf Barthel, who created the FFS reimplementation in 2001.

	The Amiga operating system has always supported long file names in the dos.library		The Amiga operating system has always supported long file names in the dos.library functions which retrieved the contents of directories (107 characters per entry). The Kickstart 3.0 ExAll() API function did not even place an upper limit on the length of a directory entry name. However, the default ROM file system only allowed for up to 30 characters per name to be used.
	functions which retrieved the contents of directories (107 characters per entry).
	The Kickstart 3.0 ExAll() API function did not even place an upper limit on the
	length of a directory entry name. However, the default ROM file system only allowed
	for up to 30 characters per name to be used.

	The LNFS mode removes the 30 character limitation for the names of files, directories,		The LNFS mode removes the 30 character limitation for the names of files, directories, hard links and soft links. The root directory name is, however, still restricted to 30 characters.
	hard links and soft links. The root directory name is, however, still restricted to
	30 characters.

			There is an additional feature in LNFS which is intended to improve performance when an LNFS volume is mounted. The file system no longer requires that the bookkeeping data structure (the "bitmap") which tracks the blocks currently allocated for data to be scanned completely before a read or write access can be made. Reading the entire bitmap just for the purpose of counting the number of bits set is costly. LNFS stores the number of blocks used as a counter value in the root directory. This counter will be updated for each change made to the bitmap, and should a validation be required, then the counter will be updated to reflect the status of the bitmap after validation.
	There is an additional feature in LNFS which is intended to improve performance when
	an LNFS volume is mounted. The file system no longer requires that the bookkeeping
	data structure (the "bitmap") which tracks the blocks currently allocated for data
	to be scanned completely before a read or write access can be made. Reading the entire
	bitmap just for the purpose of counting the number of bits set is costly. LNFS stores
	the number of blocks used as a counter value in the root directory. This counter will
	be updated for each change made to the bitmap, and should a validation be required,
	then the counter will be updated to reflect the status of the bitmap after validation.

	== Data structure changes and new data structures ==		== Data structure changes and new data structures ==

Steven Solie: /* Limitations and drawbacks */

2016-08-29T03:04:55Z

Limitations and drawbacks

@@ Line 222: / Line 222: @@
 == Limitations and drawbacks ==
-Because a copy of the metadata of the directory contents is made, this copy needs
+Because a copy of the metadata of the directory contents is made, this copy needs to be updated whenever the original changes. This alone makes DCFS mode slower than the comparable OFS/FFS variant when directory contents are modified.
 space, undoing the change) are making a directory update significantly more complex.
-Maintaining the consistency of the directory cache is the responsibility of the
+Maintaining the consistency of the directory cache is the responsibility of the file system validation process. If the volume needs to be rebuilt, two passes are necessary in order to first find out which data blocks are used, and then to rebuild the directory cache. This will easily double the total validation time.
-There is no "garbage" collection process for the directory cache. If
+There is no "garbage" collection process for the directory cache. If most of the entries are removed from a directory list block no merging
-most of the entries are removed from a directory list block no merging
+of neighbouring directory list blocks takes place. If all entries are removed from a directory list block, that block is not removed from
 = Long Name File System (LNFS) =

Steven Solie: /* How directory list entries are constructed for TYPE_DIRLIST_REV0 (32) */

2016-08-29T03:04:07Z

How directory list entries are constructed for TYPE_DIRLIST_REV0 (32)

← Older revision		Revision as of 03:04, 29 August 2016
Line 195:		Line 195:
	== How directory list entries are constructed for TYPE_DIRLIST_REV0 (32) ==		== How directory list entries are constructed for TYPE_DIRLIST_REV0 (32) ==

	If the FileListBlock.Type field is set to TYPE_DIRLIST_REV0 (32) then you		If the FileListBlock.Type field is set to TYPE_DIRLIST_REV0 (32) then you are dealing with a non-standard form of the directory list block which was the precursor to the standard form. Never use the TYPE_DIRLIST_REV0 (32) form in production code. The following documentation is provided for the sake of completeness only.
	are dealing with a non-standard form of the directory list block which was
	the precursor to the standard form. Never use the TYPE_DIRLIST_REV0 (32)
	form in production code. The following documentation is provided for the
	sake of completeness only.

	If the FileListBlock.Type field is set to TYPE_DIRLIST_REV0 then the format		If the FileListBlock.Type field is set to TYPE_DIRLIST_REV0 then the format of the directory list entries is slightly different:
	of the directory list entries is slightly different:

	<syntaxhighlight>		<syntaxhighlight>
Line 223:		Line 218:
	</syntaxhighlight>		</syntaxhighlight>

	In revision 0 the OwnerXID field is omitted from the constant size		In revision 0 the OwnerXID field is omitted from the constant size directory list entry header.
	directory list entry header.

	== Limitations and drawbacks ==		== Limitations and drawbacks ==

Steven Solie: /* How directory list entries are constructed for TYPE_DIRLIST_REV1 (33) */

2016-08-29T03:03:36Z

How directory list entries are constructed for TYPE_DIRLIST_REV1 (33)

← Older revision		Revision as of 03:03, 29 August 2016
Line 177:		Line 177:
	== How directory list entries are constructed for TYPE_DIRLIST_REV1 (33) ==		== How directory list entries are constructed for TYPE_DIRLIST_REV1 (33) ==

	If the FileListBlock.Type field is set to TYPE_DIRLIST_REV1 (33) then you		If the FileListBlock.Type field is set to TYPE_DIRLIST_REV1 (33) then you are dealing with the standard form of the directory list block. This is the only form which should be used in production code.
	are dealing with the standard form of the directory list block. This is
	the only form which should be used in production code.

	Directory list entries directly follow the directory list block header.		Directory list entries directly follow the directory list block header.

			Each entry consists of a constant size portion (FileListEntry), followed by a name and comment string in BCPL format (i.e. a single length byte is followed by exactly as many characters as indicated by the contents of the length byte; there is no terminating NUL byte). If the total size of the constant part, the name and the comment string is an odd number, then a padding byte must be added to the entry. The padding byte should be set to 0.
	Each entry consists of a constant size portion (FileListEntry), followed by a
	name and comment string in BCPL format (i.e. a single length byte is followed
	by exactly as many characters as indicated by the contents of the length byte;
	there is no terminating NUL byte). If the total size of the constant part,
	the name and the comment string is an odd number, then a padding byte must
	be added to the entry. The padding byte should be set to 0.

			Because the total size of the FileListEntry is not stored within itself, it must be calculated by looking at how long the name and comment data is, including the length byte which introduces each string. If the number obtained by adding the size of the FileListEntry and the name and comment strings (in BCPL format) is odd, 1 must be added to make it an even number (accounting for the required padding byte).
	Because the total size of the FileListEntry is not stored within itself,
	it must be calculated by looking at how long the name and comment data is,
	including the length byte which introduces each string. If the number
	obtained by adding the size of the FileListEntry and the name and
	comment strings (in BCPL format) is odd, 1 must be added to make it an
	even number (accounting for the required padding byte).

	The number of directory list entries must match the value of the		The number of directory list entries must match the value of the directory list block FileListBlock.NumEntries field.
	directory list block FileListBlock.NumEntries field.

	If there is enough spare room left in the directory list block,		If there is enough spare room left in the directory list block, then the list of directory list entries must be terminated by an ULONG value of 0. This looks like a partial FileListEntry with the Key field set to 0.
	then the list of directory list entries must be terminated by an
	ULONG value of 0. This looks like a partial FileListEntry with
	the Key field set to 0.

	The contents of FileListEntry.Created field are a truncated form of		The contents of FileListEntry.Created field are a truncated form of the 'struct DateStamp' which can be found, for example in the
		⚫	UserDirectoryBlock.Created field. The seconds and microseconds fit completely within the truncated form, but the number of days does not.
	the 'struct DateStamp' which can be found, for example in the
		⚫	As such the date range is restricted to 179 years instead of 4 billion years.
	UserDirectoryBlock.Created field. The seconds and microseconds fit
⚫	completely within the truncated form, but the number of days does not.
⚫	As such the date range is restricted to 179 years instead of 4 billion
	years.

	== How directory list entries are constructed for TYPE_DIRLIST_REV0 (32) ==		== How directory list entries are constructed for TYPE_DIRLIST_REV0 (32) ==

Steven Solie: /* How directory lists are constructed */

2016-08-29T03:02:33Z

How directory lists are constructed

← Older revision		Revision as of 03:02, 29 August 2016
Line 166:		Line 166:
	== How directory lists are constructed ==		== How directory lists are constructed ==

	The root directory and each user directory must contain a valid directory list.		The root directory and each user directory must contain a valid directory list. The RootBlock.Reserved4 and the UserDirectoryBlock.Reserved8 fields, respectively, contain the block number of the first directory list block for the root block/directory. This block number must never be 0.
	The RootBlock.Reserved4 and the UserDirectoryBlock.Reserved8 fields, respectively,
	contain the block number of the first directory list block for the
	root block/directory. This block number must never be 0.

	The FileListBlock.Type field must be set to TYPE_DIRLIST_REV1 (33). While the Amiga		The FileListBlock.Type field must be set to TYPE_DIRLIST_REV1 (33). While the Amiga file system for Kickstart 3.0 was under development, Commodore briefly supported a different type of directory list which can be identified as type TYPE_DIRLIST_REV0 (32). Details of the differences between TYPE_DIRLIST_REV1 and TYPE_DIRLIST_REV0 will be described later in the context of the directory list entry format.
	file system for Kickstart 3.0 was under development, Commodore briefly supported
	a different type of directory list which can be identified as type
	TYPE_DIRLIST_REV0 (32). Details of the differences between TYPE_DIRLIST_REV1
	and TYPE_DIRLIST_REV0 will be described later in the context of the directory list
	entry format.

	Each directory list block is part of a chain, in which the FileListBlock.NextBlock		Each directory list block is part of a chain, in which the FileListBlock.NextBlock field contains the block number of the next following entry in this chain, or 0 to indicate that no other directory list block follows.
	field contains the block number of the next following entry in this chain, or 0
	to indicate that no other directory list block follows.

	The number of entries in this directory list block is stored in		The number of entries in this directory list block is stored in FileListBlock.NumEntries. A value of 0 indicates that this block has no
	FileListBlock.NumEntries. A value of 0 indicates that this block has no
	directory entries stored.		directory entries stored.