File size/Number of file limits

On Fri, 26 Oct 2007 02:05:23 -0400, 98 Guy put finger to
keyboard and composed:

I performed a series of serial file generations to see how many files
could be created in a subdirectory while changing the length of the
file name.

The following table shows the file-name length and the corresponding
number of files that could be created before this eror was generated:

The directory or file cannot be created

6.0 - 65,533
6.3 - 32,767
12.3 - 21,845
15.3 - 21,845
17.3 - 21,845
23.3 - 16,384
47.3 - 13,107
63.3 - 9,362
96.3 - 7,282

So in the first case, with a filename composed of 6 characters (and no
suffix) I was able to create 65,533 files. In the second case, the
filename was composed of 6-characters.3-characters, and the directory
would only hold 32,767 of those files. So as the filename length
increases, there is a decrease in the number of possible files.

According to the OP:

The target 'contracts' directory is 599MB and contains 16135
objects. We are appending a few hundred files. We get the
following error:
'The directory or file cannot be created.

For the above to happen, the existing 16,135 files would have to have
long file names (about 20 characters, more likely 23 characters,
possibly slightly more).

I created a file named "dummy" and then copied it to several files on
a floppy diskette as follows.

================================================== ================
C:\WIN98SEecho blah dummy

C:\WIN98SEfor %i in (7 78 789 7890 78901 789012 7890123) do copy
dummy a:\01234567890123456%i.txt

C:\WIN98SEdir a:

012345~3 TXT 7 10-27-07 8:49a 012345678901234567.txt
012345~4 TXT 7 10-27-07 8:49a 0123456789012345678.txt
012345~5 TXT 7 10-27-07 8:49a 01234567890123456789.txt
012345~6 TXT 7 10-27-07 8:49a 012345678901234567890.txt
012345~7 TXT 7 10-27-07 8:49a 0123456789012345678901.txt
012345~8 TXT 7 10-27-07 8:49a 01234567890123456789012.txt
012345~9 TXT 7 10-27-07 8:49a 012345678901234567890123.txt
================================================== ================

I then dumped the directory structure using Debug. It appears that
there are 3 entries for filenames of 21 characters, and 4 entries for
22 and 23 characters.

================================================== ===================
4A0 42 33 00 34 00 35 00 36-00 37 00 0F 00 7B 38 00 B3.4.5.6.7...{8.
4B0 39 00 30 00 31 00 2E 00-74 00 00 00 78 00 74 00 9.0.1...t...x.t.
4C0 01 30 00 31 00 32 00 33-00 34 00 0F 00 7B 35 00 .0.1.2.3.4...{5.
4D0 36 00 37 00 38 00 39 00-30 00 00 00 31 00 32 00 6.7.8.9.0...1.2.
4E0 30 31 32 33 34 35 7E 37-54 58 54 20 00 94 53 46 012345~7TXT ..SF
4F0 5B 37 5B 37 00 00 27 46-5B 37 11 00 07 00 00 00 [7[7..'F[7......

500 43 74 00 00 00 FF FF FF-FF FF FF 0F 00 1B FF FF Ct..............
510 FF FF FF FF FF FF FF FF-FF FF 00 00 FF FF FF FF ................
520 02 33 00 34 00 35 00 36-00 37 00 0F 00 1B 38 00 .3.4.5.6.7....8.
530 39 00 30 00 31 00 32 00-2E 00 00 00 74 00 78 00 9.0.1.2.....t.x.
540 01 30 00 31 00 32 00 33-00 34 00 0F 00 1B 35 00 .0.1.2.3.4....5.
550 36 00 37 00 38 00 39 00-30 00 00 00 31 00 32 00 6.7.8.9.0...1.2.
560 30 31 32 33 34 35 7E 38-54 58 54 20 00 1D 54 46 012345~8TXT ..TF
570 5B 37 5B 37 00 00 27 46-5B 37 12 00 07 00 00 00 [7[7..'F[7......

580 43 78 00 74 00 00 00 FF-FF FF FF 0F 00 3B FF FF Cx.t.........;..
590 FF FF FF FF FF FF FF FF-FF FF 00 00 FF FF FF FF ................
5A0 02 33 00 34 00 35 00 36-00 37 00 0F 00 3B 38 00 .3.4.5.6.7...;8.
5B0 39 00 30 00 31 00 32 00-33 00 00 00 2E 00 74 00 9.0.1.2.3.....t.
5C0 01 30 00 31 00 32 00 33-00 34 00 0F 00 3B 35 00 .0.1.2.3.4...;5.
5D0 36 00 37 00 38 00 39 00-30 00 00 00 31 00 32 00 6.7.8.9.0...1.2.
5E0 30 31 32 33 34 35 7E 39-54 58 54 20 00 6E 54 46 012345~9TXT .nTF
5F0 5B 37 5B 37 00 00 27 46-5B 37 13 00 07 00 00 00 [7[7..'F[7......
================================================== ===================

- Franc Zabkar
--
Please remove one 'i' from my address when replying by email.

On Sat, 27 Oct 2007 09:05:28 +1000, Franc Zabkar
put finger to keyboard and composed:

It appears that
there are 3 entries for filenames of 21 characters, and 4 entries for
22 and 23 characters.

Sorry, that should have been "3 entries for filenames of 22
characters, and 4 entries for 23 and 24 characters".

- Franc Zabkar
--
Please remove one 'i' from my address when replying by email.

On Fri, 26 Oct 2007 12:30:21 GMT, teebo put finger to
keyboard and composed:

* Similarly, a FAT file system driver must not allow a directory
(a file that is actually a container for other files) to be larger
than 65,536 * 32 (2,097,152) bytes.

good to know that I never need more than 2MB to hold a directory list in
memory :-)

I'm thinking that this restriction could in extreme circumstances
result in file system corruption. For example, let's assume that the
directory is full (65,536 entries) and that the machine has been
booted in Win9x real DOS mode. What happens if you try to create an
additional file? AFAICS, DOS sees the existing LFNs as deleted files

nope! the LFNs are not marked as deleted files.
The first char in the filename-part in the LFN-posts are a letter
that describes number or LFN-posts the LFN-filename have. A is one LFN-post
B is two, C is three etc.

the special marking of the LFN-posts is instead by using fileattribute
DiskLabel-System-Hidden-ReadOnly

(each entry has a leading 0xE5 flag byte), so wouldn't DOS then
overwrite one of these LFNs? Or does Win9x DOS understand that these
entries are LFNs even though it doesn't support them?

dos without LFN-support understands them as "bad" and don't touch them.

(And if you run old scandisk type of diskutilities, like norton disk
doctor etc,
and the find these "errorus lines" and removes them, nothing more than the
long filenames is destroyed. the files can allways be used with the short
names)

Thanks for correcting my disinformation. The directory structure is
indeed as you describe. The following example is for a Win98 machine.

================================================== =================
C:\WIN98SErem a:\longfilename.txt

C:\WIN98SEdebug
-l 100 0 13 1
-d 100

140 42 74 00 78 00 74 00 00-00 FF FF 0F 00 D4 FF FF Bt.x.t..........
150 FF FF FF FF FF FF FF FF-FF FF 00 00 FF FF FF FF ................
160 01 6C 00 6F 00 6E 00 67-00 66 00 0F 00 D4 69 00 .l.o.n.g.f....i.
170 6C 00 65 00 6E 00 61 00-6D 00 00 00 65 00 2E 00 l.e.n.a.m...e...
180 4C 4F 4E 47 46 49 7E 31-54 58 54 20 00 5B 6F 35 LONGFI~1TXT .[o5
190 5B 37 5B 37 00 00 70 35-5B 37 00 00 00 00 00 00 [7[7..p5[7......

-q
================================================== =================

My confusion arose from the results of my experimentation on an XP
system (see the recent "UPPERCASE" thread). In that thread I observed
several deleted entries in the directory structure even though I had
not deleted any files. I had merely launched Notepad, typed "blah",
and then selected File - Save As to save my work as a text file.
After exiting Notepad and using Debug to view the directory entries, I
found entries corresponding to the newly created file, as well as
entries with 0xE5 flag bytes which denote a deleted file. Maybe this
behaviour is a consequence of an autosave function rather than some OS
quirk, ie maybe Notepad automatically saves the latest copy on exit???

- Franc Zabkar
--
Please remove one 'i' from my address when replying by email.

On Fri, 26 Oct 2007 02:05:23 -0400, 98 Guy put finger to
keyboard and composed:

I performed a series of serial file generations to see how many files
could be created in a subdirectory while changing the length of the
file name.

The following table shows the file-name length and the corresponding
number of files that could be created before this eror was generated:

The directory or file cannot be created

6.0 - 65,533
6.3 - 32,767
12.3 - 21,845
15.3 - 21,845
17.3 - 21,845
23.3 - 16,384
47.3 - 13,107
63.3 - 9,362
96.3 - 7,282

So in the first case, with a filename composed of 6 characters (and no
suffix) I was able to create 65,533 files. In the second case, the
filename was composed of 6-characters.3-characters, and the directory
would only hold 32,767 of those files. So as the filename length
increases, there is a decrease in the number of possible files.

From my testing I think the rule is as follows.

If we consider the SFN (32 bytes) to be one directory entry, then each
entry assigned to the LFN can store 13 characters. So to determine how
many entries a particular file requires, we count the characters in
the name, extension, and any periods, and divide the result by 13.

For example, a 96.3 filename would have 100 characters. Its LFN would
then require 8 entries (100 / 13 = 7.69), and its SFN would need one
more. Dividing 65535 by 9 gives 7281.7.

- Franc Zabkar
--
Please remove one 'i' from my address when replying by email.

teebo wrote:
| The FAT32 file system has a limit of 65k filename entries. Each DOS
| 8.3 filename will require one entry - long filenames require more.
|
| anyone know of a program that removes any unnecessary long filenames
| that are only there to preserve the Case of the filenames ?

Well... "START button, Run, WinFile".

It often is said that WinFile only understands SFNs. Maybe copy the
files in a folder to a new folder. Theoretically, you will get a folder
full of files with only SFNs. The source folder likely remains
unaltered. I'm not sure of any unexpected, additional, untoward
consequences!

WARNINGS:

(a) Don't get carried away & drop the LFNs in any system folder.
If they are used in the Registry-- there could be trouble!

(b) Even if not mentioned in the Registry, loosing LFNs may be
extremely uncomfortable, when you try to recall what the
files & folders are.

(c) Looks like it takes a little getting used to. Don't do anything
with it you might regret in the morning!

(d) WinFile doesn't seem to honor the following setting...

"START, Settings, Folder Options, View tab", & bolt "Show all
files"; may as well uncheck "Hide file extensions..." too.

Therefore, you may not see all files that exist in a folder.

| something to run at a directory hierarcy that removes all
| long filenames for the files that not have more than 8+3 chars
| (where only difference between the dos filename and the
| long FileName is small and BIG letters)

--
Thanks or Good Luck,
There may be humor in this post, and,
Naturally, you will not sue,
Should things get worse after this,
PCR

| something to run at a directory hierarcy that removes all
| long filenames for the files that not have more than 8+3 chars
| (where only difference between the dos filename and the
| long FileName is small and BIG letters)

What is the issue with file names with lower-case vs upper-case
letters?

| something to run at a directory hierarcy that removes all
| long filenames for the files that not have more than 8+3 chars
| (where only difference between the dos filename and the
| long FileName is small and BIG letters)

What is the issue with file names with lower-case vs upper-case
letters?

normal FAT-filenames (8+3 "dosfilenames") is allways saved in
uppercase only. (even though windows filemanager shows them as
lowercase with the first letter Uppercase)

So if you write any letter in the filename as lowercase when you
make the file, windows have to create a long-file-name too for it so
it can save the correct case somewhere.

Most people don't care about the casing of the filename though
(especially since windows shows all-uppercase filnames as lowercase
anyway...) so if you have lot of files that is not longer than
8+3 then you can save space in your directories if you could
run something that killed the long-file-name for just those files.
(I guess I could do that with some normal multirename-tool and set it
to rename all files with 8+3 chars or shorter, that don't have any
unicode or other special chars not allowed in dos, in a certain
directory with subdirectories, to be all UPPERCASE. But that will probably
be a whole bunch of clicks to make it do that, and I'm not sure it will
actually remove the long-file-name, maybe only make it upperase too?)

What you will loose is that filnames like "AbiSuite" will be
shown as "Abisuite" but I can live with that... :-)

(but the orginal poster had longer filenames so he have
no use for this anyway)

On Mon, 29 Oct 2007 03:22:10 GMT, teebo put finger to
keyboard and composed:

| something to run at a directory hierarcy that removes all
| long filenames for the files that not have more than 8+3 chars
| (where only difference between the dos filename and the
| long FileName is small and BIG letters)

What is the issue with file names with lower-case vs upper-case
letters?

normal FAT-filenames (8+3 "dosfilenames") is allways saved in
uppercase only. (even though windows filemanager shows them as
lowercase with the first letter Uppercase)

To force Explorer to "allow all uppercase names", go to View - Folder
Options - View and check the appropriate box.

So if you write any letter in the filename as lowercase when you
make the file, windows have to create a long-file-name too for it so
it can save the correct case somewhere.

Most people don't care about the casing of the filename though
(especially since windows shows all-uppercase filnames as lowercase
anyway...) so if you have lot of files that is not longer than
8+3 then you can save space in your directories if you could
run something that killed the long-file-name for just those files.
(I guess I could do that with some normal multirename-tool and set it
to rename all files with 8+3 chars or shorter, that don't have any
unicode or other special chars not allowed in dos, in a certain
directory with subdirectories, to be all UPPERCASE. But that will probably
be a whole bunch of clicks to make it do that, and I'm not sure it will
actually remove the long-file-name, maybe only make it upperase too?)

Yes, that's what happens. You need to make a copy instead.

C:\WIN98SEecho blah a:\MiXdCaSe.TxT
C:\WIN98SEren a:\MiXdCaSe.TxT MIXDCASE.TXT
C:\WIN98SEcopy a:\MIXDCASE.TXT a:\UPPRCASE.TXT
C:\WIN98SEdebug
-L 100 0 13 1
-D 100
100 41 4D 00 49 00 58 00 44-00 43 00 0F 00 0D 41 00 AM.I.X.D.C....A.
110 53 00 45 00 2E 00 54 00-58 00 00 00 54 00 00 00 S.E...T.X...T...
120 4D 49 58 44 43 41 53 45-54 58 54 20 00 3D C8 7D MIXDCASETXT .=.}
130 5D 37 5D 37 00 00 98 7E-5D 37 02 00 07 00 00 00 ]7]7...~]7......
140 55 50 50 52 43 41 53 45-54 58 54 20 00 21 9D 7E UPPRCASETXT .!.~
150 5D 37 5D 37 00 00 98 7E-5D 37 03 00 07 00 00 00 ]7]7...~]7......

What you will loose is that filnames like "AbiSuite" will be
shown as "Abisuite" but I can live with that... :-)

(but the orginal poster had longer filenames so he have
no use for this anyway)

- Franc Zabkar
--
Please remove one 'i' from my address when replying by email.

PCR wrote:

98 Guy wrote:
| | something to run at a directory hierarcy that removes all
| | long filenames for the files that not have more than 8+3 chars
| | (where only difference between the dos filename and the
| | long FileName is small and BIG letters)
|
| What is the issue with file names with lower-case vs upper-case
| letters?

As teebo may have said, I think someone may have wanted to get a maximum
number of files into one directory. To do that, one must eliminate LFNs
(Long File Names). In Win98, a LFN is created for a file that has
mixed-case letters-- even if there is no other reason to do so (I do
believe).


http://support.microsoft.com/kb/161982
http://support.microsoft.com/kb/130598

John

98 Guy wrote:
| | something to run at a directory hierarcy that removes all
| | long filenames for the files that not have more than 8+3 chars
| | (where only difference between the dos filename and the
| | long FileName is small and BIG letters)
|
| What is the issue with file names with lower-case vs upper-case
| letters?

As teebo may have said, I think someone may have wanted to get a maximum
number of files into one directory. To do that, one must eliminate LFNs
(Long File Names). In Win98, a LFN is created for a file that has
mixed-case letters-- even if there is no other reason to do so (I do
believe).


--
Thanks or Good Luck,
There may be humor in this post, and,
Naturally, you will not sue,
Should things get worse after this,
PCR