Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #176046 > unrolled thread

Re: OT?: FAT32(/16?) Question: Max. files in top level

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: OT?: FAT32(/16?) Question: Max. files in top level Xen <list@xenhideout.nl> - 2016-12-29 07:40 +0100
    Re: OT?: FAT32(/16?) Question: Max. files in top level <tomas@tuxteam.de> - 2016-12-31 10:40 +0100
      Re: OT?: FAT32(/16?) Question: Max. files in top level Gene Heskett <gheskett@shentel.net> - 2016-12-31 14:00 +0100
        Re: OT?: FAT32(/16?) Question: Max. files in top level Nicolas George <george@nsup.org> - 2016-12-31 14:10 +0100
          Re: OT?: FAT32(/16?) Question: Max. files in top level Gene Heskett <gheskett@shentel.net> - 2016-12-31 15:00 +0100
            Re: OT?: FAT32(/16?) Question: Max. files in top level Richard Owlett <rowlett@cloud85.net> - 2016-12-31 15:20 +0100
              Re: OT?: FAT32(/16?) Question: Max. files in top level Gene Heskett <gheskett@shentel.net> - 2016-12-31 16:40 +0100
            Re: OT?: FAT32(/16?) Question: Max. files in top level Nicolas George <george@nsup.org> - 2016-12-31 16:40 +0100
          Re: OT?: FAT32(/16?) Question: Max. files in top level <tomas@tuxteam.de> - 2016-12-31 16:30 +0100
      Re: OT?: FAT32(/16?) Question: Max. files in top level David Wright <deblis@lionunicorn.co.uk> - 2017-01-01 00:30 +0100
        Re: OT?: FAT32(/16?) Question: Max. files in top level <tomas@tuxteam.de> - 2017-01-01 13:20 +0100
          Re: OT?: FAT32(/16?) Question: Max. files in top level David Wright <deblis@lionunicorn.co.uk> - 2017-01-02 15:40 +0100
            Re: OT?: FAT32(/16?) Question: Max. files in top level Jude DaShiell <jdashiel@panix.com> - 2017-01-02 16:30 +0100
              Re: OT?: FAT32(/16?) Question: Max. files in top level Nicolas George <george@nsup.org> - 2017-01-02 16:30 +0100
                Re: OT?: FAT32(/16?) Question: Max. files in top level rhkramer@gmail.com - 2017-01-02 17:40 +0100
            Re: OT?: FAT32(/16?) Question: Max. files in top level <tomas@tuxteam.de> - 2017-01-02 20:50 +0100
      Re: OT?: FAT32(/16?) Question: Max. files in top level Xen <list@xenhideout.nl> - 2017-01-01 12:50 +0100

#176046 — Re: OT?: FAT32(/16?) Question: Max. files in top level

doark@mail.com schreef op 26-12-2016 3:41:

> I encountered this many times on windowz FAT32 in a non-root dir, but
> never on Linux. I suspect that it was/is one of their "Features". The
> said "Feature" still was there when using ntfs in XP if I remember
> correctly.

Perhaps it's just because Windows Explorer doesn't deal well with many 
files. Try to unpack some open source archive of some distributor that 
had to make their sources open, some 1G archive, and see how it goes. 
Not recommended :p.

Then when you've unpacked it, deleting it takes a few years as well. So 
imposing a filesystem limit may just have been a way to ensure that 
their user interface limit is not quickly reached, I don't know.

[toc] | [next] | [standalone]

#176147

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Dec 29, 2016 at 07:38:18AM +0100, Xen wrote:
> doark@mail.com schreef op 26-12-2016 3:41:
> 
> >I encountered this many times on windowz FAT32 in a non-root dir, but
> >never on Linux. I suspect that it was/is one of their "Features". The
> >said "Feature" still was there when using ntfs in XP if I remember
> >correctly.
> 
> Perhaps it's just because Windows Explorer doesn't deal well with
> many files. Try to unpack some open source archive of some
> distributor that had to make their sources open, some 1G archive,
> and see how it goes. Not recommended :p.
> 
> Then when you've unpacked it, deleting it takes a few years as well.
> So imposing a filesystem limit may just have been a way to ensure
> that their user interface limit is not quickly reached, I don't
> know.

Calculemus, as Leibnitz said. A bit of experimental informatics:

  dd if=/dev/zero of=dose bs=4096 count=64
  mkfs.vfat dose
  sudo mkfs.vfat dose
  sudo mount dose /mnt
  for i in $(seq 1 10000) ; do sudo touch /mnt/f.$(printf "%05d" $i) || echo "fail $i" ; done

The loop starts failing at i == 257 with "no space left on device"
(that's ENOSPC if I remember correctly). The "device" has still
reams of space left:

  tomas@rasputin:~$ df -h
  Filesystem                 Size  Used Avail Use% Mounted on
  [...]
  /dev/loop0                 238K     0  238K   0% /mnt

So 256 must be a limit on number of entries on the top level dir of
FATty file systems (or an implementation limit of Linux's version
of that, but guess whom I trust more to bust that badly).

Try this at home. Enjoy.

- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlhne8YACgkQBcgs9XrR2kYR9wCfdvZnntfbmoIbE1Q5LWZxLOh7
pF0An3BJbsyaWpwOEpWYNZ4xPfVQfgX2
=I4nJ
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#176152

On Saturday 31 December 2016 04:35:02 tomas@tuxteam.de wrote:

> On Thu, Dec 29, 2016 at 07:38:18AM +0100, Xen wrote:
> > doark@mail.com schreef op 26-12-2016 3:41:
> > >I encountered this many times on windowz FAT32 in a non-root dir,
> > > but never on Linux. I suspect that it was/is one of their
> > > "Features". The said "Feature" still was there when using ntfs in
> > > XP if I remember correctly.
> >
> > Perhaps it's just because Windows Explorer doesn't deal well with
> > many files. Try to unpack some open source archive of some
> > distributor that had to make their sources open, some 1G archive,
> > and see how it goes. Not recommended :p.
> >
> > Then when you've unpacked it, deleting it takes a few years as well.
> > So imposing a filesystem limit may just have been a way to ensure
> > that their user interface limit is not quickly reached, I don't
> > know.
>
> Calculemus, as Leibnitz said. A bit of experimental informatics:
>
>   dd if=/dev/zero of=dose bs=4096 count=64
>   mkfs.vfat dose
>   sudo mkfs.vfat dose
>   sudo mount dose /mnt
>   for i in $(seq 1 10000) ; do sudo touch /mnt/f.$(printf "%05d" $i)
> || echo "fail $i" ; done
>
> The loop starts failing at i == 257 with "no space left on device"
> (that's ENOSPC if I remember correctly). The "device" has still
> reams of space left:
>
>   tomas@rasputin:~$ df -h
>   Filesystem                 Size  Used Avail Use% Mounted on
>   [...]
>   /dev/loop0                 238K     0  238K   0% /mnt
>
> So 256 must be a limit on number of entries on the top level dir of
> FATty file systems (or an implementation limit of Linux's version
> of that, but guess whom I trust more to bust that badly).
>
> Try this at home. Enjoy.
>
> -- t
From personal experience decades ago, on a dos3.2 system, this is 
correct. But I can't testify about the newer, or the now several non-M$ 
versions of dos. I saw an announcement of yet another dos release just a 
couple weeks back. I assume its getting better The limitations of dos3.2 
drove me to find a better os, and os9 from microware, running on color 
computers was it. Its still alive, but called Nitros9 now since we've 
taken it apart, found several bugs and fixed them, and in the process 
made it about 2x faster without touching the cpu clock. Basically it is 
unix without the security overhead. But it does separate users, as it is 
multi-user, and multi-tasking.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#176153

[Multipart message — attachments visible in raw view] — view raw

Le primidi 11 nivôse, an CCXXV, Gene Heskett a écrit :
> From personal experience decades ago, on a dos3.2 system, this is 
> correct. But I can't testify about the newer, or the now several non-M$ 
> versions of dos. I saw an announcement of yet another dos release just a 
> couple weeks back. I assume its getting better

Think a little more about it: it is a limitation of the format, not the
operating system. If an operating system extends the format, it is no
longer compatible with the rest of the world, and then there is no
reason to use FAT at all.

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#176154

On Saturday 31 December 2016 08:01:15 Nicolas George wrote:

> Le primidi 11 nivôse, an CCXXV, Gene Heskett a écrit :
> > From personal experience decades ago, on a dos3.2 system, this is
> > correct. But I can't testify about the newer, or the now several
> > non-M$ versions of dos. I saw an announcement of yet another dos
> > release just a couple weeks back. I assume its getting better
>
> Think a little more about it: it is a limitation of the format, not
> the operating system. If an operating system extends the format, it is
> no longer compatible with the rest of the world, and then there is no
> reason to use FAT at all.
>
> Regards,

Perhaps Nicolas, but ATM I am having it shoved down my throat because I 
am playing with using a raspberrypi 3b to run a 1500 lb metal lathe, and 
for some reason the boot partition is dos, but mounted as /boot to armhf 
version of the debian jessie currently installed. Seems to me the r-pi 
bios needs fixed to boot from an ext4 file system.  But I'm not in 
charge of such, can't even blow the whistle. I'm still kicking the tires 
on the whole idea. Using the SPI bus at 32 megabaud to talk to the 
peripheral driver that runs the machine, I am finding that the noise 
radiated by the motor supplies, which are switchmode, running at 17-19 
kilohertz, have enough radiated noise to wreck a 32 megabuad 
communications bus 7 inches long.  So I am cabbaging filter parts from 
old computer psu's to see if I can quiet these power supplies down to a 
dull roar.  A film at 11 situation I fear.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#176156

On 12/31/2016 7:49 AM, Gene Heskett wrote:
> On Saturday 31 December 2016 08:01:15 Nicolas George wrote:
>
>> Le primidi 11 nivôse, an CCXXV, Gene Heskett a écrit :
>>>  From personal experience decades ago, on a dos3.2 system, this is
>>> correct. But I can't testify about the newer, or the now several
>>> non-M$ versions of dos. I saw an announcement of yet another dos
>>> release just a couple weeks back. I assume its getting better
>>
>> Think a little more about it: it is a limitation of the format, not
>> the operating system. If an operating system extends the format, it is
>> no longer compatible with the rest of the world, and then there is no
>> reason to use FAT at all.
>>
>> Regards,
>
> Perhaps Nicolas, but ATM I am having it shoved down my throat because I
> am playing with using a raspberrypi 3b to run a 1500 lb metal lathe, and
> for some reason the boot partition is dos, but mounted as /boot to armhf
> version of the debian jessie currently installed. Seems to me the r-pi
> bios needs fixed to boot from an ext4 file system.  But I'm not in
> charge of such, can't even blow the whistle. I'm still kicking the tires
> on the whole idea. Using the SPI bus at 32 megabaud to talk to the
> peripheral driver that runs the machine, I am finding that the noise
> radiated by the motor supplies, which are switchmode, running at 17-19
> kilohertz, have enough radiated noise to wreck a 32 megabuad
> communications bus 7 inches long.

Are you sure the problem is radiated noise.
Could it be a ground loop?
Prompted by vague recollections from decades past student days. YMMV


> So I am cabbaging filter parts from
> old computer psu's to see if I can quiet these power supplies down to a
> dull roar.  A film at 11 situation I fear.
>
> Cheers, Gene Heskett
>

[toc] | [prev] | [next] | [standalone]

#176161

On Saturday 31 December 2016 09:16:10 Richard Owlett wrote:

> On 12/31/2016 7:49 AM, Gene Heskett wrote:
> > On Saturday 31 December 2016 08:01:15 Nicolas George wrote:
> >> Le primidi 11 nivôse, an CCXXV, Gene Heskett a écrit :
> >>>  From personal experience decades ago, on a dos3.2 system, this is
> >>> correct. But I can't testify about the newer, or the now several
> >>> non-M$ versions of dos. I saw an announcement of yet another dos
> >>> release just a couple weeks back. I assume its getting better
> >>
> >> Think a little more about it: it is a limitation of the format, not
> >> the operating system. If an operating system extends the format, it
> >> is no longer compatible with the rest of the world, and then there
> >> is no reason to use FAT at all.
> >>
> >> Regards,
> >
> > Perhaps Nicolas, but ATM I am having it shoved down my throat
> > because I am playing with using a raspberrypi 3b to run a 1500 lb
> > metal lathe, and for some reason the boot partition is dos, but
> > mounted as /boot to armhf version of the debian jessie currently
> > installed. Seems to me the r-pi bios needs fixed to boot from an
> > ext4 file system.  But I'm not in charge of such, can't even blow
> > the whistle. I'm still kicking the tires on the whole idea. Using
> > the SPI bus at 32 megabaud to talk to the peripheral driver that
> > runs the machine, I am finding that the noise radiated by the motor
> > supplies, which are switchmode, running at 17-19 kilohertz, have
> > enough radiated noise to wreck a 32 megabuad communications bus 7
> > inches long.
>
> Are you sure the problem is radiated noise.
> Could it be a ground loop?
> Prompted by vague recollections from decades past student days. YMMV
>
Some of both I think, I can clip the ground lead of the probe onto the 
single point ground and find nearly 5 volts p-p, with sub 5ns rise and 
fall times 3" away on a 60 volt psu cover whose ground braid comes back 
to that bolt from the ground symbol on the psu terminal strip.  Corcom 
has a line filter they claim will reduce it at 30 mhz, by 50+ db. But it 
needs to be at the line terminals, meaning one per supply and theres 3.  
And the 30 mhz rating would be the most important since the spi bus is 
running at 32 megabaud. Nominally $15 each. I built a box with two of 
them from old computer psu's in it, but with output leads feet long, I 
can see thats a waste of time. I can't believe a switchmode supply, 
running at 17 kilohertz is making that much noise at 100+ megahertz. But 
it is. If I buy some of the corcom filters, I can do some judicious 
drilling and tapping and mount them right on the psu's with maybe 3/4" 
connecting leads. Solder to the spade lugs and drop a chunk of heat 
shrink over the input lugs.  If I can attenuate the noise to around 1.25 
v p-p, the bus seems bulletproof, but above 1.5 volts and its a coin 
flip.

The FCC has rules about this, but all this crap is Chinese made now, and 
no one AFAIK is checking what comes off the boat.

> > So I am cabbaging filter parts from
> > old computer psu's to see if I can quiet these power supplies down
> > to a dull roar.  A film at 11 situation I fear.
> >
> > Cheers, Gene Heskett


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[toc] | [prev] | [next] | [standalone]

#176162

[Multipart message — attachments visible in raw view] — view raw

Le primidi 11 nivôse, an CCXXV, Gene Heskett a écrit :
> > Think a little more about it: it is a limitation of the format, not
> > the operating system. If an operating system extends the format, it is
> > no longer compatible with the rest of the world, and then there is no
> > reason to use FAT at all.
> 
> Perhaps Nicolas, but ATM I am having it shoved down my throat because I 
> am playing with using a raspberrypi 3b to run a 1500 lb metal lathe
<snip>

I was not talking about your own personal project, I was talking about a
flaw in the reasoning you used to expect features from an operating in
general.

The fact is: The limit comes from the format, an newer operating system
can not improve it.

Additional note: You could have deduced that by yourself and spared
everybody's time.

[toc] | [prev] | [next] | [standalone]

#176160

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Dec 31, 2016 at 02:01:15PM +0100, Nicolas George wrote:
> Le primidi 11 nivôse, an CCXXV, Gene Heskett a écrit :
> > From personal experience decades ago, on a dos3.2 system, this is 
> > correct. But I can't testify about the newer, or the now several non-M$ 
> > versions of dos. I saw an announcement of yet another dos release just a 
> > couple weeks back. I assume its getting better
> 
> Think a little more about it: it is a limitation of the format, not the
> operating system. If an operating system extends the format, it is no
> longer compatible with the rest of the world, and then there is no
> reason to use FAT at all.

Yes, my hunch was also that it is a limit of the on-disk format (which
is, as you say, set in stone), although I expressed it in a pretty
round-about way, it seems :)

thanks
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlhnzbsACgkQBcgs9XrR2kaerACfWhl4J5oA1wPdA0DTi7cPM4dP
0TUAnROtKgKs9IG5rHWgD6dAk62+wCiF
=5k1I
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#176166

On Sat 31 Dec 2016 at 10:35:02 (+0100), tomas@tuxteam.de wrote:
> On Thu, Dec 29, 2016 at 07:38:18AM +0100, Xen wrote:
> > doark@mail.com schreef op 26-12-2016 3:41:
> > 
> > >I encountered this many times on windowz FAT32 in a non-root dir, but
> > >never on Linux. I suspect that it was/is one of their "Features". The
> > >said "Feature" still was there when using ntfs in XP if I remember
> > >correctly.
> > 
> > Perhaps it's just because Windows Explorer doesn't deal well with
> > many files. Try to unpack some open source archive of some
> > distributor that had to make their sources open, some 1G archive,
> > and see how it goes. Not recommended :p.
> > 
> > Then when you've unpacked it, deleting it takes a few years as well.
> > So imposing a filesystem limit may just have been a way to ensure
> > that their user interface limit is not quickly reached, I don't
> > know.
> 
> Calculemus, as Leibnitz said. A bit of experimental informatics:
> 
>   dd if=/dev/zero of=dose bs=4096 count=64
>   mkfs.vfat dose
>   sudo mkfs.vfat dose
>   sudo mount dose /mnt
>   for i in $(seq 1 10000) ; do sudo touch /mnt/f.$(printf "%05d" $i) || echo "fail $i" ; done
> 
> The loop starts failing at i == 257 with "no space left on device"
> (that's ENOSPC if I remember correctly). The "device" has still
> reams of space left:
> 
>   tomas@rasputin:~$ df -h
>   Filesystem                 Size  Used Avail Use% Mounted on
>   [...]
>   /dev/loop0                 238K     0  238K   0% /mnt
> 
> So 256 must be a limit on number of entries on the top level dir of
> FATty file systems (or an implementation limit of Linux's version
> of that, but guess whom I trust more to bust that badly).
> 
> Try this at home. Enjoy.

I did, but I had some difficulty replicating your result. I took a USB
stick and put a type c partition on it (mkfs.fat doesn't like writing
whole devices). Then:

#  mkfs.fat -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1
mkfs.fat 3.0.27 (2014-11-12)
Auto-selecting FAT32 for large filesystem
/dev/sdb1 has 62 heads and 62 sectors per track,
hidden sectors 0x0800;
logical sector size is 512,
using 0xf8 media descriptor, with 3891199 sectors;
drive number 0x80;
filesystem has 2 32-bit FATs and 8 sectors per cluster.
FAT size is 3793 sectors, and provides 485447 clusters.
There are 32 reserved sectors.
Volume ID is 20161231, volume label PETROLEUM2G.
# 

I now ran your one-liner and created over 20,000 empty files
at top level. So I down-sized with:

# mkfs.fat -F 16 -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1
mkfs.fat 3.0.27 (2014-11-12)
/dev/sdb1 has 62 heads and 62 sectors per track,
hidden sectors 0x0800;
logical sector size is 512,
using 0xf8 media descriptor, with 3891199 sectors;
drive number 0x80;
filesystem has 2 16-bit FATs and 64 sectors per cluster.
FAT size is 256 sectors, and provides 60789 clusters.
There is 1 reserved sector.
Root directory contains 2048 slots and uses 128 sectors.
Volume ID is 20161231, volume label PETROLEUM2G.
# 

and now at last I could exhaust the top level directory with
1023 entries. As expected, df -h gives
/dev/sdb1       1.9G     0  1.9G   0% /media/petroleum2g

Like Nicolas I couldn't unpick Gene's rant about the subject as he
doesn't seem to distinguish between DOS, the OS, and FAT, the
filesystem used by DOS. But I wouldn't expect anyone using DOS
seriously to need to have many files in top level directories.
If the OP is lucky, the photo frame might create a subdirectory
like most cameras/phones etc do, or, like some MP3 players, just
display everything it encounters regardless of the subdirectory
involved.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#176184

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Dec 31, 2016 at 05:14:50PM -0600, David Wright wrote:
> On Sat 31 Dec 2016 at 10:35:02 (+0100), tomas@tuxteam.de wrote:

[...]

> > Calculemus, as Leibnitz said. A bit of experimental informatics:

Mpfh. Seems I was a bit boisterous here :)

[...]

> > Try this at home. Enjoy.
> 
> I did, but I had some difficulty replicating your result. I took a USB
> stick and put a type c partition on it (mkfs.fat doesn't like writing
> whole devices). Then:

Thanks for trying to replicate. Didn't know about the device thing. Will
try when I've an USB stick which isn't my main backup :)

> #  mkfs.fat -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1
> mkfs.fat 3.0.27 (2014-11-12)
> Auto-selecting FAT32 for large filesystem

OK, seems FAT32 has bigger limits (and it's actually selected when
the medium is big enough).

> /dev/sdb1 has 62 heads and 62 sectors per track,
> hidden sectors 0x0800;
> logical sector size is 512,
> using 0xf8 media descriptor, with 3891199 sectors;
> drive number 0x80;
> filesystem has 2 32-bit FATs and 8 sectors per cluster.
> FAT size is 3793 sectors, and provides 485447 clusters.
> There are 32 reserved sectors.
> Volume ID is 20161231, volume label PETROLEUM2G.
> # 
> 
> I now ran your one-liner and created over 20,000 empty files
> at top level. So I down-sized with:
> 
> # mkfs.fat -F 16 -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1
> mkfs.fat 3.0.27 (2014-11-12)
> /dev/sdb1 has 62 heads and 62 sectors per track,
> hidden sectors 0x0800;
> logical sector size is 512,
> using 0xf8 media descriptor, with 3891199 sectors;
> drive number 0x80;
> filesystem has 2 16-bit FATs and 64 sectors per cluster.
> FAT size is 256 sectors, and provides 60789 clusters.
> There is 1 reserved sector.
> Root directory contains 2048 slots and uses 128 sectors.
> Volume ID is 20161231, volume label PETROLEUM2G.
> # 
> 
> and now at last I could exhaust the top level directory with
> 1023 entries. As expected, df -h gives
> /dev/sdb1       1.9G     0  1.9G   0% /media/petroleum2g

This is very interesting! Seems FAT32 has a limit of 1024 entries
in a top-level dir. OK, if all else fails, read the instructions :)

Wikipedia [1] has a very informative page: according to that, on
FAT12 and FAT16, the root directory is statically allocated, and
its maximum size is written somewhere in the file system header.
There are two bytes available for that (and the number encoded
there is "number of entries"), so that in theory there could be
up to 65K. Lo and behold, mkfs.vfat *has* a command line parameter
(-r) to set the number of root directory entries! Of course, I'd
expect any overall limit on entries per directory to apply here,
if there is any (couldn't find that out in that short time).

This all doesn't apply to FAT32 (as you had), where the root
directory is an ordinary directory. Why you got this 1024 limit
is thus still a mystery for me.

> Like Nicolas I couldn't unpick Gene's rant about the subject as he
> doesn't seem to distinguish between DOS, the OS, and FAT, the
> filesystem used by DOS. But I wouldn't expect anyone using DOS
> seriously to need to have many files in top level directories.
> If the OP is lucky, the photo frame might create a subdirectory
> like most cameras/phones etc do, or, like some MP3 players, just
> display everything it encounters regardless of the subdirectory
> involved.

Yeah, exactly. The only reason for FAT to exist these days is
compatibility, and then it makes sense to be especially conservative
(e.g. there's no limit to directory nesting depth, but most
implementations die on a path which is too long).

Anyway, thanks for digging further!

regards

[1] https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system

- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlho8s8ACgkQBcgs9XrR2kZlKwCdHLzpxBlW0Eju+lnt+3muUMce
EfQAn2QsVopodC3ZC7LNivJ5jXR8Ue2X
=W1Qy
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#176254

On Sun 01 Jan 2017 at 13:15:11 (+0100), tomas@tuxteam.de wrote:
> On Sat, Dec 31, 2016 at 05:14:50PM -0600, David Wright wrote:
> > On Sat 31 Dec 2016 at 10:35:02 (+0100), tomas@tuxteam.de wrote:
> 
> [...]
> 
> > > Calculemus, as Leibnitz said. A bit of experimental informatics:
> 
> Mpfh. Seems I was a bit boisterous here :)
> 
> [...]
> 
> > > Try this at home. Enjoy.
> > 
> > I did, but I had some difficulty replicating your result. I took a USB
> > stick and put a type c partition on it (mkfs.fat doesn't like writing
> > whole devices). Then:
> 
> Thanks for trying to replicate. Didn't know about the device thing. Will
> try when I've an USB stick which isn't my main backup :)
> 
> > #  mkfs.fat -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1
> > mkfs.fat 3.0.27 (2014-11-12)
> > Auto-selecting FAT32 for large filesystem
> 
> OK, seems FAT32 has bigger limits (and it's actually selected when
> the medium is big enough).
> 
> > /dev/sdb1 has 62 heads and 62 sectors per track,
> > hidden sectors 0x0800;
> > logical sector size is 512,
> > using 0xf8 media descriptor, with 3891199 sectors;
> > drive number 0x80;
> > filesystem has 2 32-bit FATs and 8 sectors per cluster.
> > FAT size is 3793 sectors, and provides 485447 clusters.
> > There are 32 reserved sectors.
> > Volume ID is 20161231, volume label PETROLEUM2G.
> > # 
> > 
> > I now ran your one-liner and created over 20,000 empty files
> > at top level. So I down-sized with:
> > 
> > # mkfs.fat -F 16 -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1

This was intended to create a FAT16 filesystem...

> > mkfs.fat 3.0.27 (2014-11-12)
> > /dev/sdb1 has 62 heads and 62 sectors per track,
> > hidden sectors 0x0800;
> > logical sector size is 512,
> > using 0xf8 media descriptor, with 3891199 sectors;
> > drive number 0x80;
> > filesystem has 2 16-bit FATs and 64 sectors per cluster.
> > FAT size is 256 sectors, and provides 60789 clusters.
> > There is 1 reserved sector.
> > Root directory contains 2048 slots and uses 128 sectors.
> > Volume ID is 20161231, volume label PETROLEUM2G.
> > # 
> > 
> > and now at last I could exhaust the top level directory with
> > 1023 entries. As expected, df -h gives
> > /dev/sdb1       1.9G     0  1.9G   0% /media/petroleum2g
> 
> This is very interesting! Seems FAT32 has a limit of 1024 entries
> in a top-level dir. OK, if all else fails, read the instructions :)

... so I think you mean FAT16 here.

> Wikipedia [1] has a very informative page: according to that, on
> FAT12 and FAT16, the root directory is statically allocated, and
> its maximum size is written somewhere in the file system header.
> There are two bytes available for that (and the number encoded
> there is "number of entries"), so that in theory there could be
> up to 65K. Lo and behold, mkfs.vfat *has* a command line parameter
> (-r) to set the number of root directory entries! Of course, I'd
> expect any overall limit on entries per directory to apply here,
> if there is any (couldn't find that out in that short time).
> 
> This all doesn't apply to FAT32 (as you had), where the root
> directory is an ordinary directory. Why you got this 1024 limit
> is thus still a mystery for me.

So no mystery, perhaps.

> > Like Nicolas I couldn't unpick Gene's rant about the subject as he
> > doesn't seem to distinguish between DOS, the OS, and FAT, the
> > filesystem used by DOS. But I wouldn't expect anyone using DOS
> > seriously to need to have many files in top level directories.
> > If the OP is lucky, the photo frame might create a subdirectory
> > like most cameras/phones etc do, or, like some MP3 players, just
> > display everything it encounters regardless of the subdirectory
> > involved.
> 
> Yeah, exactly. The only reason for FAT to exist these days is
> compatibility, and then it makes sense to be especially conservative
> (e.g. there's no limit to directory nesting depth, but most
> implementations die on a path which is too long).
> 
> Anyway, thanks for digging further!
> 
> regards
> 
> [1] https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#176257

msdos 6.22 which was fat16 had a limit of 112 files in top level 
directory.  Once I tried putting more than that on a floppy disk and 
couldn't figure why no more would fit until I found this out.  I don't 
know if the limit got expanded for fat32 and msdos 7 but since Microsoft 
has been devoted to tearing stuff out of dos since msdos 6.22 I suspect 
that limit was not expanded.  In those days usb did not even exist so 
this information comes from pre-usb technology for me.  Hope this helps.

On Mon, 2 Jan 2017, David Wright wrote:

> Date: Mon, 2 Jan 2017 09:32:33
> From: David Wright <deblis@lionunicorn.co.uk>
> Reply-To: debian-user@lists.debian.org
> To: debian-user@lists.debian.org
> Subject: Re: OT?: FAT32(/16?) Question: Max. files in top level
> Resent-Date: Mon,  2 Jan 2017 14:37:55 +0000 (UTC)
> Resent-From: debian-user@lists.debian.org
> 
> On Sun 01 Jan 2017 at 13:15:11 (+0100), tomas@tuxteam.de wrote:
>> On Sat, Dec 31, 2016 at 05:14:50PM -0600, David Wright wrote:
>>> On Sat 31 Dec 2016 at 10:35:02 (+0100), tomas@tuxteam.de wrote:
>>
>> [...]
>>
>>>> Calculemus, as Leibnitz said. A bit of experimental informatics:
>>
>> Mpfh. Seems I was a bit boisterous here :)
>>
>> [...]
>>
>>>> Try this at home. Enjoy.
>>>
>>> I did, but I had some difficulty replicating your result. I took a USB
>>> stick and put a type c partition on it (mkfs.fat doesn't like writing
>>> whole devices). Then:
>>
>> Thanks for trying to replicate. Didn't know about the device thing. Will
>> try when I've an USB stick which isn't my main backup :)
>>
>>> #  mkfs.fat -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1
>>> mkfs.fat 3.0.27 (2014-11-12)
>>> Auto-selecting FAT32 for large filesystem
>>
>> OK, seems FAT32 has bigger limits (and it's actually selected when
>> the medium is big enough).
>>
>>> /dev/sdb1 has 62 heads and 62 sectors per track,
>>> hidden sectors 0x0800;
>>> logical sector size is 512,
>>> using 0xf8 media descriptor, with 3891199 sectors;
>>> drive number 0x80;
>>> filesystem has 2 32-bit FATs and 8 sectors per cluster.
>>> FAT size is 3793 sectors, and provides 485447 clusters.
>>> There are 32 reserved sectors.
>>> Volume ID is 20161231, volume label PETROLEUM2G.
>>> #
>>>
>>> I now ran your one-liner and created over 20,000 empty files
>>> at top level. So I down-sized with:
>>>
>>> # mkfs.fat -F 16 -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1
>
> This was intended to create a FAT16 filesystem...
>
>>> mkfs.fat 3.0.27 (2014-11-12)
>>> /dev/sdb1 has 62 heads and 62 sectors per track,
>>> hidden sectors 0x0800;
>>> logical sector size is 512,
>>> using 0xf8 media descriptor, with 3891199 sectors;
>>> drive number 0x80;
>>> filesystem has 2 16-bit FATs and 64 sectors per cluster.
>>> FAT size is 256 sectors, and provides 60789 clusters.
>>> There is 1 reserved sector.
>>> Root directory contains 2048 slots and uses 128 sectors.
>>> Volume ID is 20161231, volume label PETROLEUM2G.
>>> #
>>>
>>> and now at last I could exhaust the top level directory with
>>> 1023 entries. As expected, df -h gives
>>> /dev/sdb1       1.9G     0  1.9G   0% /media/petroleum2g
>>
>> This is very interesting! Seems FAT32 has a limit of 1024 entries
>> in a top-level dir. OK, if all else fails, read the instructions :)
>
> ... so I think you mean FAT16 here.
>
>> Wikipedia [1] has a very informative page: according to that, on
>> FAT12 and FAT16, the root directory is statically allocated, and
>> its maximum size is written somewhere in the file system header.
>> There are two bytes available for that (and the number encoded
>> there is "number of entries"), so that in theory there could be
>> up to 65K. Lo and behold, mkfs.vfat *has* a command line parameter
>> (-r) to set the number of root directory entries! Of course, I'd
>> expect any overall limit on entries per directory to apply here,
>> if there is any (couldn't find that out in that short time).
>>
>> This all doesn't apply to FAT32 (as you had), where the root
>> directory is an ordinary directory. Why you got this 1024 limit
>> is thus still a mystery for me.
>
> So no mystery, perhaps.
>
>>> Like Nicolas I couldn't unpick Gene's rant about the subject as he
>>> doesn't seem to distinguish between DOS, the OS, and FAT, the
>>> filesystem used by DOS. But I wouldn't expect anyone using DOS
>>> seriously to need to have many files in top level directories.
>>> If the OP is lucky, the photo frame might create a subdirectory
>>> like most cameras/phones etc do, or, like some MP3 players, just
>>> display everything it encounters regardless of the subdirectory
>>> involved.
>>
>> Yeah, exactly. The only reason for FAT to exist these days is
>> compatibility, and then it makes sense to be especially conservative
>> (e.g. there's no limit to directory nesting depth, but most
>> implementations die on a path which is too long).
>>
>> Anyway, thanks for digging further!
>>
>> regards
>>
>> [1] https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system
>
> Cheers,
> David.
>
>

-- 

[toc] | [prev] | [next] | [standalone]

#176258

[Multipart message — attachments visible in raw view] — view raw

Le tridi 13 nivôse, an CCXXV, Jude DaShiell a écrit :
> msdos 6.22 which was fat16 had a limit of 112 files in top level directory.
> Once I tried putting more than that on a floppy disk and couldn't figure why
> no more would fit until I found this out.  I don't know if the limit got
> expanded for fat32 and msdos 7

Will nobody read the darn thread before posting half-wrong statements?
And possibly check their own facts against reliable sources?

The limit does not come from the OS, it is coded in the "superblock" of
the filesystem. It can be configured when creating the filesystem. The
default is indeed sometimes 112.

Also, MS-DOS 6.22 "is" not FAT16, that does not mean anything. FAT12,
FAT16 and FAT32 are three variants of the filesystems with different
compromises between wasted storage per file and wasted storage globally.
The tool that creates the filesystem will normally choose the best one
for a typical use by default. And of course, MS-DOS < 7 did not support
FAT32.

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#176263

This appears to be a never ending thread. ;-)

I started it, an, for the record, I'm good (I've got a satisfactory answer--it 
seems I can put photos in subdirectories and the photo frame will work its way 
through the subdirectories).  

But, if you (all) are having fun, carry on ;-)


On Monday, January 02, 2017 10:29:16 AM Nicolas George wrote:
> Le tridi 13 nivôse, an CCXXV, Jude DaShiell a écrit :
> > msdos 6.22 which was fat16 had a limit of 112 files in top level
> > directory. Once I tried putting more than that on a floppy disk and
> > couldn't figure why no more would fit until I found this out.  I don't
> > know if the limit got expanded for fat32 and msdos 7
> 
> Will nobody read the darn thread before posting half-wrong statements?
> And possibly check their own facts against reliable sources?
> 
> The limit does not come from the OS, it is coded in the "superblock" of
> the filesystem. It can be configured when creating the filesystem. The
> default is indeed sometimes 112.
> 
> Also, MS-DOS 6.22 "is" not FAT16, that does not mean anything. FAT12,
> FAT16 and FAT32 are three variants of the filesystems with different
> compromises between wasted storage per file and wasted storage globally.
> The tool that creates the filesystem will normally choose the best one
> for a typical use by default. And of course, MS-DOS < 7 did not support
> FAT32.
> 
> Regards,

[toc] | [prev] | [next] | [standalone]

#176271

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, Jan 02, 2017 at 08:32:33AM -0600, David Wright wrote:
> On Sun 01 Jan 2017 at 13:15:11 (+0100), tomas@tuxteam.de wrote:
> > On Sat, Dec 31, 2016 at 05:14:50PM -0600, David Wright wrote:

Yeah, sorry. I was set off by this...

> > > Auto-selecting FAT32 for large filesystem

...and overlooked this:

> > > # mkfs.fat -F 16 -i 20161231 -n PETROLEUM2G -r 2000 -v /dev/sdb1
> 
> This was intended to create a FAT16 filesystem...

you are right.

[...]

> > This is very interesting! Seems FAT32 has a limit of 1024 entries
> > in a top-level dir. OK, if all else fails, read the instructions :)
> 
> ... so I think you mean FAT16 here.

indeed.

[...]

> So no mystery, perhaps.

Again, indeed. Seems the default in this case was set at 1024 entries
for the root dir.

Thanks
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlhqrkEACgkQBcgs9XrR2kbvAwCeKqNr1ZdPWpmQ1M+aobGQp4dV
sQ8AnAgxQNwYPNTYkiuPVywhC5n4gLS4
=RcPw
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#176176

tomas@tuxteam.de schreef op 31-12-2016 10:35:
> On Thu, Dec 29, 2016 at 07:38:18AM +0100, Xen wrote:
>> doark@mail.com schreef op 26-12-2016 3:41:
>> 
>> >I encountered this many times on windowz FAT32 in a non-root dir, but
>> >never on Linux. I suspect that it was/is one of their "Features". The
>> >said "Feature" still was there when using ntfs in XP if I remember
>> >correctly.
>> 
>> Perhaps it's just because Windows Explorer doesn't deal well with
>> many files. Try to unpack some open source archive of some
>> distributor that had to make their sources open, some 1G archive,
>> and see how it goes. Not recommended :p.
>> 
>> Then when you've unpacked it, deleting it takes a few years as well.
>> So imposing a filesystem limit may just have been a way to ensure
>> that their user interface limit is not quickly reached, I don't
>> know.
> 
> Calculemus, as Leibnitz said. A bit of experimental informatics:
> 
>   dd if=/dev/zero of=dose bs=4096 count=64
>   mkfs.vfat dose
>   sudo mkfs.vfat dose
>   sudo mount dose /mnt
>   for i in $(seq 1 10000) ; do sudo touch /mnt/f.$(printf "%05d" $i)
> || echo "fail $i" ; done
> 
> The loop starts failing at i == 257 with "no space left on device"
> (that's ENOSPC if I remember correctly). The "device" has still
> reams of space left:
> 
>   tomas@rasputin:~$ df -h
>   Filesystem                 Size  Used Avail Use% Mounted on
>   [...]
>   /dev/loop0                 238K     0  238K   0% /mnt
> 
> So 256 must be a limit on number of entries on the top level dir of
> FATty file systems (or an implementation limit of Linux's version
> of that, but guess whom I trust more to bust that badly).
> 
> Try this at home. Enjoy.

That person spoke about a non-root dir which is what I responded to.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web