|
|
|
|
|
|
| Author |
Message |
50295@web.de
*nix forums beginner
Joined: 09 Feb 2005
Posts: 32
|
 Posted: Sat Feb 25, 2006 11:36 pm Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Gordon Burditt wrote:
| Quote: | (2) Why cant direc tories be hard-linked to? After all, this is what
the OS does when it automatically creates the entries "." and ".."
In UNIX V7, you could. mkdir() as a system call didn't exist.
And you could make an awful mess where, for example:
/
/.
/./.
/..
/../.
/./..
referred to 6 different directories and the last 5 were reachable
*ONLY* by the paths given. Other than neat ways for viruses
to hide stuff, I don't see what use hard-linking to directories has.
And where .. in such a directory is supposed to point is problematical.
|
I'm sorry but I dont't get the point you're trying to pass across. |
|
| Back to top |
|
 |
Gordon Burditt
*nix forums Guru
Joined: 02 Mar 2005
Posts: 773
|
| Posted: Sun Feb 26, 2006 1:22 am Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
| Quote: | (2) Why cant direc tories be hard-linked to? After all, this is what
the OS does when it automatically creates the entries "." and ".."
In UNIX V7, you could. mkdir() as a system call didn't exist.
And you could make an awful mess where, for example:
/
/.
/./.
/..
/../.
/./..
referred to 6 different directories and the last 5 were reachable
*ONLY* by the paths given. Other than neat ways for viruses
to hide stuff, I don't see what use hard-linking to directories has.
And where .. in such a directory is supposed to point is problematical.
I'm sorry but I dont't get the point you're trying to pass across.
|
Hard linking directories can make an AWFUL mess of a filesystem
and serves no useful purpose. That's a good reason not
to allow it.
Gordon L. Burditt |
|
| Back to top |
|
|
50295@web.de
*nix forums beginner
Joined: 09 Feb 2005
Posts: 32
|
| Posted: Sun Feb 26, 2006 1:58 am Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Gordon Burditt wrote:
| Quote: |
Hard linking directories can make an AWFUL mess of a filesystem
and serves no useful purpose. That's a good reason not
to allow it.
|
I suppose soft links to directories are less messy then? Right?
(scratches head) |
|
| Back to top |
|
|
Maxim S. Shatskih
*nix forums addict
Joined: 02 Apr 2005
Posts: 55
|
| Posted: Sun Feb 26, 2006 2:07 am Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
| Quote: | I suppose soft links to directories are less messy then? Right?
|
Yes, the recurser tools just ignore softlinks. Also softlinks have no problems
going cross-volume.
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com |
|
| Back to top |
|
|
toby
*nix forums addict
Joined: 01 Jul 2005
Posts: 87
|
| Posted: Sun Feb 26, 2006 4:22 am Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Olumide wrote:
| Quote: | Gordon Burditt wrote:
Hard linking directories can make an AWFUL mess of a filesystem
and serves no useful purpose. That's a good reason not
to allow it.
I suppose soft links to directories are less messy then? Right?
(scratches head)
|
It is conceptually different, although the effect may be the same (if
the target exists). Two hardlinked entities in a filesystem are
indistinguishable, while symlinks have a directedness. Analogies
include duplicate A records versus CNAMEs in DNS, or HTTP Redirect
versus ServerAliases... Each mechanism has its distinct purposes. |
|
| Back to top |
|
|
Logan Shaw
*nix forums Guru
Joined: 21 Feb 2005
Posts: 474
|
| Posted: Sun Feb 26, 2006 6:45 pm Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Jordan Abel wrote:
| Quote: | On 2006-02-24, Gordon Burditt <gordonb.l20vx@burditt.org> wrote:
fd = open("big", O_WRONLY | O_CREAT, 0666);
| O_LARGE
Where is that supposed to be defined? I don't see it in
any source file for FreeBSD (other than as part of some
symbols of the form *_TOO_LARGE, mostly in openssl ).
lseek(fd, offset, 0);
man 2 llseek
No such manual page.
Where can I buy storage, cheap, that needs more than 64 bits for
the length of the file? And what do I need it for? Archiving
the entire contents of Google hourly?
I bet he uses Linux. Linux has traditionally maintained two separate and
parallel APIs, one in which off_t is a 'long' (32 on a 32-bit system),
the other in which it's 64 bits. llseek is one of the underlying system
calls for the latter mode, and it is also sometimes misused (where
lseek64 _should_ be used) to be able to access files in "large file"
mode in the former. O_LARGEFILE (misspelled above as O_LARGE) is a bit
used internally by open64() which is also sometimes misused in the same
way.
|
Just a data point: on Solaris 8 (and presumably later versions as well
since Sun is anal-retentive about keeping interface compatibility at
both the binary and source levels[1]), the documentation says that
calling open() with O_LARGEFILE is equivalent to calling open64(),
which to me indicates that using O_LARGEFILE with open() is kosher.
Solaris also maintains a parallel set of APIs, so that 32-bit
applications can use either 32-bit or 64-bit file offsets, so Linux
isn't unique in that regard. (I don't even think Linux was first,
but I can't remember.)
- Logan
[1] which, by the way, is a good thing in many cases |
|
| Back to top |
|
|
Logan Shaw
*nix forums Guru
Joined: 21 Feb 2005
Posts: 474
|
| Posted: Sun Feb 26, 2006 6:58 pm Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Olumide wrote:
| Quote: | Whiel we're on the subject of file systems, I would like to ask about
hard links.
(1) I know they can only refer to data on the same
volume/filestore/partition. The question is why? My reasoning (and I
may have read this somewhere a long time ago) is that because each
volume/filestore/partition has a list of inode numbered from 0 or 1 to
whatever, and linking a target name merely creates a new directory
entry that points to the same inode
|
You are on the right track. The correct answer is that there is no
such thing as a "hard link". Or to be more precise, if you do this:
date > a
ln a b
then it is not correct to say that "a" is a file and "b" is a hard
link to it. Both "a" and "b" are on equal standing and there is
no distinction between them. Assuming "a" and "b" do not already
exist, the above two commands are exactly equivalent to this:
date > b
ln b a
Once you have done either of the two sequences of commands (that is,
create "a" first, then link "b" to it, or vice versa), there is no
way to tell which sequence you chose, because there is no difference
in the outcome. Both "a" and "b" will have the same modification
date, etc., etc., because they have the same i-node.
A simple way to think of this is that, in Unix, files don't have
names. Instead, in Unix directories have names *for* files. This
is different than how it works on many other systems (such as the
DOS FAT filesystem), where files have names as part of their own
structure.
Anyway, the point here is the reason a "hard link" can't be on
a different filesystem is the same reason a regular file can't be
on a different filesystem, because they are the same thing anyway.
(And as it turns out, the reason for this is that a directory does
not have the ability to refer to a file on a different filesystem.
Which does make sense if you think about it.)
| Quote: | (2) Why cant direc tories be hard-linked to? After all, this is what
the OS does when it automatically creates the entries "." and ".."
|
There is no technical limitation preventing this from being easy
enough to implement. The problem is, it gets very confusing very
quickly. Consider this:
mkdir foo
mkdir bar
mkdir foo/abc
ln foo/abc bar/abc
cd foo/abc
Now, after that sequence of commands, what should ".." (inside foo/abc)
refer to? ".." refers to the parent directory, right? Well, what is
abc's parent? Is it foo or is it bar? It's both! abc has two parents,
and ".." can only point to one. Which one should you choose? This
just gets ugly really quickly.
- Logan |
|
| Back to top |
|
|
50295@web.de
*nix forums beginner
Joined: 09 Feb 2005
Posts: 32
|
| Posted: Sun Feb 26, 2006 7:35 pm Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Olumide wrote:
| Quote: | While we're on the subject of file systems, I would like to ask about
hard links.
(1) I know they can only refer to data on the same
volume/filestore/partition. ...
|
Just to confirm, is this correct. Are *NIX directory entries limited to
inodes on the same volume OR partition OR filestore? (I'm not too sure
about the partition bit. I guess directory entries will be limited, if
each partition numbered its inodes from 0 ...) |
|
| Back to top |
|
|
Jordan Abel
*nix forums Guru
Joined: 25 Oct 2005
Posts: 1366
|
| Posted: Sun Feb 26, 2006 8:01 pm Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
On 2006-02-26, Logan Shaw <lshaw-usenet@austin.rr.com> wrote:
| Quote: | Jordan Abel wrote:
On 2006-02-24, Gordon Burditt <gordonb.l20vx@burditt.org> wrote:
fd = open("big", O_WRONLY | O_CREAT, 0666);
| O_LARGE
Where is that supposed to be defined? I don't see it in
any source file for FreeBSD (other than as part of some
symbols of the form *_TOO_LARGE, mostly in openssl ).
lseek(fd, offset, 0);
man 2 llseek
No such manual page.
Where can I buy storage, cheap, that needs more than 64 bits for
the length of the file? And what do I need it for? Archiving
the entire contents of Google hourly?
I bet he uses Linux. Linux has traditionally maintained two separate and
parallel APIs, one in which off_t is a 'long' (32 on a 32-bit system),
the other in which it's 64 bits. llseek is one of the underlying system
calls for the latter mode, and it is also sometimes misused (where
lseek64 _should_ be used) to be able to access files in "large file"
mode in the former. O_LARGEFILE (misspelled above as O_LARGE) is a bit
used internally by open64() which is also sometimes misused in the same
way.
Just a data point: on Solaris 8 (and presumably later versions as well
since Sun is anal-retentive about keeping interface compatibility at
both the binary and source levels[1]), the documentation says that
calling open() with O_LARGEFILE is equivalent to calling open64(),
which to me indicates that using O_LARGEFILE with open() is kosher.
Solaris also maintains a parallel set of APIs, so that 32-bit
applications can use either 32-bit or 64-bit file offsets, so Linux
isn't unique in that regard. (I don't even think Linux was first,
but I can't remember.)
- Logan
[1] which, by the way, is a good thing in many cases
|
I won't disagree that binary compatibility is good - but off_t should
have been 64-bit to start with. On FreeBSD, off_t has NEVER been less
than 64 bits. |
|
| Back to top |
|
|
Logan Shaw
*nix forums Guru
Joined: 21 Feb 2005
Posts: 474
|
| Posted: Mon Feb 27, 2006 12:16 am Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Jordan Abel wrote:
| Quote: | I won't disagree that binary compatibility is good - but off_t should
have been 64-bit to start with. On FreeBSD, off_t has NEVER been less
than 64 bits.
|
Sure, that would have been nice, but aren't we talking about an API that
has been around since 10 megabytes was a really luxurious amount of
secondary storage? :-)
- Logan |
|
| Back to top |
|
|
John S. Dyson
*nix forums beginner
Joined: 16 Mar 2005
Posts: 9
|
| Posted: Mon Feb 27, 2006 3:05 am Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
In article <slrne042du.1qfk.random832@random.yi.org>,
Jordan Abel <random832@gmail.com> writes:
| Quote: | On 2006-02-26, Logan Shaw <lshaw-usenet@austin.rr.com> wrote:
Jordan Abel wrote:
On 2006-02-24, Gordon Burditt <gordonb.l20vx@burditt.org> wrote:
fd = open("big", O_WRONLY | O_CREAT, 0666);
| O_LARGE
Where is that supposed to be defined? I don't see it in
any source file for FreeBSD (other than as part of some
symbols of the form *_TOO_LARGE, mostly in openssl ).
lseek(fd, offset, 0);
man 2 llseek
No such manual page.
Where can I buy storage, cheap, that needs more than 64 bits for
the length of the file? And what do I need it for? Archiving
the entire contents of Google hourly?
I bet he uses Linux. Linux has traditionally maintained two separate and
parallel APIs, one in which off_t is a 'long' (32 on a 32-bit system),
the other in which it's 64 bits. llseek is one of the underlying system
calls for the latter mode, and it is also sometimes misused (where
lseek64 _should_ be used) to be able to access files in "large file"
mode in the former. O_LARGEFILE (misspelled above as O_LARGE) is a bit
used internally by open64() which is also sometimes misused in the same
way.
Just a data point: on Solaris 8 (and presumably later versions as well
since Sun is anal-retentive about keeping interface compatibility at
both the binary and source levels[1]), the documentation says that
calling open() with O_LARGEFILE is equivalent to calling open64(),
which to me indicates that using O_LARGEFILE with open() is kosher.
Solaris also maintains a parallel set of APIs, so that 32-bit
applications can use either 32-bit or 64-bit file offsets, so Linux
isn't unique in that regard. (I don't even think Linux was first,
but I can't remember.)
- Logan
[1] which, by the way, is a good thing in many cases
I won't disagree that binary compatibility is good - but off_t should
have been 64-bit to start with. On FreeBSD, off_t has NEVER been less
than 64 bits.
Actually, the API for FreeBSD V1.X had 32 bit lseek arguments. FreeBSD V2.X |
had the proper (64bit) offset api, but wasn't fully implemented until about
V2.2.X... (I know, I wrote a lot of the lower level infrastructure.)
John |
|
| Back to top |
|
|
Jordan Abel
*nix forums Guru
Joined: 25 Oct 2005
Posts: 1366
|
| Posted: Mon Feb 27, 2006 5:11 am Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
On 2006-02-27, John S. Dyson <toor@iquest.net> wrote:
| Quote: | In article <slrne042du.1qfk.random832@random.yi.org>,
Jordan Abel <random832@gmail.com> writes:
On 2006-02-26, Logan Shaw <lshaw-usenet@austin.rr.com> wrote:
Jordan Abel wrote:
On 2006-02-24, Gordon Burditt <gordonb.l20vx@burditt.org> wrote:
fd = open("big", O_WRONLY | O_CREAT, 0666);
| O_LARGE
Where is that supposed to be defined? I don't see it in
any source file for FreeBSD (other than as part of some
symbols of the form *_TOO_LARGE, mostly in openssl ).
lseek(fd, offset, 0);
man 2 llseek
No such manual page.
Where can I buy storage, cheap, that needs more than 64 bits for
the length of the file? And what do I need it for? Archiving
the entire contents of Google hourly?
I bet he uses Linux. Linux has traditionally maintained two separate and
parallel APIs, one in which off_t is a 'long' (32 on a 32-bit system),
the other in which it's 64 bits. llseek is one of the underlying system
calls for the latter mode, and it is also sometimes misused (where
lseek64 _should_ be used) to be able to access files in "large file"
mode in the former. O_LARGEFILE (misspelled above as O_LARGE) is a bit
used internally by open64() which is also sometimes misused in the same
way.
Just a data point: on Solaris 8 (and presumably later versions as well
since Sun is anal-retentive about keeping interface compatibility at
both the binary and source levels[1]), the documentation says that
calling open() with O_LARGEFILE is equivalent to calling open64(),
which to me indicates that using O_LARGEFILE with open() is kosher.
Solaris also maintains a parallel set of APIs, so that 32-bit
applications can use either 32-bit or 64-bit file offsets, so Linux
isn't unique in that regard. (I don't even think Linux was first,
but I can't remember.)
- Logan
[1] which, by the way, is a good thing in many cases
I won't disagree that binary compatibility is good - but off_t should
have been 64-bit to start with. On FreeBSD, off_t has NEVER been less
than 64 bits.
Actually, the API for FreeBSD V1.X had 32 bit lseek arguments. FreeBSD V2.X
had the proper (64bit) offset api, but wasn't fully implemented until about
V2.2.X... (I know, I wrote a lot of the lower level infrastructure.)
|
How far back does the cvsweb go, in terms of what version of freebsd? I
traced off_t through all the headers it was in, and it was never
typedef'd to anything other than long long or int64_t. |
|
| Back to top |
|
|
Nick Roberts
*nix forums beginner
Joined: 12 Feb 2006
Posts: 5
|
| Posted: Mon Feb 27, 2006 1:33 pm Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Maxim S. Shatskih wrote:
| Quote: | (2) Why cant direc tories be hard-linked to? After all, this is what
the OS does when it automatically creates the entries "." and ".."
Because this pollutes the notion of the "parent directory" - what directory is
parent - link1 or link2?
This also can break any tool which does directory recursion.
|
I think this is the most important point.
As I understand it, the inventors of Unix originally intended all Unix
file systems to be 'acyclic', meaning that there should never be any
path that could reach a directory through one of its own subdirectories.
Unix had a variety of programs that recursed (or had an option to cause
it to recurse) through a subtree of the directory structure, meaning
that the program operated on the files in a directory, and then operated
on all the files (and subdirectories) in its subdirectories. These
programs did not (originally) have any precaution against the infinite
recursion that would be caused a cyclicity; the program would just get
stuck*.
Nowadays, modern Unixen (and their many progeny) have directory
symlinks, and sometimes other kinds of directory links, and recursive
programs have had to be modified to give them some resilience against
infinite recursion.
For more, go to Google Groups, and search for 'acyclic' in a.o.d.
--
Nick Roberts
* This used to be called a 'crash' in the old days, but nowadays 'crash'
seems to be used exclusively for a program terminating due to error. Of
course, a program that gets stuck often eventually runs out of (virtual)
memory, and then crashes. The problem nowadays is that it can take a
long time for a program to run out of virtual memory, and doing so can
cause mayhem.  |
|
| Back to top |
|
|
Maxim S. Shatskih
*nix forums addict
Joined: 02 Apr 2005
Posts: 55
|
| Posted: Mon Feb 27, 2006 8:42 pm Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
| Quote: | Just to confirm, is this correct. Are *NIX directory entries limited to
inodes on the same volume
|
Yes.
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com |
|
| Back to top |
|
|
Henry Townsend
*nix forums Guru Wannabe
Joined: 26 Feb 2005
Posts: 129
|
| Posted: Tue Feb 28, 2006 4:10 am Post subject:
Re: Whats the practical maximum file size using indexed allocation (I nodes)
|
|
|
Maxim S. Shatskih wrote:
| Quote: | Just to confirm, is this correct. Are *NIX directory entries limited to
inodes on the same volume
Yes.
|
Yes but it must be added, the question itself seems to reveal a lack of
understanding. True, Unix (what's a *NIX anyway?) directory entries have
this limitation but that's like saying potatoes will fall if you drop
them from a tower - yes, and so will anything else. All directory
entries, Unix or not, are inherently limited to references within their
own filesystem, just as pointers in a program cannot point into another
program's address space and a table in a relational database cannot
contain data from another database. These are not quirks of
implementation; they are requirements of reality.
And BTW, in the Unix world at least, the terms "volume", "partition",
"drive", and "filesystem" refer to potentially divergent things.
Directory entries can point only within their _filesystem_; the volume
is something else again. |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Thu Jan 08, 2009 5:07 pm | All times are GMT
|
|
Bankruptcy | Guitar Books | Free Photo Storage | Babb Fest | Credit Counseling
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
Programming
»
Unix internals
