|
|
|
|
|
|
| Author |
Message |
Richard L. Hamilton
*nix forums Guru Wannabe
Joined: 22 Feb 2005
Posts: 277
|
 Posted: Thu Feb 09, 2006 9:40 pm Post subject:
Re: An mmap question
|
|
|
In article <ds8hhm$e2q$1@wsc10.lrz-muenchen.de>,
Thomas Maier-Komor <maierkom@lpr.e-technik.tu-muenchen.de> writes:
| Quote: | Richard L. Hamilton wrote:
In article <1139096335.382581.202920@f14g2000cwb.googlegroups.com>,
"Olumide" <50295@web.de> writes:
Thomas Maier-Komor wrote:
If they don't have a parent/child relationship, they can still share
filedescriptors using IPC mechanisms.
I'm not asking about file descriptors. My question (if your read it
carefully) is about how two unrelated processes mapping the same
portion of the same file end up sharing the same physical memory frames
- if at all they do. Otherwise stated: are the file-mapped segments of
both process mapped to the same physical memory frames? If so, how and
why?
/dev/zero is different from most other things; each mapping of /dev/zero
produces a new block of "anonymous" (not backed by an existing file)
memory, and even as a shared mapping, only processes that are descendants
of the one that performed the particular mmap() operation against /dev/zero
will share the memory that it allocated.
I agree. But to mmap /dev/zero you will need a file descriptor, which
can be shared among unrelated processes. A possible method would be to
use UNIX domain sockets to pass the file descriptors. In consequence a
page mmap'ed from /dev/zero can be shared among unrelated processes as
well.
|
Or if applicable to one's system, there's a somewhat similar mechanism for
passing file descriptors over a STREAMS pipe. But if one process passes a
file descriptor to another, and they both mmap() /dev/zero, why should the
two mappings refer to the same memory? Or to put it another way, is it
the open() instance by which /dev/zero mappings are unique, or is it the
mmap() itself? I don't think the documentation requires the behavior you
imply. Indeed, a cursory look at the code leaves me thinking it works as
I suspect, not as you imply:
http://cvs.opensolaris.org/source/xref/on/usr/src/uts/common/io/mem.c#858
(the Solaris 8 code abstracts this a bit less, and is thus clearer IMO, but
isn't publically shareable)
[After writing that, I just looked ahead one post, and noticed you changed
your mind and decided that wasn't possible after all; and also mentioned
mmap() of regular files for shared memory, as I do further down. Not
looking to start anything, so let's say we both got on the right track, by
slightly different routes.  ]
| Quote: | mmap() of most other devices, or of regular files, will be shared
where all parties requested a shared mapping of some common portion of
the same file. In fact, even if they requested a private mapping,
it will be shared until any party requesting a private mapping writes
to it, at which time they'll get a private copy.
Just a note: that is called copy-on-write and is a performance
enhancements most UNIXs do, but it is not specified to be the required
implementation method. If you want to dive really deep into these areas
of UNIX, you might want to take a look at "Solaris internals". Be aware
that a new version of this book is due to come in April or May (if I
remember correctly).
|
I know what copy-on-write is, but I didn't know if the OP did or not.
The best bet is not to make any assumptions about whether or not some other
process could change the contents of a page your process has a private
mapping of, except in cases where you _know_ it couldn't (not counting
/dev/mem or /proc/*/as manipulation). At least, I think that's what you meant.
| Quote: | If you want shared memory between unrelated processes not set up by
a common ancestor, either mmap() a regular file, or use System V style
shared memory segments, or the POSIX shm_open()/shm_close() functions
(typically built on top of mmap()). How they agree on just what
file to map, IPC key to use, or name to pass to shm_open(), are your
problem...
I think using shm_open et al is at least better documented than passing
file descriptors across UNIX domain sockets. So if you want to do this,
go this way.
|
I _hate_ how undocumented that (or the equivalent method over STREAMS) has
historically been. Haven't checked the man pages recently, but it used to
be the only decent doc for that was if one shelled out the $$ for Stevens'
book[s]. Worth having anyway of course, but pricey.
But yes, shm_open() seems to me (as someone who hasn't paid $$ for copies
of the standards, nor looked at the latest SUS recently) to be the most
explicitly standards-based approach, although new enough that it might
not be on older platforms (but arguably not that hard to implement oneself
on any platform that has mmap() with MAP_SHARED, and maybe ftruncate()).
Even though it's probably not the way to go for new stuff, I tend to have
a soft spot for SysV shared memory, because there are tools like ipcs to
report on it. Given knowledge of the implementation, one could in many
cases create something similar for shm_open() (or figure out lsof output
well enough to get by), but the whole point of shm_open() is to hide
implementation. IMO, it wouldn't have hurt to have specified a comparable
tool for shm_open(), and just left it optional, in case it wasn't feasible
on some implementations. But if that was done, I haven't seen it, nor
have I heard of the discussions leading to the standard to know whether it
was even considered.
--
mailto:rlhamil@smart.net http://www.smart.net/~rlhamil
Lasik/PRK theme music:
"In the Hall of the Mountain King", from "Peer Gynt" |
|
| Back to top |
|
 |
Thomas Maier-Komor
*nix forums Guru
Joined: 20 Feb 2005
Posts: 508
|
| Posted: Mon Feb 06, 2006 11:34 pm Post subject:
Re: An mmap question
|
|
|
Thomas Maier-Komor wrote:
| Quote: | Richard L. Hamilton wrote:
/dev/zero is different from most other things; each mapping of /dev/zero
produces a new block of "anonymous" (not backed by an existing file)
memory, and even as a shared mapping, only processes that are descendants
of the one that performed the particular mmap() operation against
/dev/zero
will share the memory that it allocated.
I agree. But to mmap /dev/zero you will need a file descriptor, which
can be shared among unrelated processes. A possible method would be to
use UNIX domain sockets to pass the file descriptors. In consequence a
page mmap'ed from /dev/zero can be shared among unrelated processes as
well.
|
OK, after thinking it over one more time, I have to correct myself.
Sharing the file descriptor is not enough when it comes to sharing pages
between unrelated processes. To get the same page of virtual memory you
need its concrete mapping, but /dev/zero has no backing store, as you
said before. So remapping the same file descriptor won't give anything,
as the page never hits anything sharable concerning this specific file
descriptor.
But you still have a chance to share pages of virtual memory in this
manner. But one has to create a file. On Solaris /tmp looks a perfect
fit to me, as it is based on virtual memory. Other systems probably have
something similar.
Tom |
|
| Back to top |
|
|
Thomas Maier-Komor
*nix forums Guru
Joined: 20 Feb 2005
Posts: 508
|
| Posted: Mon Feb 06, 2006 10:14 pm Post subject:
Re: An mmap question
|
|
|
Richard L. Hamilton wrote:
| Quote: | In article <1139096335.382581.202920@f14g2000cwb.googlegroups.com>,
"Olumide" <50295@web.de> writes:
Thomas Maier-Komor wrote:
If they don't have a parent/child relationship, they can still share
filedescriptors using IPC mechanisms.
I'm not asking about file descriptors. My question (if your read it
carefully) is about how two unrelated processes mapping the same
portion of the same file end up sharing the same physical memory frames
- if at all they do. Otherwise stated: are the file-mapped segments of
both process mapped to the same physical memory frames? If so, how and
why?
/dev/zero is different from most other things; each mapping of /dev/zero
produces a new block of "anonymous" (not backed by an existing file)
memory, and even as a shared mapping, only processes that are descendants
of the one that performed the particular mmap() operation against /dev/zero
will share the memory that it allocated.
|
I agree. But to mmap /dev/zero you will need a file descriptor, which
can be shared among unrelated processes. A possible method would be to
use UNIX domain sockets to pass the file descriptors. In consequence a
page mmap'ed from /dev/zero can be shared among unrelated processes as
well.
| Quote: | mmap() of most other devices, or of regular files, will be shared
where all parties requested a shared mapping of some common portion of
the same file. In fact, even if they requested a private mapping,
it will be shared until any party requesting a private mapping writes
to it, at which time they'll get a private copy.
|
Just a note: that is called copy-on-write and is a performance
enhancements most UNIXs do, but it is not specified to be the required
implementation method. If you want to dive really deep into these areas
of UNIX, you might want to take a look at "Solaris internals". Be aware
that a new version of this book is due to come in April or May (if I
remember correctly).
| Quote: | If you want shared memory between unrelated processes not set up by
a common ancestor, either mmap() a regular file, or use System V style
shared memory segments, or the POSIX shm_open()/shm_close() functions
(typically built on top of mmap()). How they agree on just what
file to map, IPC key to use, or name to pass to shm_open(), are your
problem...
|
I think using shm_open et al is at least better documented than passing
file descriptors across UNIX domain sockets. So if you want to do this,
go this way. |
|
| Back to top |
|
|
Richard L. Hamilton
*nix forums Guru Wannabe
Joined: 22 Feb 2005
Posts: 277
|
| Posted: Mon Feb 06, 2006 9:37 pm Post subject:
Re: An mmap question
|
|
|
In article <1139096335.382581.202920@f14g2000cwb.googlegroups.com>,
"Olumide" <50295@web.de> writes:
| Quote: | Thomas Maier-Komor wrote:
If they don't have a parent/child relationship, they can still share
filedescriptors using IPC mechanisms.
I'm not asking about file descriptors. My question (if your read it
carefully) is about how two unrelated processes mapping the same
portion of the same file end up sharing the same physical memory frames
- if at all they do. Otherwise stated: are the file-mapped segments of
both process mapped to the same physical memory frames? If so, how and
why?
|
/dev/zero is different from most other things; each mapping of /dev/zero
produces a new block of "anonymous" (not backed by an existing file)
memory, and even as a shared mapping, only processes that are descendants
of the one that performed the particular mmap() operation against /dev/zero
will share the memory that it allocated.
mmap() of most other devices, or of regular files, will be shared
where all parties requested a shared mapping of some common portion of
the same file. In fact, even if they requested a private mapping,
it will be shared until any party requesting a private mapping writes
to it, at which time they'll get a private copy.
If you want shared memory between unrelated processes not set up by
a common ancestor, either mmap() a regular file, or use System V style
shared memory segments, or the POSIX shm_open()/shm_close() functions
(typically built on top of mmap()). How they agree on just what
file to map, IPC key to use, or name to pass to shm_open(), are your
problem...
| Quote: | But maybe this is the point, where you should get a good book.
A good book is always a good idea.
|
--
mailto:rlhamil@smart.net http://www.smart.net/~rlhamil
Lasik/PRK theme music:
"In the Hall of the Mountain King", from "Peer Gynt" |
|
| Back to top |
|
|
50295@web.de
*nix forums beginner
Joined: 09 Feb 2005
Posts: 32
|
| Posted: Sat Feb 04, 2006 11:38 pm Post subject:
Re: An mmap question
|
|
|
Thomas Maier-Komor wrote:
| Quote: | If they don't have a parent/child relationship, they can still share
filedescriptors using IPC mechanisms.
|
I'm not asking about file descriptors. My question (if your read it
carefully) is about how two unrelated processes mapping the same
portion of the same file end up sharing the same physical memory frames
- if at all they do. Otherwise stated: are the file-mapped segments of
both process mapped to the same physical memory frames? If so, how and
why?
| Quote: | But maybe this is the point, where you should get a good book.
|
A good book is always a good idea. |
|
| Back to top |
|
|
Thomas Maier-Komor
*nix forums Guru
Joined: 20 Feb 2005
Posts: 508
|
| Posted: Sat Feb 04, 2006 10:11 pm Post subject:
Re: An mmap question
|
|
|
Olumide wrote:
| Quote: | Thomas Maier-Komor wrote:
Thanks Tom. What say you about sharing physical memory frames (please
refer to my original question).
if you mmap /dev/zero shared, you will share the same physical pages
between parent/child processes that are fork'ed.
(I hope I dont sound too obstinate but) The communicating processes are
not related related.
|
If they don't have a parent/child relationship, they can still share
filedescriptors using IPC mechanisms.
But maybe this is the point, where you should get a good book.
Tom |
|
| Back to top |
|
|
50295@web.de
*nix forums beginner
Joined: 09 Feb 2005
Posts: 32
|
| Posted: Thu Feb 02, 2006 5:12 pm Post subject:
Re: An mmap question
|
|
|
Thomas Maier-Komor wrote:
| Quote: | Thanks Tom. What say you about sharing physical memory frames (please
refer to my original question).
if you mmap /dev/zero shared, you will share the same physical pages
between parent/child processes that are fork'ed.
|
(I hope I dont sound too obstinate but) The communicating processes are
not related related. |
|
| Back to top |
|
|
Thomas Maier-Komor
*nix forums Guru
Joined: 20 Feb 2005
Posts: 508
|
| Posted: Wed Feb 01, 2006 11:28 pm Post subject:
Re: An mmap question
|
|
|
Olumide wrote:
| Quote: | Thomas Maier-Komor wrote:
This result of mmaping /dev/zero are null initialized pages in memory
that do not have backing store (i.e. no on disk inodes are assigned). A
private mmap'ed file is always copy on write after fork, while a shared
mmap is shared across forks. Mapping /dev/zero is a little bit like
calloc, but the dynamic behavior of the memory pages is different across
forks.
Thanks Tom. What say you about sharing physical memory frames (please
refer to my original question).
- Olumide
|
if you mmap /dev/zero shared, you will share the same physical pages
between parent/child processes that are fork'ed.
As I said before: you should read mmap(2), fork(2), and zero(7d).
Additionally, any decent UNIX systems programming containing IPC should
help you, too.
Tom |
|
| Back to top |
|
|
50295@web.de
*nix forums beginner
Joined: 09 Feb 2005
Posts: 32
|
| Posted: Wed Feb 01, 2006 3:53 pm Post subject:
Re: An mmap question
|
|
|
Thomas Maier-Komor wrote:
| Quote: | This result of mmaping /dev/zero are null initialized pages in memory
that do not have backing store (i.e. no on disk inodes are assigned). A
private mmap'ed file is always copy on write after fork, while a shared
mmap is shared across forks. Mapping /dev/zero is a little bit like
calloc, but the dynamic behavior of the memory pages is different across
forks.
|
Thanks Tom. What say you about sharing physical memory frames (please
refer to my original question).
- Olumide |
|
| Back to top |
|
|
Thomas Maier-Komor
*nix forums Guru
Joined: 20 Feb 2005
Posts: 508
|
| Posted: Mon Jan 30, 2006 11:00 pm Post subject:
Re: An mmap question
|
|
|
Olumide wrote:
| Quote: | Hi -
'Got a two questions:
Major:
Most of textbook mmap code I've seen often use map the file /dev/zero
or some other special file - this seems to be common when the objective
is goal is IPC. I find this perticularly strange because the file
/dev/zero surely cannot be modified - or can it? If not, why map to
/dev/zero then? ... Or is the file /dev/zero used instead of creating a
temporary file, (I'm refering to IPC with mmaps) in which case the the
implementation relies on the fact that the commnicating processes will
share the physical memory frames as a result of having mapped the same
file (/dev/zero) which will not (or cannot?) be modified anyway.
|
This result of mmaping /dev/zero are null initialized pages in memory
that do not have backing store (i.e. no on disk inodes are assigned). A
private mmap'ed file is always copy on write after fork, while a shared
mmap is shared across forks. Mapping /dev/zero is a little bit like
calloc, but the dynamic behavior of the memory pages is different across
forks.
I would recommend reading the docs on opengroup.org or the Solaris man
pages for mmap(2), fork(2), and zero(7d).
| Quote: |
Minor:
Whenever I trace a process on my Linux machine, I see its using
old_mmap() a lot. Why old_mmap() and not mmap()?
|
probably because there are two implementation of mmap under Linux. But I
don't know what the reason might be...
HTH,
Tom |
|
| Back to top |
|
|
50295@web.de
*nix forums beginner
Joined: 09 Feb 2005
Posts: 32
|
| Posted: Mon Jan 30, 2006 6:12 pm Post subject:
An mmap question
|
|
|
Hi -
'Got a two questions:
Major:
Most of textbook mmap code I've seen often use map the file /dev/zero
or some other special file - this seems to be common when the objective
is goal is IPC. I find this perticularly strange because the file
/dev/zero surely cannot be modified - or can it? If not, why map to
/dev/zero then? ... Or is the file /dev/zero used instead of creating a
temporary file, (I'm refering to IPC with mmaps) in which case the the
implementation relies on the fact that the commnicating processes will
share the physical memory frames as a result of having mapped the same
file (/dev/zero) which will not (or cannot?) be modified anyway.
Minor:
Whenever I trace a process on my Linux machine, I see its using
old_mmap() a lot. Why old_mmap() and not mmap()?
Thanks,
- Olumide |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Sat Nov 22, 2008 11:48 am | All times are GMT
|
|
Hypotonic Cerebral Palsy | Credit Cards | Personal Loans | Flights to Cairo | Advertising
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
Programming
»
Unix internals
