|
|
|
|
|
|
| Author |
Message |
Dag-Erling Smorgrav
*nix forums Guru Wannabe
Joined: 23 Mar 2002
Posts: 110
|
 Posted: Sun Jul 07, 2002 7:28 pm Post subject:
Re: Package system flaws?
|
|
|
Doug Barton <DougB@FreeBSD.org> writes:
| Quote: | What is it about working on ports that makes people automatically
gravitate to the most hideous, complicated solutions? Why not just
create a tarball with the stuff to install, create the metadata file (in
whatever format), then create a tarball with the two together? Then,
step one is to unroll the tarball, read the metadata, grab dependencies
if needed, then proceed to do the rest of the install.
|
We want to be able to install a package from a non-rewindable source
without storing a temporary copy on disk. This means the metadata
must without fail be at the very beginning of the package.
DES
--
Dag-Erling Smorgrav - des@ofug.org
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
 |
Cy Schubert - CITS Open S
*nix forums beginner
Joined: 14 May 2002
Posts: 8
|
| Posted: Sun Jul 07, 2002 8:02 pm Post subject:
Re: Package system flaws?
|
|
|
In message <3D289529.95C1EC8A@softweyr.com>, Wes Peters writes:
| Quote: | Dag-Erling Smorgrav wrote:
Wes Peters <wes@softweyr.com> writes:
Dan Moschuk wrote:
Where can we improve?
In all of the above areas, plus all the ones we haven't addressed yet.
*grin*
One thing we could improve a lot is the package file format. We
currently use gzipped tarballs, which have to be completely unpacked
before processing can begin. One improvement we can make is to use an
archive format such as zip that allows us to extract individual files
quickly, so we can extract the metadata and check dependencies
etc. without unpacking the entire package.
I'd rather go one step further and separate the metadata from the files
completely. I'm thinking an XML-ish format for the metadata, with
blobs of files stored in whatever format makes the most sense inter-
spersed. Zip format makes a lot of sense and is available in library
format, but I could easily tolerate tar or cpio, as well as a variety
of compression formats. The metadata should not be compressed for
speed of package processing.
|
This seems like a reasonable approach. I don't think that the format
of the files matters that much. The zip file format has a couple of
advantages. First, as you mention, it's available in library format.
Secondly, vendors like Sun are using this format to distribute patches.
I think this format will quickly become the UNIX standard, superseding
tar.
Whether metadata is compressed or not is immaterial. With the speed of
processors and the price of disk these days, I don't think it matters
either way, except for possibly network congestion. I suspect that
metadata would not be the lions share of the data anyhow.
| Quote: |
This would also allow us to separate the files in the package into
distinct pieces, and apply metadata to the collections of files.
Picture, for instance, a package that installs an X application. We
have executable files that need to go into $PKG_BIN, perhaps a library
or two that needs to in $PKG_LIB, an application resource file that
needs to go into $X_APPRESDIR and some fonts that need to be installed
in $X_FONTDIR. We provide defaults for each of these, let the user
override them as necessary.
|
So far so good.
I would also include the possibility of source being distributed with a
package. In my previous life as an MVS systems programmer, packages
(in MVS they're called functions) may be delivered in source form,
binary form, or both. For example, JES/2 (Job Entry Subsystem/2), one
of the MVS spooling subsystems, was distributed in source and object
form. When JES would be installed, sources and binaries were installed
by the same package (in different locations of course). I as the
systems programmer for a site could apply fixes (PTF's and APARFIXES)
or apply my own modifications to the code (USERMODS) to the source.
SMP/E would automatically invoke the assembler to rebuild any affected
binaries. Alternatively I could apply fixes or modifications in binary
form (using an application called superzap -- IMASPZAP). (Pretty scary
that I can remember this stuff after not having looked at it for 10
years.)
When upgrading SMP/E would flag my USERMODS that were being backed out
in order to accommodate the upgrade. This way I as the systems
programmer for a site wouldn't have to remember a lot of details about
my system, SMP/E did all that for me. I personally used SMP/E to
update system config files too. That saved me from having to write a
lot of stuff down because it was already recorded in my target zone
(database). Dates and times and all that good stuff were recored.
I don't think we need anything as elaborate as this. IMO not many
FreeBSD users make custom modifications to their systems. Those of us
who do, myself included, probably need our heads examined (or in my
case I should make a case to have them applied to the source tree).
However in the MVS world, I have never worked at a site that did not
customize the source or apply binary patches in any way. Different
requirements and different culture I guess.
All of this helped make SMP/E the monstrosity it became. It was great
for techno-wizards but for the average MVS guy, I've been told it was
too much. I think we need to find the happy medium between
functionality and scaryness.
Much of this stuff is pretty useless here, but if you want to use any
of these ideas, be my guest.
| Quote: |
This would save both time
and space (the current system can fail if the temp directory doesn't
have room for the unpacked package, even if the installation directory
has room to spare). A further improvement is to use a custom archive
format that always stores the metadata at the beginning of the archive
so we can install packages from the net without having to download and
store them locally first (zip isn't suitable for this as it stores the
content directory at the end of the archive, and the files within the
archive can appear in any order).
Yes, an excellent idea. With a format as I've proposed, it would be
possible to specify the file collections as URLs rather than encoded
archives. You could download the entire package collection in about
the same amount of space as the current ports collection. Of course
this means pkg_add would have to be linked with libfetch, too. ;^)
|
This too is a good idea. However I personally like to keep local copy
of package distribution files locally just in case they're needed for
DRP recovery. In a DRP situation one may not want to install the
latest version of a package, because the only focus should be to get
the application(s) working again, not upgrade the system, and there is
no guarantee that a certain version of a package will be there days,
months, or even years from now. Keeping one's own copy is the only way
to guarantee that. The option to fetch and store the files that
comprise a package might be worthwhile looking at.
--
Cheers, Phone: 250-387-8437
Cy Schubert Fax: 250-387-5766
Team Leader, Sun/Alpha Team Email: Cy.Schubert@osg.gov.bc.ca
Open Systems Group, CITS
Ministry of Management Services
Province of BC
FreeBSD UNIX: cy@FreeBSD.org
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Terry Lambert
*nix forums Guru
Joined: 19 Mar 2002
Posts: 434
|
| Posted: Sun Jul 07, 2002 8:20 pm Post subject:
Re: Package system flaws?
|
|
|
Dag-Erling Smorgrav wrote:
| Quote: | Doug Barton <DougB@FreeBSD.org> writes:
What is it about working on ports that makes people automatically
gravitate to the most hideous, complicated solutions? Why not just
create a tarball with the stuff to install, create the metadata file (in
whatever format), then create a tarball with the two together? Then,
step one is to unroll the tarball, read the metadata, grab dependencies
if needed, then proceed to do the rest of the install.
We want to be able to install a package from a non-rewindable source
without storing a temporary copy on disk. This means the metadata
must without fail be at the very beginning of the package.
|
I think you just want lookahead, actually.
HTTP and FTP are both rewindable; the expense is relative to the
server, but FTP supports "reget" and HTTP supports range requests.
Putting the metadata up front resloves the issue of downloading
data you don't want, and also resolves the problem of being able
to collect all the metadata at the start, so that you can make
accurate time estimates, and the user can make a time-cost-basis
decision on whether or not to go ahead.
Modern transfer protocols argue in general for a new archive format
that collects the metadata at the start of the archive to permit
such decisions to be made.
From a general perspective, archives need to be able to be nested
recursively, and to be extracted recursively. Obviously, this is a
control mechanism enabling requirement: full extraction as a
default is necessary for "bootstrap", and selective extraction is
necessary as a component/function/customization selection method.
Also from a general perspective, it needs to be possible to blindly
extract archives for "bootstrap" purposes: a bind extraction of an
archive from a current directory of "/" needs to result in not only
installation of the package contents in the correct default location,
but also in correct registration of those contents into the system,
minimally for one archive... but ideally for "n" archives... until a
more complete tools system has been bootstrapped.
If you guys all want to think about something, think first about the
issues surrounding bootstrapping, and second, about the issues that
surround update and failure recovery, since they are the most critical
and least well supported by the current tools.
-- Terry
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Dag-Erling Smorgrav
*nix forums Guru Wannabe
Joined: 23 Mar 2002
Posts: 110
|
| Posted: Sun Jul 07, 2002 8:36 pm Post subject:
Re: Package system flaws?
|
|
|
Doug Barton <DougB@FreeBSD.org> writes:
| Quote: | Ok, then what about storing the meta data as a seperate file? Why do they
have to be in the same package?
|
How do you ensure the metadata match the data if they're in separate
files?
DES
--
Dag-Erling Smorgrav - des@ofug.org
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Christian Weisgerber
*nix forums beginner
Joined: 29 May 2002
Posts: 18
|
| Posted: Sun Jul 07, 2002 9:45 pm Post subject:
Re: Package system flaws?
|
|
|
Dan Moschuk <dan@freebsd.org> wrote:
| Quote: | I've been doing some thinking lately about our ports system versus what
other systems have adopted and was curious as to what people think on
the subject? What does FreeBSD do well? Where can we improve? How does it
rate against the umpteen Linux flavours?
|
Compared to OpenBSD, the FreeBSD ports system lacks FAKE, FLAVORS,
and MULTI_PACKAGES. regress would be nice, too.
Compared to Debian and RPM the ability to do comprehensive updates
from packages is missing. Install 500 ports. Wait a year for
revisions, library versions, and even dependencies to change all
over. Now let's upgrade all installed ports from a current set of
binary packages. That sort of thing.
--
Christian "naddy" Weisgerber naddy@mips.inka.de
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Garrett Wollman
*nix forums addict
Joined: 08 May 2002
Posts: 59
|
| Posted: Sun Jul 07, 2002 10:04 pm Post subject:
Re: Package system flaws?
|
|
|
In article <3D289529.95C1EC8A@softweyr.com> Wes wrote:
| Quote: | spersed. Zip format makes a lot of sense and is available in library
format, but I could easily tolerate tar or cpio, as well as a variety
of compression formats. The metadata should not be compressed for
speed of package processing.
|
`pax' format provides for a convenient extension-header mechanism that
would make this relatively easy to accomplish without inventing a new
archive format.
-GAWollman
--
Garrett A. Wollman | [G]enes make enzymes, and enzymes control the rates of
wollman@lcs.mit.edu | chemical processes. Genes do not make ``novelty-
Opinions not those of| seeking'' or any other complex and overt behavior.
MIT, LCS, CRS, or NSA| - Stephen Jay Gould (1941-2002)
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Doug Barton
*nix forums addict
Joined: 24 Apr 2002
Posts: 91
|
| Posted: Sun Jul 07, 2002 10:27 pm Post subject:
Re: Package system flaws?
|
|
|
On Sun, 7 Jul 2002, Terry Lambert wrote:
| Quote: | We want to be able to install a package from a non-rewindable source
without storing a temporary copy on disk. This means the metadata
must without fail be at the very beginning of the package.
|
Ok, then what about storing the meta data as a seperate file? Why do they
have to be in the same package?
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Doug Barton
*nix forums addict
Joined: 24 Apr 2002
Posts: 91
|
| Posted: Sun Jul 07, 2002 10:39 pm Post subject:
Re: Package system flaws?
|
|
|
On 8 Jul 2002, Dag-Erling Smorgrav wrote:
| Quote: | Doug Barton <DougB@FreeBSD.org> writes:
Ok, then what about storing the meta data as a seperate file? Why do they
have to be in the same package?
How do you ensure the metadata match the data if they're in separate
files?
|
coolport-1.0-package.tgz
coolport-1.0-meta.gz
c'mon... use a little creativity. :)
--
"We have known freedom's price. We have shown freedom's power.
And in this great conflict, ... we will see freedom's victory."
- George W. Bush, President of the United States
State of the Union, January 28, 2002
Do YOU Yahoo!?
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Jordan K Hubbard
*nix forums beginner
Joined: 07 Jul 2002
Posts: 17
|
| Posted: Sun Jul 07, 2002 10:41 pm Post subject:
Re: Package system flaws?
|
|
|
On Sunday, July 7, 2002, at 01:55 PM, Dag-Erling Smorgrav wrote:
| Quote: | That's precisely what I want to avoid. With tar, you always have to
uncompress the entire package to find even just one file (unless you
use the GNU-specific --fast-read option, in which case it stops once
it finds the file you asked for)
|
Actually, it's the FreeBSD-specific --fast-read option. I added it so
that I could extract the +CONTENTS file without having tar go all the
way through the tarball looking for additional +CONTENTS files.
I'm actually with Wes anyway - zip would be a fine format on account of
the fact that it:
1) Allows random access AND compression
2) Has comment fields for each archive member and for the zip file
itself which can be [ab]used for storing checksums, PGP signatures, etc.
3) Can be programmatically accessed using some of the libzip libraries
available so you aren't always at the mercy of being at arm's length
with a command like tar (which is nice when you're trying to write a
package-extracting front-end which is graphical and also has little
progress meters and such which advance as each item is extracted from
the archive)..
But then again, I also promised myself I'd stay out of this discussion
so I'll shut up now and continue to watch all of you debate this. :)
--
Jordan K. Hubbard
Engineering Manager, BSD technology group
Apple Computer
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Garance A Drosihn
*nix forums Guru Wannabe
Joined: 21 Mar 2002
Posts: 190
|
| Posted: Mon Jul 08, 2002 12:00 am Post subject:
Re: Package system flaws?
|
|
|
At 7:08 PM -0700 7/6/02, Wes Peters wrote:
| Quote: | Dan Moschuk wrote:
I'm not sure that this classifies as an architectual discussion
(for now) so if you feel its appropriate please feel free to
redirect to ports@.
I'd certainly like to see it evolve into an architectural
discussion, even if all it accomplishes is to layout the work
that needs to be done and provide some sort of road map of what
the next step or two might be.
|
There is probably room for several different architectural
discussions.
One is whether we use tar vs some other archive format to hold
the data and the metadata for each port. I agree that is an
important topic, but I have no strong opinions on it. Whatever
is decided upon by others would probably work fine for me.
I do find the ports collection one of the most useful things in
the freebsd community, but the more comfortable you get with
depending on it, the more you see little cracks where various
problems crop up -- over and over again.
- - - -
Separate from that, one thing I think we need is some mechanism
which makes certain that the generated package-list for a port
is exactly correct. I install about 70-80 ports, and I know
several of them install files which are not included in their
package-list. Try running
find /usr/local /usr/X11R6 -type f -print0 | \
xargs -0 /usr/local/sbin/pkg_which -v | fgrep '?'
Whatever we do for this, it should be something which is
simple enough to use that it does not add work for the ports
maintainers.
Conceptually what I would like to do is have ports *built* for
/usr/local (just as they are now), but then when it comes time
to do the make-install, the make-install would do something
like:
(chroot /usr/local/TEMP ; make install)
and then copy all files from /usr/local/TEMP/usr/local
to the real /usr/local, based on the package-list for
the port.
It obviously wouldn't be quite *that* simple to do, but the
idea is that you could be certain that the package list
included every file that was actually installed. It would also
be easy to see every file which SHOULD have been installed,
because they would all be still sitting in the
/usr/local/TEMP/usr/local directories. I believe that openbsd
has something like this, but I do not know the details.
I have also thought that maybe it could be done with Kirk's
filesystem snapshots, where you'd do a snapshot before building
a port, and another one afterwards, and find out what files it
changed. This would be a rather heavy-handed way to approach
the problem, but it avoids some of the technical issues that
come up with the chroot approach.
- - - -
Portupgrade is a great step forward in rebuilding ports, but it
also has it's limitations. It has an idea of dependences between
ports, but it only knows about the dependences as of the last time
the port was built. If progAA was built a month ago, and at that
time it only depended on progBB, then 'portupgrade -Rr -n progAA'
will only consider progAA and progBB. If the *new* version of
progAA (the one you will build) now depends on progCC, you will
not find out about that until portupgrade does the 'make', and
it turns around to build progCC.
This also shows up if you have NOCLEANDEPENDS=true in your
/etc/make.conf file. Portupgrade will do a:
make clean ; make ; make install; make clean
for every port that it builts -- but it only knows to do that for
the ports that it *already* knows about. If you ask it to install
progAA, that depends on progBB and progCC, then those 'make clean's
are only done for progAA. The work-directories for progBB and
progCC are not cleaned out before they are made, and they are
not cleaned out after they are installed.
- - - -
Portupgrade also looses track of what dependencies are, as various
things get rebuilt. I just pkg_deinstall'ed a number of ports,
and pkg_deinstall missed a good number of package dependencies. By
that I mean it quite happy removed some port, even though there
were other ports still installed which in fact depended on that
port. [that was fine for me in this case, but it was still a
little disappointing...]
That's a problem when things get new versions. It gets even more
confusing when ports are renamed, or entirely removed.
- - - -
The current ports mechanism has the idea "this port depends on
file XYZZY, and if that file does not exist then you should first
build port PLOUGH". It also needs the idea that "this port
depends on XYZZY, but it must the XYZZY built by XFree86-4, and
not the one built by XFree86-3".
- - - -
As we move into more platforms (sparc64, ia-64, ppc), then I
expect that will add even more to the work of the ports
collection.
- - - -
In addition to the above issues, there is also the issue of how
large and unwieldy the ports collection is, and how it's "one
single giant collection" (as far as something like the INDEX is
concerned, for instance). It would be nice to rearrange it a
bit, such that I can easily clip off all foreign-language ports
that I know I don't want. Right now I can kinda do that by
changing my cvsup file, and changing my portupgrade-config
file, and editing /usr/ports/Makefile, and...
| Quote: | I'm more interested in the binary package side personally, a
holdover from a previous professional involvement in this area.
I have a number of ideas for things that could be improved in
relatively small projects if someone wants to discuss those
with me.
|
I have had a number of ideas, but no time to really pursue
them. I think we try to stuff too much information into the
name of a port, and we try to do too much to shoehorn all
ports-processing into standard makefile variables and standard
make-cmd processing.
[in the background, the sound of people screaming is heard...]
And lastly, one of the problems with our ports and package system
is that it has grown so large without addressing some of these
issues. Any attempt to switch to a significantly different
mechanism would have to include a plan to EASE INTO the new
approach, probably over months of time. There is no way we are
going to convert 7,000 ports to any new mechanism, on multiple
HW platforms, on a wide-range of freebsd-versions, in some single
one-week cut-over period.
--
Garance Alistair Drosehn = gad@gilead.netel.rpi.edu
Senior Systems Programmer or gad@freebsd.org
Rensselaer Polytechnic Institute or drosih@rpi.edu
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Jordan K Hubbard
*nix forums beginner
Joined: 07 Jul 2002
Posts: 17
|
| Posted: Mon Jul 08, 2002 1:02 am Post subject:
Re: Package system flaws?
|
|
|
On Sunday, July 7, 2002, at 06:59 PM, Mark Valentine wrote:
| Quote: | 1) Allows random access AND compression
At the expense of having to seek to the end first? What about access
to the
metadata over a slow data stream, like you might have got with tar
--fast-read?
|
Having to seek to the end is, indeed, one of the major draw-backs of
zip. I have no idea why the originators, in their infinite wisdom, put
it there.
I also don't much like the red hat approach of concocting a totally new
archive format, however, even if its an agglomeration of an existing
archive format. Perhaps Garrett's pax recommendation has more merit
than we thought, unless people have other candidates to put forward.
- Jordan
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Terry Lambert
*nix forums Guru
Joined: 19 Mar 2002
Posts: 434
|
| Posted: Mon Jul 08, 2002 1:54 am Post subject:
Re: Package system flaws?
|
|
|
Doug Barton wrote:
| Quote: | On Sun, 7 Jul 2002, Terry Lambert wrote:
We want to be able to install a package from a non-rewindable source
without storing a temporary copy on disk. This means the metadata
must without fail be at the very beginning of the package.
Ok, then what about storing the meta data as a seperate file? Why do they
have to be in the same package?
|
For the same reason the fact that root owns /etc/master.passwd is
not in a seperate file?
-- Terry
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Mark Valentine
*nix forums addict
Joined: 13 Jun 2002
Posts: 62
|
| Posted: Mon Jul 08, 2002 1:59 am Post subject:
Re: Package system flaws?
|
|
|
| Quote: | From: jkh@queasyweasel.com (Jordan K Hubbard)
Date: Mon 8 Jul, 2002
Subject: Re: Package system flaws?
Actually, it's the FreeBSD-specific --fast-read option. I added it so
that I could extract the +CONTENTS file without having tar go all the
way through the tarball looking for additional +CONTENTS files.
|
Thanks for the clarification. It would be nice if the package format didn't
rely on a specific network client functionality for performance (thought you
can mitigate this with a CGI/PHP script on the server in some environments).
Of course the format designer can still try to say he's only trying to
solve the network package install problem for FreeBSD, or that the entry
barrier for efficient implementation is this specific functionality, but
that limits the usefulness of the design a little...
| Quote: | I'm actually with Wes anyway - zip would be a fine format on account of
the fact that it:
1) Allows random access AND compression
|
At the expense of having to seek to the end first? What about access to the
metadata over a slow data stream, like you might have got with tar --fast-read?
Cheers,
Mark.
--
Mark Valentine, Thuvia Labs <mark@thuvia.co.uk> <http://www.thuvia.co.uk>
"Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses
"We're kind of stupid that way." *munch* *munch* and endorses FreeBSD
-- <http://www.calvinandhobbes.com> <http://www.freebsd.org>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Mark Valentine
*nix forums addict
Joined: 13 Jun 2002
Posts: 62
|
| Posted: Mon Jul 08, 2002 3:34 am Post subject:
Re: Package system flaws?
|
|
|
| Quote: | From: Jordan K Hubbard <jkh@queasyweasel.com
Date: Sun 7 Jul, 2002
Subject: Re: Package system flaws?
Perhaps Garrett's pax recommendation has more merit
than we thought, unless people have other candidates to put forward.
|
It made me sit up and think, but you almost certainly have to extend pax(1)
to realise the extensions (correct me if I'm wrong), and I rather like a
format I can script with existing utilities.
Cheers,
Mark.
--
Mark Valentine, Thuvia Labs <mark@thuvia.co.uk> <http://www.thuvia.co.uk>
"Tigers will do ANYTHING for a tuna fish sandwich." Mark Valentine uses
"We're kind of stupid that way." *munch* *munch* and endorses FreeBSD
-- <http://www.calvinandhobbes.com> <http://www.freebsd.org>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Garrett Wollman
*nix forums addict
Joined: 08 May 2002
Posts: 59
|
| Posted: Mon Jul 08, 2002 3:56 am Post subject:
Re: Package system flaws?
|
|
|
In article <p05111745b94e9452f3b3@[128.113.24.47]> Garance writes:
| Quote: | Separate from that, one thing I think we need is some mechanism
which makes certain that the generated package-list for a port
is exactly correct.
|
The ports-build cluster does that, but only for packages.
The problem stems mainly from the fact that, when you build a port
from source, you're not starting from a clean environment (unlike
bento). Many configure scripts will then glom onto whatever random
software you happen to have installed (``Gee! You seem to have emacs
installed! Wouldn't you like to play with my Emacs Lisp files?'') and
builds or installs extra stuff that doesn't make it into the official
package. Since bento never notices it, maintainers are not frequently
motivated to deal with the problem.
(I've run into this recently working on FreeRADIUS, which I've had to
instruct to build only static executables, with no loadable modules,
in order to deal cleanly with this behavior.)
The ports system really does not have a good way of dealing with
conditional dependencies (``I can use one of these if you have it'').
For many packages, there are simply too many possible combinations for
the maintainer to effectively test all of them. It gets even worse
when some packages depend on specific options of other packages (viz.,
GNOME needs libxml+Python but KDE and docproj do not).
There really isn't any good mechanism of supporting this -- there are
already too many ports which differ from another based only on the
setting of some option or other.
It would be nice to have a mechanism like the following (since we're
all expressing our wishlists here):
- Each port contains an enumeration of the possible variants and
on-off options. Some options might not make sense with all possible
variants, so the listing of options is per-variant. Some options
might not be compatible with each other, so there needs to be a way
to handle that as well.
(The distinction I'm making here is that a `variant' changes the
behavior of the resulting program or library, in such a way as other
programs or libraries using it might depend upon, whereas an `option'
simply causes additional files to be installed, which might or might
not add functionality.)
- Each variant must have a human-friendly short name and comment,
including the default variant. Likewise options, except that the
default state (which must be `off') does not have a name.
The whole thing can be referred to as:
$PORTNAME${variant+"-$variant"}${options+"+$(echo $options | tr ' ' '+')"}
- Whether building from source or installing a package, users are
required to explicitly select the variant of their choice.
If no variant is selected, the user gets a list of possibilities and
is invited to select one (and any options that might apply).
pkg_info options and make targets are provided for automatically
displaying this list without attempting to install.
- The package build cluster does combinatorial expansion on each port,
and builds provisional packages for all variants and permissible
option combinations. (Maintainers are encouraged not to go
overboard on this!)
(Here's the important new part, which would require a significantly
enhanced package format....)
- The package build cluster, having constructed all of the possible
combinations of options and variants, and having made provisional
packages of each one, runs a unification process across all of the
provisional packages to create a single ``fat'' package with
metainformation indicating which versions of which files are present
for which variant and option combinations.
This mechanism could also be used to create multi-architecture
packages, but in most cases the executables take up the lion's share
of the space so doing this makes little sense. The functionality
should be available, though, in case third-party package creators
wish to do so.
Possible Makefile syntax might look something like this:
VARIANTS= generic foo bar
COMMENT_generic= "Whatever pkg-plist used to say"
OPTIONS_generic= ipv6
COMMENT_foo= "Use the foo graphical user interface"
OPTIONS_foo= ipv6 quux:!ipv6
COMMENT_bar= "Use the bar graphical user interface"
COMMENT_ipv6= "Include support for IP version 6"
COMMENT_quux= "Build a loadable completion module for the quux shell"
..if "${VARIANT}" = "foo" || "${VARIANT}" = "generic"
..if "${OPTIONS:M/ipv6/}" != ""
PATCH_FILES+= ${PORTNAME}-ipv6-patch
..endif
..endif
..if "${VARIANT}" = "foo"
CONFIGURE_ARGS+=--with-libfoo
LIB_DEPENDS+= foo.23:${PORTSDIR}/x11/foo-lib
..if "${OPTIONS:M/quux/}" != ""
CONFIGURE_ARGS+=--enable-quux-completion
BUILD_DEPENDS+= ${PREFIX}/include/quux/complete.h:${PORTSDIR}/shells/quux::generic+dso
# depend specifically on the `dso' option of the `generic' variant of quux
..endif
..endif
..if "${VARIANT}" = "bar"
CONFIGURE_ARGS+=--with-libbar
LIB_DEPENDS+= bar.42:${PORTSDIR}/x11/bar-base::42
# depend on variant `42' of the bar-base package
..endif
OK, so that's a lot of work, and I'm not exactly volunteering. But it
would be very nice to have.
-GAWollman
--
Garrett A. Wollman | [G]enes make enzymes, and enzymes control the rates of
wollman@lcs.mit.edu | chemical processes. Genes do not make ``novelty-
Opinions not those of| seeking'' or any other complex and overt behavior.
MIT, LCS, CRS, or NSA| - Stephen Jay Gould (1941-2002)
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Thu Jan 08, 2009 5:20 am | All times are GMT
|
|
Repair Bad Credit | Guitar Lessons | Debt Consolidation | Remortgages | Free SMS
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
*nix
»
BSD
»
FreeBSD
»
mail-lists
»
Architecture
