|
|
|
|
|
|
| Author |
Message |
andreas@klemm.apsfilter.o
*nix forums beginner
Joined: 03 Jul 2006
Posts: 4
|
 Posted: Thu Jul 20, 2006 7:00 am Post subject:
Re: Simple BSD Backup System
|
|
|
In article <7hmvg.28662$IU2.19148@newsfe2-win.ntli.net>,
John Richards <john.richards280@tesco.net> writes:
| Quote: | Read carefully the manpages of dump / mksnap_ffs (handbook sections
17.11 and forward). Something like :
dump -L0f- /dev/*your partition* | gzip > *the dump destination*
Will make a compressed image of a partition. Automating it and making
it simpler involves a few lines of shell script.
Many thanks. I am beginning to see that dump is actually a very flexible
and powerful system - once one gets the syntax sorted out. I tried the
|
Elizabeth Zwicky analyzed different ways of making backup
under Unix. She uses a perl script to "torture test" backup
programs like dump, tar, etc under different flavours of Unix.
This script created maximum allowed number of subdirectories of
the Unix filesystems, created files with maximum allowed number
of characters in name. Then also files with a "hole" in it and
stuff like that.
It turned out that dump is "optimized" for backing up everything
that can happen as file or directory under a ufs (unix file system,
aka Berkeley fast file system).
So in general, dump/restore is the way to go on BSD or better
said, machines with ufs / ffs.
| Quote: | above formula (with partitions altered mutatis mutandis of course) and
it indeed produced a dump file in the destination directory. It is only
14Mb in size though, which cannot be right for a complete installation
including KDE and the full ports collection, even for a compressed file!
|
If you creeated different file systems and did a dump of "/" perhaps,
then you perhaps only dumped the root fs ? That would fit this size ...
| Quote: | I am at a loss what has gone wrong though. It also only took 9 seconds,
which again cannot be right. Even my beloved Partimage would take a good
5 minutes for a partition this size, and produce a file of at least
600Mb. I dare not try "restore" as it would certainly corrupt an at
present good setup.
|
You can simply display whats in the backup using this command:
gzcat dumpfile.gz | restore tvf -
You could also use restore's -i functionality (interactive).
Instead of directly restoring, it gives you a prompt and
you can traverse the backup with commands like
cd, ls, to see, what is in and to selectively add and delete
files to be restored.
Another thing is that restore restores file systems relative
to your current working directory. So if you have a big filesystem,
then you can restore there. Or if you have a big memory filesystem
unter /tmp, you can restore even there for a test.
Here the script I use to make backups under my trash partition
/data in /data/backup:
rw-r--r-- 1 root wheel 23237158 Jul 19 01:00 dump-_.gz
-rw-r--r-- 1 root wheel 23202730 Jul 3 01:00 dump-_.old.gz
-rw-r--r-- 1 root wheel 11629204128 Jul 19 02:10 dump-_home.gz
-rw-r--r-- 1 root wheel 11543366725 Jul 3 02:50 dump-_home.old.gz
-rw-r--r-- 1 root wheel 377411432 Jul 19 01:21 dump-_usr.gz
-rw-r--r-- 1 root wheel 400748335 Jul 3 01:26 dump-_usr.old.gz
-rw-r--r-- 1 root wheel 318620677 Jul 19 01:24 dump-_usr_X11R6.gz
-rw-r--r-- 1 root wheel 328937640 Jul 3 01:31 dump-_usr_X11R6.old.gz
-rw-r--r-- 1 root wheel 1141368191 Jul 19 01:34 dump-_usr_local.gz
-rw-r--r-- 1 root wheel 1165860354 Jul 3 01:46 dump-_usr_local.old.gz
-rw-r--r-- 1 root wheel 870489019 Jul 19 01:08 dump-_var.gz
-rw-r--r-- 1 root wheel 803371721 Jul 3 01:09 dump-_var.old.gz
#! /bin/sh
#Filesystem 1K-blocks Used Avail Capacity Mounted on
#/dev/ad6s1a 1012974 54440 877498 6% /
#devfs 1 1 0 100% /dev
#/dev/ad4s1 12277696 5632288 6645408 46% /dosc
#/dev/ad6s2a 1012974 63654 868284 7% /amd64
#/dev/ad6s2e 2024690 772702 1090014 41% /amd64/usr
#/dev/ad6s2d 1012974 214 931724 0% /amd64/var
#/dev/ad4s3d 54145634 38843314 10970670 78% /data
#/dev/ad6s3h 39601806 4 36433658 0% /export
#/dev/ad6s3g 25385516 13242336 10112340 57% /home
#/dev/ad6s1d 4056022 2159094 1572448 58% /usr
#/dev/ad6s3e 4058062 351322 3382096 9% /usr/X11R6
#/dev/ad6s3f 6090094 1149792 4453096 21% /usr/local
#/dev/ad6s3d 4058062 1120818 2612600 30% /var
#/dev/ad4s1 12277696 5632288 6645408 46% /dosc
#/dev/ad4s5 1018096 614472 403624 60% /dosf
#/dev/ad4s6 28653984 23503968 5150016 82% /dosg
#/dev/ad4s7 28653984 14846672 13807312 52% /dosh
#/dev/ad4s8 28670032 7517776 21152256 26% /dosi
#procfs 4 4 0 100% /proc
#linprocfs 4 4 0 100% /usr/compat/linux/proc
#devfs 1 1 0 100% /var/named/dev
#/dev/md0 126702 368 116198 0% /tmp
PATH=/bin:/sbin:/usr/bin:/usr/sbin
export PATH
FS="/ /var /usr /usr/X11R6 /usr/local /home"
DUMP="/sbin/dump 0Luf -"
DEST=/data/backup
[ -d $DEST ] || /bin/mkdir -p $DEST
trap '[ "$dump_in_progress" -eq "yes" ] && /bin/rm -f ${DEST}/dump-${name}.gz && /bin/mv ${DEST}/dump-${name}.old.gz ${DEST}/dump-${name}.gz; exit' 1 9 15
for i in $FS
do
[ -d $i/.snap ] || /bin/mkdir -p $i/.snap
/usr/sbin/chown root:operator $i/.snap
/bin/chmod 0770 $i/.snap
name=`echo $i | /usr/bin/sed -e "s=/=_=g"`
# ... root wheel 117243904 Sep 1 08:44 dump-_var.gz
# ... root wheel 560117563 Sep 1 01:08 dump-_var.old.gz
if [ -f ${DEST}/dump-${name}.gz ]
then
/bin/mv ${DEST}/dump-${name}.gz ${DEST}/dump-${name}.old.gz
fi
dump_in_progress=yes
${DUMP} ${i} \
| /usr/bin/gzip \
| /bin/dd \
| Quote: | ${DEST}/dump-${name}.gz
if [ $? -eq 0 ]; then |
# keep backup
# rm ${DEST}/dump-${name}.old.gz
else
# in case of error restore old backup
mv ${DEST}/dump-${name}.old.gz ${DEST}/dump-${name}.gz
fi
dump_in_progress=no
done
Gruß
Andreas ///
--
http://www.64bits.de
http://www.apsfilter.org
http://people.FreeBSD.org/~andreas |
|
| Back to top |
|
 |
andreas@klemm.apsfilter.o
*nix forums beginner
Joined: 03 Jul 2006
Posts: 4
|
| Posted: Thu Jul 20, 2006 7:11 am Post subject:
Re: Simple BSD Backup System
|
|
|
In article <e9l7au$ntd$1@news5.zwoll1.ov.home.nl>,
nobody@nowhere.invalid (Peter Boosten) writes:
| Quote: | John Richards <john.richards280@tesco.net> wrote:
Yes, they can, but they are HUGE! They are clearly not compressed at
all, and seem to be effectively, if not theoretically, of every byte in
the partition, not just the used ones. I find they are regularly well
over 4G in size, and take several hours to create and to restore - and
they do not seem very reliable into the bargain. But if there is nothing
else, and a backup is absolutely essential, then they are better than
nothing, I suppose. Personally, I think I would prefer just to
re-install FreeBSD! - particularly as I keep all my important data in a
separate, shared, MSDOS partition anyway.
(I am talking of Acronis 8. I don't know if later versions are any
better. Do you?)
Well, I can confirm the 'huge'-part. But hey, they saved me once when
I crashed my disk.
I rsync my important data every night to another box, works good enough
for me.
|
And don't forget to save your partition table with dump.
In former versions than Acronis 9 Acronis was unable to restore
Sector 0 of disk if you did not choose backing up the whole disc.
Since I had some huge "trash" partitions, saving the whole disc
never was an option for me. When I lost sector 0 while fiddeling
around with another boot manager I lost it.
Was kind of hard work, to get partition table restored.
But I got it managed with excel and some freebsd partition
finding tools in the ports collection.
So better save sector 0 separately and put it on CD-ROM
and Harddisk as an additional safety belt.
I bugged Acronis so much on that, that the new version 9 now
contains the MBR save / restore part.
And yes, they make a binary copy of the whole BSD partition,
which they do compress, but you also save unused space with it.
I also bugged them for builtin BSD filesystem support like
they did for Linux. But it seems that there are too few whiners
on that  )
Andreas ///
--
http://www.64bits.de
http://www.apsfilter.org
http://people.FreeBSD.org/~andreas |
|
| Back to top |
|
|
John Richards
*nix forums beginner
Joined: 03 Sep 2005
Posts: 23
|
| Posted: Thu Jul 20, 2006 8:30 am Post subject:
Re: Simple BSD Backup System
|
|
|
| Quote: | If you creeated different file systems and did a dump of "/" perhaps,
then you perhaps only dumped the root fs ? That would fit this size ...
|
I found this whole message enormously helpful. I only quote this small
snippet, as it is the only part I can constructively comment on directly
at this stage. I think you are quite right. I am now beginning to get
the idea of general (simple) dump syntax, and have been able to dump all
the /, /var and /usr slices. Clearly the first attempt which I quoted
was, for reasons I do not yet fully understand, just the "a" slice.
This and a few other helpful messages have opened the doors for me in
this subject, and I am very grateful. I don't see how anyone, on first
coming to BSD is meant to pick up the essential syntax and primary
function of "dump" from the Handbook exampes -
# /sbin/rdump 0dsbfu 54000 13000 126 komodo:/dev/nsa8 /dev/da0a 2>&1
and
# /sbin/dump -0uan -f - /usr | gzip -2 | ssh -c blowfish \
targetuser@targetmachine.example.com dd
of=/mybigfiles/dump-usr-l0.gz
Most of the Handbook is extremely well written, clear and precise, but
here I think a few SIMPLE examples, like the one I was helpfully offered
earlier
dump -L0f- /dev/*your partition* | gzip > *the dump destination*
would be far more appropriate in a primary handbook.
I am not trying to rubbish the Handbook, and even less FreeBSD. This
meant as CONSTRUCTIVE criticism, from the viewpoint of a newcomer to
BSD. Those of you who have degrees in IT and rub shoulders every day
with like-educated professionals may not always appreciate the
difficulties for those whose specialities may lie in other departments,
but who nonetheless want to take an intelligent interest in, and
constructive use of BSD. I can see that a whole book could
constructively be written on "dump", but the primary handbook should, I
think all would agree, deal with simple basics, not specialised uses,
simplest syntaxes, not highly elaborate ones.
I repeat. This is not meant negatively. |
|
| Back to top |
|
|
Michel Talon
*nix forums Guru
Joined: 21 Feb 2005
Posts: 557
|
| Posted: Thu Jul 20, 2006 9:21 am Post subject:
Re: Simple BSD Backup System
|
|
|
John Richards <john.richards280@tesco.net> wrote:
| Quote: |
Most of the Handbook is extremely well written, clear and precise, but
here I think a few SIMPLE examples, like the one I was helpfully offered
earlier
dump -L0f- /dev/*your partition* | gzip > *the dump destination*
would be far more appropriate in a primary handbook.
|
In fact i find all the man pages should have significative simple and more
complex *examples* of how to use the commands. This is far from being the case
For example the dump command has just one example, which is far from
explaining significant usages, and is not even in an "examples" section.
Clearly man pages are written by programmers, whose main aim is to document
the options they have programmed, and not by users. This being said the
FreeBSD man pages are among the good ones ...
--
Michel TALON |
|
| Back to top |
|
|
F. Senault
*nix forums beginner
Joined: 28 Apr 2005
Posts: 30
|
| Posted: Thu Jul 20, 2006 9:25 am Post subject:
Re: Simple BSD Backup System
|
|
|
Le 20 juillet à 10:36, John Richards a écrit :
| Quote: | This and a few other helpful messages have opened the doors for me in
this subject, and I am very grateful. I don't see how anyone, on first
coming to BSD is meant to pick up the essential syntax and primary
function of "dump" from the Handbook exampes -
|
Please note that the handbook clearly notes this a bit above the
examples :
| Note: If you use dump on your root directory, you would not back up
| /home, /usr or many other directories since these are typically mount
| points for other file systems or symbolic links into those file systems.
I believe the handbook is pretty well written if you take time to read
it from beginning to end. If you jump into it without a good knowledge
of the system as a whole, you'll miss bits of information that are
explained before and not restated.
By the way, you should become a maniac of the man command too. BSD
systems' man pages are notoriously very well maintained.
/.../
| Quote: | I am not trying to rubbish the Handbook, and even less FreeBSD. This
meant as CONSTRUCTIVE criticism, from the viewpoint of a newcomer to
BSD. Those of you who have degrees in IT and rub shoulders every day
with like-educated professionals may not always appreciate the
difficulties for those whose specialities may lie in other departments,
but who nonetheless want to take an intelligent interest in, and
constructive use of BSD.
|
FWIW, I'm a completely self taught IT professional, whose native
language is not English. I managed to learn Linux by myself, then jump
ships to FreeBSD with the handbook under my arm. All it takes is some
time, a box to play with, and, I believe, some patience - to read the
books, to read the manpages, and to experiment.
| Quote: | I can see that a whole book could
constructively be written on "dump", but the primary handbook should, I
think all would agree, deal with simple basics, not specialised uses,
simplest syntaxes, not highly elaborate ones.
|
Yes, I can quite agree with that. Now, all we have to do is to submit a
patch with the correct examples... 
| Quote: | I repeat. This is not meant negatively.
|
BTW, there are a few better examples in the FAQ, section 9. Take a look
at this, for instance :
http://www.freebsd.org/doc/faq/disks.html#NEW-HUGE-DISK
(BTW, does someone understand why the commands use restore xf and not
restore rf ? I've always used the "magic" invocation of dump -0af- / |
restore -rf- myself...)
Fred
--
...the variable PI can be given that value [3.141592653589793] with a
DATA statement... This simplifies the modifying of the program, should
the value of PI ever change. (SDS Sigma series Fortran manual) |
|
| Back to top |
|
|
Jason Bourne
*nix forums Guru Wannabe
Joined: 22 Feb 2005
Posts: 135
|
| Posted: Thu Jul 20, 2006 9:31 am Post subject:
Re: Simple BSD Backup System
|
|
|
Bruce Burden wrote:
| Quote: | Jason Bourne <j_bourne_treadstone@hotmail.com> wrote:
:
: Start with something simple such as just backing up the "/" partition.
|
What I was going after here was just getting the syntax down for one
partition by way of example. The other comment about looking for the other
partitions in fstab may not have been clear enough either though.
| Quote: | :
Just as an aside, "/" may not be that simple if Linux
--> BSD folks follow the (default) Linux model of partitions.
ie, SUSE gives you / and swap.
Well, I guess it is simple, and familiar to Windoze, but
having 1 monster partition makes my skin crawl.
|
I agree totally. Some setup(s) need to be partitioned differently. A desktop
end user may not need the same size /var as a mail or database server.
Flexibility is a good thing!
-Jason |
|
| Back to top |
|
|
Ulrich Spoerlein
*nix forums beginner
Joined: 11 Nov 2004
Posts: 16
|
| Posted: Thu Jul 20, 2006 7:05 pm Post subject:
Re: Simple BSD Backup System
|
|
|
Just some random comments I'm gonna throw into the discussion.
Greg Hennessy wrote:
| Quote: | There are cases where it's not the best method of backup, but those
cases, as far as I've seen, are fairly specialized.
Databases are the usual culprits.
|
Yes, I faced this a few days ago, and *NO*, databases are not "fairly
specialized".
For now, I simple dump the live filesystems through snapshots, yet
ideally one should to the following:
1. shutdown database
2. mksnap_ffs (hope it finishes quickly!)
3. start database
4. dump the snapshot
5. remove the snapshot
Since the DB is accessed at least every 5 minutes for roughly 2 minutes,
the 3 minute window is fairly hard to hit when dumping from crontab(
:(
A cool DB-feature would be a command that would lock the DB, flush all
its internal buffers, and lock the DB for an additional 3-5 seconds.
Within that locking-frame one could start the snapshotting process. This
should guarantee valid and consistent database dumps. But on the other
hand, we have ACID so this point should be mood.
| Quote: | I think dump is probably the best way to back up a Linux system, too.
I thought that linus waved his magic wand years ago and said dump was
verboten under penalty of banishment from the gnu garden ?
|
His famous post showed that he is not very knowledgable about file
systems and backups (Remember the "Tapes are for whimps"...). AFAIR he
rambled about how running dump( on an RW-mounted filesystem sucked ...
d'oh of course! That's what snapshots are for, after all! Bless UFS!
| Quote: | I don't understand the obsession with getting bit-for-bit copies of
filesystem, even to the point of including the unused space.
The only time I would do so is when moving a complete installation en bloc
(pun intended), disk to disk with G4U or a bootable Ghost CD. It's the
quickest and most painless for that sort of task when one has physical
access to the disks.
|
Only feasible if you have really enough space for your backup data. It
bites you, if the target disk is slightly smaller. You can't do levelled
dumps, etc.
Ulrich Spoerlein
--
A: Yes.
| Quote: | Q: Are you sure?
A: Because it reverses the logical flow of conversation.
Q: Why is top posting frowned upon? |
|
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Fri Nov 21, 2008 10:08 pm | All times are GMT
|
|
Loans | Proxy | Car Loan | Record Internet Radio with Tags | Mortgages
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
*nix
»
BSD
»
FreeBSD
