Open/NetBSD have periodic(8) (well, actually /etc/perdiodic) scripts to backup fdisk(8) and disklabel(8) output. This is exceptionally useful in the event that you need to recover your system from an offsite storage tape backup to another facility on new (identical) hardware. Without disk partition / slice layout information, you have no idea how your system was originally partitioned. Of course, depending on whether you're using software or hardware RAID, you might not even know how your RAID was setup. We also need a way to extract software RAID configurations (vinnum, geom, gmirror, CCD ex.) to a config file reference. For example, RAIDFrame's raidctl(8) has a nice "-G" flag: "-G dev Generate the configuration of the RAIDframe device in a format suitable for use with the -c or -C options." There is one caveat: If you're using software RAID CCD, you have your ccd.conf(5) and the output of the component disks` bsdlabel(8) using my solution to determine where on the disk the CCD slice began. There's no way to backup hardware RAID configuration layouts until the OS has a unified RAID API (see OpenBSD's msic@ list for details on something coming in the next release there). Fix: I wrote an ugly little script for periodic to accomplish this here initially, until someone commits a higher quality one to the fbsd src tree: The script is viewable at the following URL as well: http://digitalfreaks.org/~lavalamp/220.backup-bsdlabels ------------- #!/bin/sh # # $FreeBSD: src/etc/periodic/daily/220.backup-bsdlablels****** # # If there is a global system configuration file, suck it in. # if [ -r /etc/defaults/periodic.conf ] then . /etc/defaults/periodic.conf source_periodic_confs fi case "$daily_backup_bsdlabels_enable" in [Yy][Ee][Ss]) bak=/var/backups disks=`sysctl -n kern.disks` if [ -z "$disks" ]; then echo '$daily_backup_disklabels_enable" is set but no disk probed by kernel.' \ "perhaps NFS diskless client." rc = 2 else for i in $disks; do # first order of business is to check for an existing backup-backup if [ -f $bak/fdisk.$i.bak ] ; then rc=1 echo "rotating $bak/fdisk.$i.bak" cp -p $bak/fdisk.$i.bak $bak/fdisk.$i.bak2 || rc=3 fi echo "backing up fdisk for $i" fdisk $i > "$bak/fdisk.$i.bak" 2>/dev/null || rc=3 # again exept now we have to get a list of patitions/slices # sparc64 can have...9 hopefully slices on a sunlabel? part_slices=$(echo /dev/${i}s[0-9]) for j in $(echo "$part_slices" | sed 's/\/dev\///'); do if [ -f $bak/disklabel.${j}.bak ] ; then rc=1 echo "rotating $bak/disklabel.${j}.bak" cp -p $bak/disklabel.${j}.bak $bak/disklabel.${j}.bak2 || rc=3 fi echo "backing up disklabel for ${j}" disklabel /dev/${j} > "$bak/disklabel.${j}.bak" 2>/dev/null || rc=3 done done fi;; *) rc=0;; esac How-To-Repeat: Sit around for a few weeks every year and brainstorm DRP (Disaster Recovery Planning) scnearios in a 100% FreeBSD environment.
On Wed, Sep 21, 2005 at 02:18:51AM +0000, Seklecki@freebsd.org wrote: S> >How-To-Repeat: S> Sit around for a few weeks every year and brainstorm DRP (Disaster Recovery Planning) scnearios in a 100% FreeBSD environment. I would argue that storing bsdlabel(8), fdisk(8) or other storage configuration information on the storage itself is the best idea. -- Totus tuus, Glebius. GLEBIUS-RIPN GLEB-RIPE
[...] > I would argue that storing bsdlabel(8), fdisk(8) or other storage > configuration information on the storage itself is the best idea. The submitter said: % This is exceptionally useful in the event that you need to recover % your system from an offsite storage tape backup to another facility % on new (identical) hardware. Personally I'm always trying to create a copy of fdisk/disklabel and backup them when install new machine but it is very easy to forget about that procedure when you add a disk or change its layout. I helped to other people to repair their disklabels several times already and from time to time see this question in FreeBSD maillists. There are several tools for partitions/slices restore in ports. Even one is under src/. Believe me, this information saves a lot of man hours and nerve and Net|OpenBSD have this functionality for a reason. -- Maxim Konovalov
On Wed, Sep 21, 2005 at 06:20:24PM +0400, Maxim Konovalov wrote: M> > I would argue that storing bsdlabel(8), fdisk(8) or other storage M> > configuration information on the storage itself is the best idea. M> M> The submitter said: M> M> % This is exceptionally useful in the event that you need to recover M> % your system from an offsite storage tape backup to another facility M> % on new (identical) hardware. M> M> Personally I'm always trying to create a copy of fdisk/disklabel and M> backup them when install new machine but it is very easy to forget M> about that procedure when you add a disk or change its layout. I M> helped to other people to repair their disklabels several times M> already and from time to time see this question in FreeBSD maillists. M> There are several tools for partitions/slices restore in ports. Even M> one is under src/. Believe me, this information saves a lot of man M> hours and nerve and Net|OpenBSD have this functionality for a reason. I know. I always save it, but outside the storage in question. -- Totus tuus, Glebius. GLEBIUS-RIPN GLEB-RIPE
On Tue, 2006-03-14 at 18:27 -0500, Tom Rhodes wrote: > On Mon, 13 Mar 2006 23:47:44 -0500 > "Brian A. Seklecki" <bseklecki@mx00.pub.collaborativefusion.com> wrote: I've updated this to work with $prefix/local/periodic./daily* and use echo(1) less, where possible, except in conjunction with sed(1). I've also addressed the "there's no sysctl with geom devices" with periodic.conf(5) variables: $daily_backup_bsdlabels_include $daily_backup_bsdlabels_exclude (regexp passed to sed(1)) So you can do: # enable daily_backup_bsdlabels_enable="YES" # don't backup unmoued CF->USB storage daily_backup_bsdlabels_exclude="da[0-9]" # You might die trying! daily_backup_bsdlabels_xtra_disks="mirror/gm0" http://digitalfreaks.org/~lavalamp/220.backup-bsdlabels Still ugly as hell but it works! ~BAS PS our bourne shell is strange. part_slices=/dev/ad0s[0-9] = works disk=ad0 part_slices=/dev/${disk}s[0-9] = doesn't work so: part slices=$(echo /dev/${disk}s[0-9]) = ugly, but works. ~BAS > echo(1) has to go in this code. > > What about printf(1)? > -- Brian A. Seklecki <bseklecki@collaborativefusion.com> Collaborative Fusion, Inc.
Is there any progress to integrate this patch in to base system (6.x or 7.x)? It seems useful to me. I was thinking to integrate similar feature in to my own backup scripts, but start googling first and find this periodic script. I think there is nothing bad on backuping disk layout on filesystem itself, because filesystem is (could be) easily backed up by tar, rsync, dump or any other backup system, so one then have disk layout (fdisk + bsdlabel) stored in the backup archive with data. The last note - I can not find updated version of 220.backup-bsdlabels with daily_backup_bsdlabels_xtra_disks announced variable (I am using gmirror on almost all my servers) Miroslav Lachman
NetBSD and OpenBSD have had that code in place for 10 years. Not sure why we dont have it yet. The advent of an enterprise class F/OSS backup system (Bacula) really emphasizes the need for this. ~BAS On Mon, 9 Jul 2007, Miroslav Lachman wrote: > Is there any progress to integrate this patch in to base system (6.x or 7.x)?
On Thu, 27 Sep 2007, 15:50-0000, Brian A. Seklecki wrote: > NetBSD and OpenBSD have had that code in place for 10 years. Not > sure why we dont have it yet. The advent of an enterprise class > F/OSS backup system (Bacula) really emphasizes the need for this. > Could you please point me out where is the code in NetBSD? -- Maxim Konovalov
as discussed in this thread http://lists.freebsd.org/pipermail/freebsd-current/2009-January/002389.html the patch needs to be updated and improved in order to get committed to CURRENT. also netbsd/openbsd don't seem to be using this script any longer.
State Changed From-To: open->suspended Script needs to be updated to work on -current. To submitter: sorry that it has sat in GNATS without action for so long.
State Changed From-To: suspended->open Back to open, since with 'gpart backup' there's a new and easy way to backup the partition table.
Responsible Changed From-To: freebsd-bugs->freebsd-geom Over to geom folks. Maybe they can come up with a periodic(8) script to take care of this ancient PR.
I am giving up on this PR. 12 years are too much. I am providing my local scripts for users wanting to have this kind of backups. You can put them in to /usr/local/etc/periodic/daily/ (or /etc/periodic/daily/) Then you can set some variables in /etc/periodic.conf daily_backup_gpart_enable="YES" daily_backup_gpart_verbose="YES" daily_backup_gpart_exclude="" ## regexp pattern for grep -E daily_backup_gmirror_enable="YES" daily_backup_zfs_enable="YES" daily_backup_zfs_verbose="YES" daily_backup_zfs_props_enable="YES"
Created attachment 187996 [details] Backup of disk partitions layout daily_backup_gpart_enable="YES" daily_backup_gpart_verbose="YES" daily_backup_gpart_exclude="" ## regexp pattern for grep -E
Created attachment 187997 [details] Backup of gmirror list detailed info daily_backup_gmirror_enable="YES"
Created attachment 187998 [details] Backup of zpool info, zfs list and zfs properties daily_backup_zfs_enable="YES" daily_backup_zfs_verbose="YES" daily_backup_zfs_props_enable="YES"
I had never seen this PR before. I'll take it and work on getting it in to the tree.
batch change: For bugs that match the following - Status Is In progress AND - Untouched since 2018-01-01. AND - Affects Base System OR Documentation DO: Reset to open status. Note: I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.
Exactly what I was expecting - six month later and nothing happened. So please pretty please close this PR to not scare potential new submitters that their work will lay untouched for 13 years... This is really scary example of how things doesn't work in FreeBSD.
During the July 2020 Bugathon, a volunteer refreshed the first of these patches, extending it to include support for backing up the EFI System Partition. https://reviews.freebsd.org/D25628 There is some additional feedback in the review, since the man pages and etc/defaults/periodic.conf also need to be updated. Miroslav: Sorry for the lack of progress, there are 10,000 open PRs, it is far too easy for one to go unnoticed.
A commit references this bug: Author: allanjude Date: Sat Jul 11 20:53:32 UTC 2020 New revision: 363110 URL: https://svnweb.freebsd.org/changeset/base/363110 Log: Add a periodic script to backup the partition table and boot code Optionally, alert you if the contents change from the previous backup PR: 86388 Submitted by: Rob Fairbanks <rob.fx907@gmail.com>, Miroslav Lachman <000.fbsd@quip.cz> (Original Version) MFC after: 4 weeks Relnotes: yes Sponsored by: Klara Inc. Event: July 2020 Bugathon Differential Revision: https://reviews.freebsd.org/D25628 Changes: head/share/man/man5/periodic.conf.5 head/usr.sbin/periodic/etc/daily/221.backup-gpart head/usr.sbin/periodic/periodic.conf
(In reply to Allan Jude from comment #19) Are there any good reason to not include scripts to backup gmirror, ZFS or other layout info too?
(In reply to Miroslav Lachman from comment #21) Hey Miroslav, I have submitted patches that include man page and periodic.conf(5) changes for the gmirror and the zfs scripts. They can be reviewed at: https://reviews.freebsd.org/D25631 - gmirror periodic script patch https://reviews.freebsd.org/D25638 - zfs periodic script patch Any comments are welcome. Thanks, Rob
(In reply to Miroslav Lachman from comment #21) I expect to have time to review and land the additional 2 patches in the next few days
(In reply to Rob from comment #22) Thank you! I am glad it will finally find a way in to the base.
There is a report that the script is not being installed. It seems we need to add the scripts to the Makefile in the corresponding directory, like: usr.sbin/periodic/etc/daily
Allan, Here's the fix for installing 221.backup-gpart properly. https://reviews.freebsd.org/D25653 I'll update the other two revisions as well.
I forgot to tag r363169 with the PR
A commit references this bug: Author: rew Date: Fri Nov 6 22:58:32 UTC 2020 New revision: 367436 URL: https://svnweb.freebsd.org/changeset/base/367436 Log: Add a periodic script to backup output generated from `zfs list`, `zfs get`, `zpool list`, and `zpool get` commands. Disabled by default. PR: 86388 Submitted by: Miroslav Lachman <000.fbsd@quip.cz> Reviewed by: allanjude, 0mp Approved by: allanjude (mentor) MFC after: 4 weeks Event: July 2020 Bugathon Differential Revision: https://reviews.freebsd.org/D25638 Changes: head/share/man/man5/periodic.conf.5 head/usr.sbin/periodic/etc/daily/223.backup-zfs head/usr.sbin/periodic/etc/daily/Makefile head/usr.sbin/periodic/periodic.conf
A commit references this bug: Author: rew Date: Sat Nov 7 04:15:25 UTC 2020 New revision: 367443 URL: https://svnweb.freebsd.org/changeset/base/367443 Log: Add a periodic script to backup output generated from `gmirror list`. Disabled by default. PR: 86388 Submitted by: Miroslav Lachman <000.fbsd@quip.cz> Reviewed by: allanjude, gbe Approved by: allanjude (mentor) MFC after: 4 weeks Event: July 2020 Bugathon Differential Revision: https://reviews.freebsd.org/D25631 Changes: head/share/man/man5/periodic.conf.5 head/usr.sbin/periodic/etc/daily/222.backup-gmirror head/usr.sbin/periodic/etc/daily/Makefile head/usr.sbin/periodic/periodic.conf
A commit references this bug: Author: gbe Date: Thu Dec 10 09:55:05 UTC 2020 New revision: 368507 URL: https://svnweb.freebsd.org/changeset/base/368507 Log: MFC r363110 (by allanjude): Add a periodic script to backup the partition table and boot code Optionally, alert you if the contents change from the previous backup PR: 86388 Submitted by: Rob Fairbanks <rob.fx907@gmail.com>, Miroslav Lachman <000.fbsd@quip.cz> (Original Version) Relnotes: yes Sponsored by: Klara Inc. Event: July 2020 Bugathon Differential Revision: https://reviews.freebsd.org/D25628 Changes: _U stable/12/ stable/12/share/man/man5/periodic.conf.5 stable/12/usr.sbin/periodic/etc/daily/221.backup-gpart stable/12/usr.sbin/periodic/periodic.conf
A commit references this bug: Author: gbe Date: Thu Dec 10 13:25:46 UTC 2020 New revision: 368514 URL: https://svnweb.freebsd.org/changeset/base/368514 Log: MFC r367436 and r367443 by rew: r367436 Add a periodic script to backup output generated from `zfs list`, `zfs get`, `zpool list`, and `zpool get` commands. Disabled by default. r367443 Add a periodic script to backup output generated from `gmirror list`. Disabled by default. PR: 86388 Submitted by: Miroslav Lachman <000 dot fbsd at quip dot cz> Reported by: Seklecki <Seklecki at FreeBSD dot org> Reviewed by: allanjude, 0mp, gbe Event: July 2020 Bugathon Differential Revision: https://reviews.freebsd.org/D25638 Differential Revision: https://reviews.freebsd.org/D25631 Changes: _U stable/12/ stable/12/share/man/man5/periodic.conf.5 stable/12/usr.sbin/periodic/etc/daily/222.backup-gmirror stable/12/usr.sbin/periodic/etc/daily/223.backup-zfs stable/12/usr.sbin/periodic/etc/daily/Makefile stable/12/usr.sbin/periodic/periodic.conf
A commit references this bug: Author: gbe Date: Thu Dec 10 13:32:52 UTC 2020 New revision: 368515 URL: https://svnweb.freebsd.org/changeset/base/368515 Log: MFC r363169 by allanjude: Actually install the new 221.backup-gpart periodic script PR: 86388 Submitted by: Rob Fairbanks <rob dot fx907 at gmail dot com> Reported by: Michael Butler <imb at protected-networks dot net> Sponsored by: Klara Inc. Changes: _U stable/12/ stable/12/usr.sbin/periodic/etc/daily/Makefile