From FBSD_tips

Back to "Admin building blocks"

Or "How to keep your data safe, in a house of mirrors."

Contents 1 Goals 2 Labelling 3 Configuring 4 Failure 5 Recovery 6 Further options 7 Discussion

[edit] Goals

Now let's try something a little more complicated. I have a series of external drives that I gmirror and use rsync to copy running machines to (see Rsync mirror without root). So now let's glabel (see Keeping track of your drives with glabel) a pair of drives, and then gmirror them. I label them something fairly descriptive e.g. mxtrenc1 is Matxor enclosure 1, then put a small label on the drive with the same label. This makes it easy to track down and replace failures later.

[edit] Labelling

If you do not have glabel loaded on boot, do it now.

kldload geom_label

# glabel label mxtrenc1 ad10
# glabel label mxtrenc2 ad8
# gmirror label backup1 /dev/label/mxtrenc1 /dev/label/mxtrenc2
# newfs /dev/mirror/backup1
/dev/mirror/backup1: 95396.3MB (195371564 sectors) block size 16384, fragment size 2048
        using 520 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.
super-block backups (for fsck -b #) at:
 160, 376512, 752864, ...

[edit] Configuring

Because this drive is drive is only used as a backup, I will tune it for space.

# tunefs -o space -m 0 /dev/mirror/backup1
tunefs: minimum percentage of free space changes from 8% to 0%
tunefs: should optimize for space with minfree < 8%
tunefs: optimization preference changes from time to space

Now we can mount it :

# mount /dev/mirror/backup1 /mnt
# df /mnt
Filesystem          1K-blocks Used    Avail Capacity  Mounted on
/dev/mirror/backup1  94607292    4 94607288     0%    /mnt

Checking the status is easy.

#  gmirror status backup1
          Name    Status  Components
mirror/backup1  COMPLETE  label/mxtrenc2
                          label/mxtrenc1

[edit] Failure

Now, lets simulate a failure. We will remove a component (in the case of a real failure you may have to use ONLY the 'forget' if the drive can not spin up at all), then prepare a replacement and insert it (note that I have to prefix the provider with 'label/').

# gmirror deactivate backup1 label/mxtrenc1
# gmirror forget  backup1

We can see the array id degraded.

# gmirror status
          Name    Status  Components
mirror/backup1  DEGRADED  label/mxtrenc2

[edit] Recovery

Now label the replacement drive :

# glabel label mxtrenc3 ad6

And insert it.

# gmirror insert backup1 label/mxtrenc3

And check the status.

# gmirror status
          Name    Status  Components
mirror/backup1  DEGRADED  label/mxtrenc2
                          label/mxtrenc3 (2%)

And again a little while later

# gmirror status
          Name    Status  Components
mirror/backup1  COMPLETE  label/mxtrenc2
                          label/mxtrenc3

[edit] Further options

Of course there is no rule that you can only have pairs mirroring ...

# glabel label mxtrenc4 ad4
Test (/root) > gmirror insert backup1 label/mxtrenc4
Test (/root) > gmirror status
          Name    Status  Components
mirror/backup1  COMPLETE  label/mxtrenc2
                          label/mxtrenc3
                          label/mxtrenc4

[edit] Discussion

Something that can happen when the system is restarting from an abnormal stop, one where the gmirrored drives were not cleanly unmounted, is some serious contention for the drives being mirrored. This happen because the resync will kick off almost immediately upon bnooting, and the system's rc startup will kick off a bgfsk and then go on with the boot. One way to avoid this was pointed out to me (thanks jpaetzel!) in /etc/rc.conf. The setting background_fsck_delay="" is in seconds and will put off the bgfsk for that number of seconds. As the only error that bgfsk look for is inodes that are marked used that aren't, and then clear them, this is not a risky setting.

Gongo 16:55, 11 October 2007 (UTC)