ZFS

Patrick Stein @jollyjinx

Why should I care ?

• HFS is journaled
• TimeMachine covers my back
• I'm using git
• I'm using offsite backups

Have you ever had:

• Wrong colors in pictures ?
• Errors with diskutil verify ?
  
• TimeMachine problems ?
  

ZFS Features

ZPool

• Consists of zvolumes (usually a single disk)
• Attach disks when you have them
• Attach/Detach mirrors, logs, cache devices on the fly
• Support for:
  • Mirroring
  • RAID-Z 1-3
    • write hole
    • variable striping size
    • swap in larger drives to get more space
  • Spare drives
  • Seperate Log/Cache devices

ZFS Filesystems

• Create/Destroy filesystems as you go
• Quota for filesystems
• Snapshots
  • State in Time
  • Send/receive snapshot
  • Send/receive incremental snapshot
• Inheritance of attributes like:
  ditto,dedup,compression (lzjb,gzip),checksum,case sensitivity

ZFS Commands

• zpool
  • create/destroy
  • attach/detach
  • status
  • list
  • iostat
• zfs
  • create fs/snapshot
  • destroy fs/snapshot
  • set/get attribute
  • send/receive
• zstat
• zdb

ZFS on Mac

• Get it from GetGreenBytes.com
• Either use the SystemPreferencePane or:
• Format a disk and create a pool:

  #diskutil eraseDisk 'ZFS Pool' '%noformat%' /dev/rdisk2
  #zpool create tank /dev/disk2s2
  

• Done !

Mac Specific

• AFP problem in Lion/MTL use liberate AFP
• TimeMachine backup onto a ZFS Volume
• TimeMachine style backups
• autoscrub

• Mac ZFS Links:
  http://www.getgreenbytes.com/zevo/
  https://github.com/joshado/liberate-applefileserver
  https://github.com/jollyjinx/ZFS-TimeMachine
  https://github.com/jollyjinx/ZFS-TimeMachine#timemachine-backups-to-zfs-volumes
  
• General ZFS Info:
  http://en.wikipedia.org/wiki/ZFS
  https://blogs.oracle.com/bonwick/entry/zfs_dedup
  https://blogs.oracle.com/bonwick/entry/raid_z
  https://blogs.oracle.com/bonwick/en_US/entry/zfs_end_to_end_data
  https://blogs.oracle.com/bonwick/entry/smokin_mirrors
  https://blogs.oracle.com/bonwick/entry/zfs_block_allocation
  https://blogs.oracle.com/elowe/entry/zfs_saves_the_day_ta
  https://blogs.oracle.com/ahrens/entry/is_it_magic
  http://web.archive.org/web/20060428092023/http://www.sun.com/2004-0914/feature/
  http://www.slideshare.net/esproul/zfs-nuts-and-bolts
  http://hub.opensolaris.org/bin/download/Community+Group+zfs/docs/ondiskformat0822.pdf
  http://www.tumfatig.net/20111026/zfs-explained-and-demoed-briefly/
  http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=6&ved=0CFgQFjAF&url=http%3A%2F%2Fconstantin.glez.de%2Fblog%2F2010%2F07%2Fsolaris-zfs-synchronous-writes-and-zil-explained&ei=Gkt0UJz_E4bLsgactIDQDw&usg=AFQjCNEW1ig1d1IXs5JxETdgSCHd88L4ng
  
• Jeff Bonwick & Bill Moore, ZFS: The last word in Filesystems part1-3:
  http://www.youtube.com/watch?v=NRoUC9P1PmA
  http://www.youtube.com/watch?v=TwCXVp_u86o
  http://www.youtube.com/watch?v=ybjdAEUfXzw
  
• Jeff Bonwick & Bill Moore, ZFS: The next word in Filesystems part1-6:
  http://www.youtube.com/watch?v=Spd5qwGz35k
  http://www.youtube.com/watch?v=YxjCM3G6qyk
  http://www.youtube.com/watch?v=dW6eRfCpGHg
  http://www.youtube.com/watch?v=IBYHonrYutM
  http://www.youtube.com/watch?v=YEO-I5R_5uQ
  http://www.youtube.com/watch?v=pJl_cNDz-Fk
  

Hands on - Create a pool

# zpool status
no pools available

# mkdir /tmp/test
# mkfile 100m /tmp/test/a
# zpool create tank /tmp/test/a

# zpool status
  pool: tank
 state: ONLINE
 scan: none requested
config:

	NAME           STATE     READ WRITE CKSUM
	tank           ONLINE       0     0     0
	  /tmp/test/a  ONLINE       0     0     0

errors: No known data errors
# zpool list
NAME    SIZE   ALLOC    FREE     CAP  HEALTH  ALTROOT
tank  95.5Mi  77.5Ki  95.4Mi      0%  ONLINE  -

Hands on - Attributes

# cp /sbin/* /Volumes/tank/
# zpool list
NAME    SIZE   ALLOC    FREE     CAP  HEALTH  ALTROOT
tank  95.5Mi  3.83Mi  91.7Mi      4%  ONLINE  -

# rm -rf /Volumes/tank/*
zsh: sure you want to delete all the files in /Volumes/tank [yn]? y
# zpool list
NAME    SIZE   ALLOC    FREE     CAP  HEALTH  ALTROOT
tank  95.5Mi  5.32Mi  90.2Mi      5%  ONLINE  -

# sync
# zpool list
NAME    SIZE   ALLOC    FREE     CAP  HEALTH  ALTROOT
tank  95.5Mi   327Ki  95.2Mi      0%  ONLINE  -

# zfs set compression=gzip-9 tank
# cp /sbin/* /Volumes/tank/
# zpool list
NAME    SIZE   ALLOC    FREE     CAP  HEALTH  ALTROOT
tank  95.5Mi  1.20Mi  94.3Mi      1%  ONLINE  -

Hands on - Create a mirror

# mkfile 100m /tmp/test/b
# zpool attach tank /tmp/test/a /tmp/test/b
# zpool status
  pool: tank
 state: ONLINE
 scan: resilvered 1.83Mi in 0h0m with 0 errors on Thu Oct 11 13:22:33 2012
config:

	NAME             STATE     READ WRITE CKSUM
	tank             ONLINE       0     0     0
	  mirror-0       ONLINE       0     0     0
	    /tmp/test/a  ONLINE       0     0     0
	    /tmp/test/b  ONLINE       0     0     0

errors: No known data errors

Hands on - Destroy device data

# zpool export tank
# dd if=/dev/random of=/tmp/test/b count=10 bs=1m conv=notrunc
10+0 records in
10+0 records out
10485760 bytes transferred in 1.722770 secs (6086570 bytes/sec)

# zpool import -d /tmp/test tank
# zpool status
  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
 scan: scrub repaired 0 in 0h0m with 0 errors on Thu Oct 11 13:35:17 2012
config:

	NAME                     STATE     READ WRITE CKSUM
	tank                     ONLINE       0     0     0
	  mirror-0               ONLINE       0     0     0
	    /private/tmp/test/a  ONLINE       0     0     0
	    /private/tmp/test/b  ONLINE       0     0    11

errors: No known data errors

Hands on - Resilvering

# zpool scrub tank
# zpool status
  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
 scan: scrub repaired 1.58Mi in 0h0m with 0 errors on Thu Oct 11 13:38:15 2012
config:

	NAME                     STATE     READ WRITE CKSUM
	tank                     ONLINE       0     0     0
	  mirror-0               ONLINE       0     0     0
	    /private/tmp/test/a  ONLINE       0     0     0
	    /private/tmp/test/b  ONLINE       0     0   135

errors: No known data errors
# zpool clear tank

Hands on - Resilvering check

# zpool scrub tank
# zpool status
  pool: tank
 state: ONLINE
 scan: scrub repaired 1.59Mi in 0h0m with 0 errors on Thu Oct 11 14:54:24 2012
config:

	NAME                     STATE     READ WRITE CKSUM
	tank                     ONLINE       0     0     0
	  mirror-0               ONLINE       0     0     0
	    /private/tmp/test/a  ONLINE       0     0     0
	    /private/tmp/test/b  ONLINE       0     0     0

errors: No known data errors

Hands on - Destroying data 1

# zpool detach tank /private/tmp/test/b
# zpool export tank
# dd if=/dev/random of=/tmp/test/a count=10 bs=1m conv=notrunc
10+0 records in
10+0 records out
10485760 bytes transferred in 1.721552 secs (6090877 bytes/sec)
# zpool import -d /tmp/test tank

# zpool import -d /tmp/test tank
# zpool status
  pool: tank
 state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
 scan: scrub repaired 1.59Mi in 0h0m with 0 errors on Thu Oct 11 14:54:24 2012
config:

	NAME                   STATE     READ WRITE CKSUM
	tank                   ONLINE       0     0    15
	  /private/tmp/test/a  ONLINE       0     0    72

errors: 15 data errors, use '-v' for a list

Hands on - Destroying data 2

# zpool scrub tank
# sudo zpool status -v
  pool: tank
 state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
 scan: scrub repaired 24.5Ki in 0h0m with 87 errors on Thu Oct 11 15:01:44 2012
config:

	NAME                   STATE     READ WRITE CKSUM
	tank                   ONLINE       0     0   102
	  /private/tmp/test/a  ONLINE       0     0   276

errors: Permanent errors have been detected in the following files:

        /Volumes/tank/SystemStarter
        /Volumes/tank/autodiskmount
        /Volumes/tank/disklabel
.
.
.
        /Volumes/tank/newfs_msdos
        /Volumes/tank/newfs_udf

# sum /Volumes/tank/*
sum: /Volumes/tank/SystemStarter: Input/output error
sum: /Volumes/tank/autodiskmount: Input/output error
.
.
.
29849 58 /Volumes/tank/shutdown
58965 45 /Volumes/tank/umount

Hands on - Destroying data 3

# cp /sbin/* /Volumes/tank/
# zpool scrub tank
# sudo zpool status -v tank
  pool: tank
 state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
 scan: scrub repaired 0 in 0h0m with 2 errors on Thu Oct 11 15:26:20 2012
config:

	NAME                   STATE     READ WRITE CKSUM
	tank                   ONLINE       0     0   144
	  /private/tmp/test/a  ONLINE       0     0   330

errors: Permanent errors have been detected in the following files:

        /Volumes/tank/.fseventsd/00000000001c3081
        /Volumes/tank/.fseventsd/00000000001c3082
# sudo rm  /Volumes/tank/.fseventsd/00000000001c3081
# sudo rm  /Volumes/tank/.fseventsd/00000000001c3082

# sudo zpool status -v
  pool: tank
 state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
 scan: scrub repaired 0 in 0h0m with 2 errors on Thu Oct 11 15:26:20 2012
config:

	NAME                   STATE     READ WRITE CKSUM
	tank                   ONLINE       0     0   144
	  /private/tmp/test/a  ONLINE       0     0   330

errors: Permanent errors have been detected in the following files:

        tank:<0x46>
        tank:<0x47>

Hands on - Destroying data 4

# zpool scrub tank
# sudo zpool status -v
  pool: tank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
 scan: scrub repaired 0 in 0h0m with 0 errors on Thu Oct 11 15:27:15 2012
config:

	NAME                   STATE     READ WRITE CKSUM
	tank                   ONLINE       0     0   144
	  /private/tmp/test/a  ONLINE       0     0   330

errors: No known data errors

# zpool clear tank
# zpool scrub tank
# zpool status
  pool: tank
 state: ONLINE
 scan: scrub repaired 0 in 0h0m with 0 errors on Thu Oct 11 15:34:08 2012
config:

	NAME                   STATE     READ WRITE CKSUM
	tank                   ONLINE       0     0     0
	  /private/tmp/test/a  ONLINE       0     0     0

errors: No known data errors

Thanx

Patrick Stein @jollyjinx