Create ZFS pool using Mirrored, RAIDZ or nonredundant virtual devices on Ubuntu
Last updated:
linux nas ubuntu zfs
Overview
The Z File System (ZFS) can be used to pool together several disks. For example, three 2TB disks could be connected to your computer to create a single pool with 6TB storage capacity.
This article will show you how to use the commandline to create a ZFS pool. Each ZFS pool is made up of one or more virtual devices, also known as vdevs. A virtual device is usually a group of disks. These virtual devices can be configured in different ways depending on your storage and performance requirements. Common configurations include RAIDZ
and Mirror
.
The following assumptions are made regarding your setup:-
- Your computer or server is running Ubuntu
- You have multiple hard drives connected
All data on the drives will be lost!
The examples in this article create and destroy lots of ZFS pools.
If your disks already contain data, then back-up that data before proceeding.
Install ZFS
We need to install ZFS on Linux before we can use the ZFS commands. Log into your machine, open up a terminal prompt or log into it via ssh. My server is headless, so I am going to ssh from my Windows laptop into my Ubuntu server using PuTTY.
To install ZFS user-level tools simply run
sudo apt-get update
sudo apt-get install zfsutils-linux
Let's see if ZFS is installed correctly by listing our ZFS pools.
sudo zpool status
no pools available
We now know that ZFS is working, but we don't have any pools yet.
Check your disks
Let's check that our hard drives are connected and find out what they are named.
lsblk -I 8 -d
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 14.9G 0 disk
sdb 8:16 0 1.8T 0 disk
sdc 8:32 0 1.8T 0 disk
sdd 8:48 0 1.8T 0 disk
sde 8:64 0 1.8T 0 disk
sdf 8:80 0 1.8T 0 disk
sdg 8:96 0 1.8T 0 disk
From this is can see that I have 7 disks connected. The first disk, named sda, is 14.9G, and contains the Ubuntu operating system. The other 6 disks, named sdb to sdg, are all 1.8T. I am going to use a varoius mix of these 6 disks to show you the different ZFS configurations.
Create a nonredundant ZFS pool
The most basic virtual device configuration is where there is no data redundancy. The data is striped across all available disks, but if one disk fails, then all the data will be lost.
I am going to create a pool named DUMPSTER from 6 of my disks, being sdb, sdc, sdd, sde, sdf and sdg. Make sure you replace the pool name and disk names with your own.
sudo zpool create DUMPSTER sdb sdc sdd sde sdf sdg
Let's list our pools again.
sudo zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
DUMPSTER 10.9T 1.30M 10.9T - 0% 0% 1.00x ONLINE -
We now have a pool named DUMPSTER which is 10.9T (6 x 1.8T). Let's check that it has mounted correctly.
df -h | grep DUMPSTER
DUMPSTER 11T 128K 11T 1% /DUMPSTER
As you can see, our pool has mounted on /DUMPSTER
. Note that the usable space is 11T (6 x 1.8T). This is because we have chosen to have no redundancy. If any of the disks fail, we would lose all of our data.
We can make sure everything is working as expected by checking the status of our pool.
sudo zpool status
pool: DUMPSTER
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
DUMPSTER ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
sdg ONLINE 0 0 0
errors: No known data errors
As you can see, our pool named DUMPSTER is online and is made up of 6 disks.
Let's remove the pool before moving on to mirrored configurations.
All data will be lost
sudo zpool destroy DUMPSTER
Create a Mirrored ZFS pool
Mirrored means that the virtual device stores identical copies of data on two or more disks. If any disk in a mirror fails, any other disk in that mirror can provide the same data.
Two-way Mirror (RAID 1)
This is a Mirrored configuration consisting of 2 disks. Sometimes referred to as RAID 1
.
I am going to create a pool named DUMPSTER. This will contain a Mirrored virtual device made up of 2 of my disks, being sdb and sdc. Make sure you replace the pool name and disk names with your own.
sudo zpool create DUMPSTER mirror sdb sdc
Let's list our pools again.
sudo zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
DUMPSTER 1.81T 432K 1.81T - 0% 0% 1.00x ONLINE -
We now have a pool named DUMPSTER which is 1.81T (1 x 1.8T). Let's check that it has mounted correctly.
df -h | grep DUMPSTER
DUMPSTER 1.8T 128K 1.8T 1% /DUMPSTER
As you can see, our pool has mounted on /DUMPSTER
. Note that the usable space is only 1.8T (1 x 1.8T). This is because we have chosen Mirrored
. This means we can't use 1.8T, however, if one of the disks fail in this Mirrored virtual device, we can replace it without losing any of our data.
We can make sure everything is working as expected by checking the status of our pool.
sudo zpool status
pool: DUMPSTER
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
DUMPSTER ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
errors: No known data errors
As you can see, our pool named DUMPSTER is online, it contains a Mirrored
virtual device named mirror-0 which is made up of 2 disks.
Let's remove the pool before moving on to a three-way mirrored configurations.
All data will be lost
sudo zpool destroy DUMPSTER
Three-way Mirror
This is a Mirrored configuration consisting of 3 disks.
I am going to create a pool named DUMPSTER. This will contain a Mirrored virtual device made up of 3 of my disks, being sdb, sdc and sdd. Make sure you replace the pool name and disk names with your own.
sudo zpool create DUMPSTER mirror sdb sdc sdd
Let's list our pools again.
sudo zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
DUMPSTER 1.81T 408K 1.81T - 0% 0% 1.00x ONLINE -
Once again, we have a pool named DUMPSTER which is 1.81T (1 x 1.8T). Let's check that it has mounted correctly.
df -h | grep DUMPSTER
DUMPSTER 1.8T 128K 1.8T 1% /DUMPSTER
As you can see, our pool has mounted on /DUMPSTER
. Note that the usable space is only 1.8T (1 x 1.8T). This is because we have chosen to have 3 disks in our Mirrored
virtual device. This means we can't use 3.6T, however, if up to two of the disks fail in this Mirrored virtual device, we can replace it without losing any of our data.
We can make sure everything is working as expected by checking the status of our pool.
sudo zpool status
pool: DUMPSTER
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
DUMPSTER ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
errors: No known data errors
As you can see, our pool named DUMPSTER is online, it contains a Mirrored
virtual device named mirror-0 which is made up of 3 disks.
Let's remove the pool before moving on to a striped mirror configurations.
All data will be lost
sudo zpool destroy DUMPSTER
Striped Mirror (RAID 10)
A striped mirror means that the pool contains multiple equal sized Mirrored virtual devices with the data being striped across them. This has the benefits of Mirrored plus the performance enhancement of striping. This is sometimes referred to as RAID 10
.
I am going to create a pool named DUMPSTER. This will contain 2 Mirrored virtual devices, each made up of 2 disks. Make sure you replace the pool name and disk names with your own.
sudo zpool create DUMPSTER mirror sdb sdc mirror sdd sde
Let's list our pools again.
sudo zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
DUMPSTER 3.62T 408K 3.62T - 0% 0% 1.00x ONLINE -
We now have a pool named DUMPSTER which is 3.62T (2 x 1.8T). Let's check that it has mounted correctly.
df -h | grep DUMPSTER
DUMPSTER 3.6T 128K 3.6T 1% /DUMPSTER
As you can see, our pool has mounted on /DUMPSTER
. Note that the usable space is only 3.6T (2 x 1.8T). This is because we have chosen Striped Mirror
. This means we can't use 3.6T, however, if one disk fails in any of the Mirrored virtual devices, we can replace it without losing any of our data.
We can make sure everything is working as expected by checking the status of our pool.
sudo zpool status
pool: DUMPSTER
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
DUMPSTER ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
errors: No known data errors
As you can see, our pool named DUMPSTER is online, it contains a two Mirrored
virtual devices named mirror-0 and mirror-1 each made up of 2 disks.
Let's remove the pool before moving on to a RAIDZ configurations.
All data will be lost
sudo zpool destroy DUMPSTER
Create a RAIDZ ZFS pool
RAIDZ does not duplicate data, instead it stripes data across the disks and calculates parity, storing the result. When a disk fails, the missing data can be rebuilt using the parity information. Typically, RAIDZ utilises more of the available disk space. There are currently three types of RAIDZ.
RAIDZ1 (RAID 5)
RAIDZ1, also referred to as single-parity redundancy and sometimes known as RAID 5
. It has a 1 disk redundancy level, which means you can lose up to 1 disk before losing any of your data.
I am going to create a pool named DUMPSTER. This will contain a RAIDZ1 virtual device made up of 4 of my disks, being sdb, sdc, sdd, and sde. Make sure you replace the pool name and disk names with your own.
sudo zpool create DUMPSTER raidz1 sdb sdc sdd sde
Let's list our pools again.
sudo zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
DUMPSTER 7.25T 864K 7.25T - 0% 0% 1.00x ONLINE -
We now have a pool named DUMPSTER which is 7.25T (4 x 1.8T). Let's check that it has mounted correctly.
df -h | grep DUMPSTER
DUMPSTER 5.2T 128K 5.2T 1% /DUMPSTER
As you can see, our pool has mounted on /DUMPSTER
. Note that the usable space is only 5.2T (3 x 1.8T). This is because we have chosen RAIDZ1
. This means we can't use 1.8T, however, if a disk fails, we can replace it without losing any of our data.
We can make sure everything is working as expected by checking the status of our pool.
sudo zpool status
pool: DUMPSTER
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
DUMPSTER ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
errors: No known data errors
As you can see, our pool named DUMPSTER is online, it contains a RAIDZ1
virtual device named raidz1-0 which is made up of 4 disks.
Let's remove the pool before moving on to a RAIDZ2 configurations.
All data will be lost
sudo zpool destroy DUMPSTER
RAIDZ2 (RAID 6)
RAIDZ2, also referred to as double-parity redundancy and sometimes known as RAID 6
. It has a 2 disk redundancy level, which means you can lose up to 2 disks before losing any of your data.
I am going to create a pool named DUMPSTER from 6 of my disks, being sdb, sdc, sdd, sde, sdf and sdg. Make sure you replace the pool name and disk names with your own.
sudo zpool create DUMPSTER raidz2 sdb sdc sdd sde sdf sdg
Let's list our pools again.
sudo zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
DUMPSTER 10.9T 1.30M 10.9T - 0% 0% 1.00x ONLINE -
We now have a pool named DUMPSTER which is 10.9T (6 x 1.8T). Let's check that it has mounted correctly.
df -h | grep DUMPSTER
DUMPSTER 7.1T 256K 7.1T 1% /DUMPSTER
As you can see, our pool has mounted on /DUMPSTER
. Note that the usable space is only 7.1T (4 x 1.8T). This is because we have chosen raidz2
. We can't use 3.6T, however, if up to 2 disks fail, we can replace them without losing any of our data.
We can make sure everything is working as expected by checking the status of our pool.
sudo zpool status
pool: DUMPSTER
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
DUMPSTER ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
sdg ONLINE 0 0 0
errors: No known data errors
As you can see, our pool named DUMPSTER is online, it contains a RAIDZ2
virtual device named raidz2-0 which is made up of 6 disks.
Let's remove the pool before moving on to a RAIDZ3 configurations.
All data will be lost
sudo zpool destroy DUMPSTER
RAIDZ3
RAIDZ3 is sometimes referred to as triple-parity redundancy. It has a 3 disk redundancy level, which means you can lose up to 3 disks before losing any of your data.
I am going to create a pool named DUMPSTER from 6 of my disks, being sdb, sdc, sdd, sde, sdf and sdg. Make sure you replace the pool name and disk names with your own.
sudo zpool create DUMPSTER raidz3 sdb sdc sdd sde sdf sdg
Let's list our pools again.
sudo zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
DUMPSTER 10.9T 1.30M 10.9T - 0% 0% 1.00x ONLINE -
We now have a pool named DUMPSTER which is 10.9T (6 x 1.8T). Let's check that it has mounted correctly.
df -h | grep DUMPSTER
DUMPSTER 5.0T 128K 5.0T 1% /DUMPSTER
As you can see, our pool has mounted on /DUMPSTER
. Note that the usable space is only 5.0T (3 x 1.8T). This is because we have chosen raidz3
. We can't use 4.8T (3 x 1.8T), however, if up to 3 disks fail, we can replace them without losing any of our data.
We can make sure everything is working as expected by checking the status of our pool.
sudo zpool status
pool: DUMPSTER
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
DUMPSTER ONLINE 0 0 0
raidz3-0 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
sdg ONLINE 0 0 0
errors: No known data errors