Setting up media server using Ubuntu and SnapRaid

This is the document I put together on how to install and set everything on my Nas Killer 3.0. You can see my actual build specs here.

Its pretty common in the serverbuilds community to use Unraid, which is a great solution. I wanted to be able to build some other more linux-y stuff, not have to pay, and get my terminal chops back up to snuff (plus I’ve been an ubuntu guy for a long time). SnapRAID is great at expansion like Unraid, but is not real time. That seems pretty acceptable for a media server to me.

Install Ubuntu Server on your new box

For these first steps, you typically are connected directy to the box with a keyboard and monitor. You don’t need a mouse as you don’t have any guis.

Get Bootable USB

  1. Get and install Rufus
  2. Download Ubuntu Server (I did 18.04)
  3. Use Rufus to load Ubuntu to USB

Install

  1. Use Del to load bios and change boot order to USB drive first (removed this later)
  2. Follow prompts…nothing crazy…no extra packages, SSH. Installed to whole SSD drive.
  3. Reboot - Make sure to change back boot order so you boot from SSD not usb if still installed
  4. Login with username and password

Initial Updates and checks

Check to make sure you have universe and multiverse repos by running “nano /etc/apt/sources.list” and checking if those repositories are listed - This wasn’t the case for me once.

  1. sudo apt update
  2. sudo apt upgrade

Setup SSH Server

To do this, I followed the following link: https://www.smarthomebeginner.com/install-ssh-server-on-ubuntu-1204/. Plus the following:

  1. Change the port to a random number
  2. Change the users and other settings as given in the link
  3. Restart SSH
  4. Login with Putty from a windows machine

Format Drives to EXT4

This assumes that you have brand new drives that you want formatted for Ubuntu. This WILL ERASE all content on your drives.

My Drives were all formatted for ZFS…and needed to be changed. SnapRAID Supports both XFS and EXT4. I did EXT4 just because that is the default for Ubuntu.

To see your partitions and formats: sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL

I followed this link to help with this process: How To Partition and Format Storage Devices in Linux | DigitalOcean

For each data drive that needs reformatting do the following where " is something like sdb. I also just number my drives to disk1 or disk2 after the -L:

  1. sudo parted /dev/<DRIVE> mklabel gpt
  2. sudo parted -a opt /dev/<DRIVE> mkpart primary ext4 0% 100%
  3. sudo mkfs.ext4 -L disk<NUMBER> -m 1 -T largefile4 /dev/<DRIVE>1

Note that I reduced the reserved space to 1% for my 8tb drives, you may want to leave this as 5% for smaller drives. The -T tells it what kind of file size I’ll have. For movies, they will generally be large.

The parity drive(s) should be similar, other than you shouldn’t leave any space (-m 0) (see SnapRAID):

  1. sudo parted /dev/<DRIVE> mklabel gpt
  2. sudo parted -a opt /dev/<DRIVE> mkpart primary ext4 0% 100%
  3. sudo mkfs.ext4 -L parity1 -m 0 -T largefile4 /dev/<DRIVE>1

Mount Drives and Setup mergerFS

First, we want to install mergerFS: sudo apt install mergerFS fuse. mergerFS is useful at taking multiple drives and making it a single storage pool. That way you can just add media to a single folder and it will decide which drive to put it on.

A useful link for this section is: The Perfect Media Server 2017

We want to check your formatted disks: sudo lsblk --fs will show the drives.

If you named your drives like I have, you need to make directories in the /mnt/ folder for each drive, as well as for the pool (we’re gonna call that storage).

mkdir /mnt/disk{1,2,3,4,5}
mkdir /mnt/parity{1,2} # adjust this command based on your parity setup
mkdir /mnt/storage

Now, we need to edit your fstab file (this is the file that is used to auto-mount drives. To do that we’ll use a simple file editor called nano. You can move your cursor around. Save with ctrl+s and exit with ctrl+x. To edit the file, run sudo nano /etc/fstab to make it look like:

OTHER STUFF THAT IS ALREADY THERE...

# My Additions of harddrives
LABEL=disk1     /mnt/disk1      ext4    defaults 0 0
LABEL=disk2     /mnt/disk2      ext4    defaults 0 0
LABEL=disk3     /mnt/disk3      ext4    defaults 0 0
LABEL=disk4     /mnt/disk4      ext4    defaults 0 0
LABEL=parity1   /mnt/parity1    ext4    defaults 0 0


# Combine data harddrives into single storage mount using mergerFS
/mnt/disk*     /mnt/storage     fuse.mergerFS direct_io,defaults,allow_other,minfreespace=50G,fsname=mergerFS 0 0

Now you want to test and remount everything. Run the following commands one by one to test it out:

sudo mount -a
df -h -x tmpfs -x devtmpfs
echo "success" | sudo tee /mnt/storage/test_file
cat /mnt/storage/test_file
sudo rm /mnt/storage/test_file

Additionally, it can be nice to have the users have access to the storage medium. To do this run sudo chown <username> /mnt/storage/ This also helps with the samba steps.

Install Docker

You’ll use docker later to install packages and tools, additionally, we’ll use it here to build some tools, so we do in now.

Docker tells you how to do this:
Redirecting… - DO NOT USE SCRIPT METHOD

  1. sudo apt-get remove docker docker-engine docker.io
  2. See Link

Test by running sudo docker run hello-world, sudo docker rmi hello-world

A lot of docker tutorials want your user added to the docker power stuff so you don’t have to type sudo each time…I think that’s dumb and type sudo each time. So I ignored this step: sudo usermod -aG docker $USER.

In my notes I have this command: sudo systemctl enable docker - I don’t remember doing this, but you may need to.

3 Likes

Install and Setup SnapRAID

Setting up SnapRAID on Ubuntu to Create a Flexible Home Media Fileserver | Zack Reed - Design and Coding - Official from SnapRAID but is cumbersome. Some folks have made it easier with a script and a docker image to install everything for you. So lets use: The Perfect Media Server 2017 to do this.

Summary: We create a docker to get and build the SnapRAID, then install the package it made, the docker container kills itself when done.

apt install git
git clone https://github.com/lord-carlos/docker-SnapRAID.git
cd docker-SnapRAID/
chmod +x build.sh
sudo ./build.sh
cd build/
dpkg -i SnapRAID*.deb

Test that it worked by typing SnapRAID -v, you may have to use sudo SnapRAID -v.

To configure SnapRAID, we need to edit /etc/SnapRAID.conf, that is done by using nano again: sudo nano /etc/SnapRAID.conf should have some default stuff. Make it look like Setting up SnapRAID on Ubuntu to Create a Flexible Home Media Fileserver | Zack Reed - Design and Coding says.

# Example configuration for SnapRAID

# Defines the file to use as parity storage
# It must NOT be in a data disk
# Format: "parity FILE_PATH"
parity /mnt/parity1/SnapRAID.parity

# Defines the files to use as content list
# You can use multiple specification to store more copies
# You must have least one copy for each parity file plus one. Some more don't hurt
# They can be in the disks used for data, parity or boot,
# but each file must be in a different disk
# Format: "content FILE_PATH"
content /var/SnapRAID.content
content /mnt/disk1/.SnapRAID.content
content /mnt/disk2/.SnapRAID.content
content /mnt/disk3/.SnapRAID.content
content /mnt/disk4/.SnapRAID.content

# Defines the data disks to use
# The order is relevant for parity, do not change it
# Format: "disk DISK_NAME DISK_MOUNT_POINT"
disk d1 /mnt/disk1/
disk d2 /mnt/disk2/
disk d3 /mnt/disk3/
disk d4 /mnt/disk4/

# Excludes hidden files and directories (uncomment to enable).
#nohidden

# Defines files and directories to exclude
# Remember that all the paths are relative at the mount points
# Format: "exclude FILE"
# Format: "exclude DIR/"
# Format: "exclude /PATH/FILE"
# Format: "exclude /PATH/DIR/"
exclude *.bak
exclude *.unrecoverable
exclude /tmp/
exclude /lost+found/
exclude snapshots/
exclude *.!sync
exclude .AppleDouble
exclude ._AppleDouble
exclude .DS_Store
exclude ._.DS_Store
exclude .Thumbs.db
exclude .fseventsd
exclude .Spotlight-V100
exclude .TemporaryItems
exclude .Trashes
exclude .AppleDB
# Adding exclusion of downloads since we get a bunch of partial files here:
exclude /Downloads/
exclude *.backup\~
exclude *.partial\~
exclude node_modules/
exclude .git/

This docker installs left some trash, I followed linux - "docker images" shows image, "docker rmi" says "no such image" or "reference does not exist" - Stack Overflow to remove it.

Schedule SnapRAID

The Perfect Media Server 2017 - Explains that there is a script to do this for us. Its old and uses python2 and a default python conf file to run.

  1. Create a folder /opt/snaptraid-runner/ (see link)
  2. Download sample conf and py script (see link) into folder
  3. Setup Conf:
    1. Threshold to 250
    2. Email, probably just error long term, put your email in
    3. Smtp - using the blog’s link has the proper gmail settings of port 465, ssl true. Use your gmail account, then you need to get an app password for the password. Or see Mailgun below.
    4. I left scrub settings default to true, 12 and 10

To schedule make sure you use the root crontab. Either sudo -i before or sudo crontab -e. Add the proper scheduling you want to do. Mine looks like:

00 08 * * * python2 /opt/snapraid-runner/snapraid-runner.py -c /opt/snapraid-runner/snapraid-runner.conf

My conf file looks like:

[snapraid]
; path to the snapraid executable
executable = /usr/local/bin/snapraid
; path to the snapraid config to be used
config = /etc/snapraid.conf
; abort operation if there are more deletes than this, set to -1 to disable
deletethreshold = 250

[logging]
; logfile to write to, leave empty to disable
file = snapraid.log
; maximum logfile size in KiB, leave empty for infinite
maxsize = 5000

[email]
; when to send an email, comma-separated list of [success, error]
sendon = success,error
; set to false to get full program output via email
short = true
subject = [SnapRAID] Status Report:
from = <CUSTOM MAIL ADDRESS>
to = <YOUR EMAIL ADDRESS>

[smtp]
host = smtp.mailgun.org
; leave empty for default port
port = 465
; set to "true" to activate
ssl = true
user = <CUSTOM MAIL ADDRESS>
password = <PASSWORD>

[scrub]
; set to true to run scrub after sync
enabled = true
percentage = 12
older-than = 10

Running SnapRAID manually
The SnapRAID documentation shows how to use a lot of the commands by hand, but I’ll go over the high level here.

  • sudo snapraid diff will tell you what has changed…
  • sudo snapraid sync will run a backup. The script does this daily, but it may fail if you’ve deleted a lot of content. It’s handy to be able to manually run when needed. Note - Sync can be slow, so you want your SSH connection to stay alive during the whole process
  • See documentation for recovery options.

Samba for FileSharing

I copied the old file to /etc/samba/smb.orig, then I added without changing anything:

[storage]
   comment = Storage on <SERVER NAME>
   path = /mnt/storage
   browseable = yes
   read only = no
   guest ok = yes
   hide unreadable = yes

Then restart the server using sudo systemctl restart smbd

You should be able to browse in windows under the network, but this won’t let you write or change files. https://www.howtogeek.com/howto/16196/how-to-disconnect-non-mapped-unc-path-drives-in-windows/

You may need to open up a command prompt on windows and use net use to delete the existing one and re-add it using the new credentials.

Connect at \\SERVERNAME\Sharename

I later followed the same steps for the opt folder, naming it configopt

Adding more drives later

To add a drive later, I have to follow the following steps:

  1. Physically install the drive
  2. Physically connect to the server (instead of SSH) to load the bios and ensure that it boots from the SSD instead of the newly installed drive.
  3. Format drive like in the formatting section above
  4. Add it to the FSTAB (and remount, a reboot also makes this easy)
  5. Add to SnapRAID.conf to include it into the array
  6. Run sudo snapraid sync to initialize the new drive into the array.

mergerFS Tools (Optional)

These are a suite of tools that help manipulate your mergerFS drives. You can remove duplicates, balance, or other strange moving tools. It’s not necessary, but sometimes can be useful.

Installation:

  1. cd ~ - to get to your home directory
  2. git clone https://github.com/trapexit/mergerfs-tools - Get the code on your local machine
  3. sudo apt install build-essential - Get the build tools you’ll need to install everything
  4. cd mergerfs-tools - Go into the code directory
  5. sudo make install - Build and install the commands

Move Files between drives
I had a strange situation. I had a couple nice 8tb drives, and a super old slow 2.5" drive I was adding. In addition to media, I have some old static backup files that I only access once every blue moon. I decided that I wanted the backup content on the slow drive, as to leave more room on the big drives for fast media etc. So how do I within the mergerFS array do this properly?

  1. sudo cp -dR --preserve=mode,ownership,xattr /mnt/disk1/Pictures /mnt/disk2/ - Copy from one drive to the other without any date, makes using the tools easier.
  2. mergerfs.dedup -v -d newest /mnt/storage/Pictures/ - to simulate what files would be deleted.
  3. mergerfs.dedup -v -d -e newest /mnt/storage/Pictures/ - to actually run.

Thanks to GitHub - trapexit/mergerfs: a featureful union filesystem

Alias Setup

Do you want to do stupid things over and over…No? Do this:

  1. nano ~/.bash_profile
  2. Add the text you want and save.
  3. Run source ~/.bash_profile

My file looks like:

alias sudo='sudo '
alias df='df -h -x tmpfs -x devtmpfs'

You can add other shortcuts here. I often also add alias grep= grep --color=auto to mine.

3 Likes

I’ve been a newbie in this nas area, I really want to know why hook ZFS with mergerFS and Snapraid, How does it work and what is the point of this? Thank you.

you misread i think, ZFS was what they were formatted prior. Ext4 formatting during this tutorial

1 Like

What performance are you getting from this? I have SnapRAID + MergerFS on an Openmediavault 4.x install and I think I’m starting to get diminishing returns, specially when there is more than one simultaneous task going on

These are not fully raided options for higher performance drive striping. It is an unraid like solution where content is still stored on individual drives and limited by that speed. That is more than enough for what I’m doing. If you need the speed a more traditional raid is probably best.

Thanks @eskemojoe007 - this helped me understand the relationship between mergerfs and snapRaid :slight_smile:

One thing I don’t really get about parity drives, is if some bit gets flipped, how does the system know which drive to trust? Maybe some hash verifications?

For example:
Drive 1, Drive 2, Parity 1
1, 1, 0 → GOOD
1, 0, 0 → BAD, but is it Drive 1, Drive 2 or Parity1?