[Guide] Move Your Docker Image and Appdata to an Unassigned Drive (DEPRECATED)

Unraid 6.9 Update

With the release of unraid 6.9 comes the ability to define multiple cache pools, which eliminates the need for moving your Docker data to an unassigned device. If you’ve already followed this guide and your Docker data is on an unassigned device, there’s no need to change things back. However, if you’re reading this guide for the first time, and you’re using unraid 6.9 or higher, I would recommend not using this guide and instead creating a separate cache pool for your Docker and VM data.

Summary

By default, unraid puts the Docker vDisk image and appdata on your the array, with no caching enabled. This is usually fine for one or two containers, but as you add more containers to the mix, your disk IO can increase and responsiveness for the containers can go down as a result. To fix this, you could set the shares for your Docker image and appdata to always be on the cache drive. This will work, but then you’re using up a chunk of the cache space constantly, and if you ingest a lot of new files frequently, you can run into a disk IO problem again. A better solution is to move the files to an unassigned drive, preferably an SSD. Keeping your Docker files on a dedicated drive can increase responsiveness for any container, but it can be especially helpful if you run containers that work with databases. This article will walk you through the steps needed to move your Docker installation to an unassigned drive.

:stop_sign: WARNING :stop_sign: WARNING :stop_sign: WARNING :stop_sign: WARNING :stop_sign: WARNING :stop_sign:

This is an advanced topic that uses the terminal to move files, mount disk images, and make manual edits to configuration files. I have tested these commands on my own system successfully, but I am not liable if you break your installation. Please read through this entire guide thoroughly before attempting the process. If you have any questions, please post them here or on the discord and do not proceed until you have been given an answer.

Read Me First

This process assumes you have the array STARTED, and the disk you want to copy the image to is mounted as an unassigned device. If this isn’t the case, :stop_sign: STOP HERE :stop_sign: and ask for assistance.

In several of the steps below, you will need to enter commands in the terminal. Commands to be entered will be placed in code blocks. There will be some places in these code blocks where you need replace a placeholder with a specific folder or value from your system. Look for words with angled braces around them, like <unassigned_drive> and replace everything, including the angled braces, with the proper value. Be careful not to replace or remove any characters outside the braces.

0. Back up your appdata folder
  • I would recommend installing the CA Appdata Backup/Restore plugin from the Apps tab, and running a full backup of your entire appdata folder prior to moving anything. The process shouldn’t take more than a few minutes, and it will give you a safety net just in case you run into a problem. You may also want to consider using the plugin to schedule recurring backups. Regardless of how you backup the folder, make sure the backup is completed before you proceed.
1. Stop all of your running containers
  • Save your work in any containers that are running, then go to the Docker tab in the WebUI and click the Stop all button.
2. Disable Docker and grab the current paths
  • Go to Settings > Docker, and set Enable Docker to No, then Apply the settings and wait for the page to reload. You’ll notice the Docker tab is no longer showing in the top menu. While you are here, make note if the Docker vDisk location and Default appdata storage location. You will need these paths in a future step. Take a screenshot, or copy them to a notepad to make things easier. Copy the full path including the leading and trailing slashes. If you haven’t changed them from the defaults, they are:
    Docker vDisk location: /mnt/user/system/docker/docker.img
    Default appdata storage location: /mnt/user/appdata/
3. Get the unassigned drive name
  • Go to the Main tab and find the name of the unassigned drive you want to use. For example, in this screenshot I have a disk named ua_docker. This drive name is what you will use in all the steps below when they ask for <unassigned_drive>

4. Open a terminal
  • Click the >_ icon in the upper right to open the Web Terminal, or use Putty or your favorite terminal emulator to connect to the server.
5. Create the folder for the Docker vDisk
  • Enter the following command:

    mkdir -p /mnt/disks/<unassigned_disk>/system/docker && chown -R nobody.users /mnt/disks/<unassigned_disk>/system
    

    replacing <unassigned_disk> with the name of the disk from step 3.

6. Move the Docker vDisk image to the new drive
  • Enter the following command:

    mv <docker_vdisk> /mnt/disks/<unassigned_disk>/system/docker/docker.img
    

    replacing <docker_vdisk> with the location you copied in step 2, and <unassigned_disk> with the name of the disk from step 3. This process may take a few minutes depending on how large your Docker image is.

7. Move the appdata folder to the new drive
  • Enter the following command:
    mv <default_appdata> /mnt/disks/<unassigned_disk>/
    
    replacing <default_appdata> with the appdata location from step 2, and <unassigned_disk> with the disk name from step 3. This step may take a while, depending on how many containers you have.
8. Mount the Docker vDisk image
  • Enter the following command:

    mkdir /mnt/docker_temp && mount /mnt/disks/<unassigned_disk>/system/docker/docker.img /mnt/docker_temp
    

    replacing <unassigned_disk> with the folder from step 3.

9. Update the container configuration files to reflect the new appdata path
  • Enter the following command:

    find /mnt/docker_temp/containers -name '*.json' -exec sed -i -r 's/appdata(.*):(r[ow])"/appdata\1:\2,slave"/g; s+<default_appdata>+/mnt/disks/<unassigned_disk>/appdata/+g' {} \;
    

    Replace <default_appdata> with the appdata folder from step 2, and <unassigned_disk> with the disk name from step 3.

10. Update the dockerMan templates to reflect the new folder
  • Enter the following command:

    find /boot/config/plugins/dockerMan/templates-user/*.xml -exec sed -i -r 's+<default_appdata>+/mnt/disks/<unassigned_disk>/appdata/+g; s/(.*)appdata(.*)Mode="(r[ow])"(.*)/\1appdata\2Mode="\3,slave"\4/g;' {} \;
    

    Replace <default_appdata> with the appdata folder from step 2, and <unassigned_disk> with the disk name from the beginning.

11. Update Unraid Docker Config
  • Enter the following command:

    sed -i 's+/mnt/user/+/mnt/disks/<unassigned_drive>/+g' /boot/config/docker.cfg
    

    Replacing <unassigned_drive> with the disk name from step 3.

12. OPTIONAL - Update CA Backup/Restore Appdata
  • If you have the CA Backup/Restore Appdata plugin installed, enter the following command. If you don’t have this plugin installed, skip to the next step:

    sed -i 's+/mnt/user/+/mnt/disks/<unassigned_drive>/+g' /boot/config/plugins/ca.backup2/BackupOptions.json
    

    Replacing <unassigned_drive> with the disk name from step 3.

13. Clean up after yourself
  • Enter the following command:

    umount /mnt/docker_temp && rmdir /mnt/docker_temp && exit
    

    You can now close the terminal window.

14. Restart the Docker engine
  • Using the WebUI, go to Settings > Docker, change Enable Docker to Yes and click Apply. Wait for the page to load.
15. Verify your work
  • Go to the Docker tab and verify that all of your containers are still listed. You can now start your containers again and verify they’re working.
16. It's Miller Time
  • That’s it, you’re done. If your containers are all running, grab a beer or 3 and celebrate. You have successfully moved your Docker installation.

TL;DR Version:

IF your Docker vDisk image and appdata are both in their default locations (/mnt/user/system/docker/docker.img and /mnt/user/appdata respectively), you can use the script below to take a shortcut.

  1. Back up your appdata folder. Use the CA Backup/Restore Appdata plugin.
  2. Stop the Docker engine (Settings > Docker > Enable Docker = No > Apply)
  3. Go to the Main tab and grab the name of the disk you want to use. Even if you already know the name, you need to leave the Docker settings screen for the script to work, so just go to the Main tab.
  4. Copy the commands below into your favorite text editor
  5. Search for and replace all instances of REPLACE_ME with the drive name from step 2
  6. Open the terminal window. Copy and paste the commands from your text editor into the terminal one at a time.
    • Please note, if you don’t have the CA Backup/Restore Appdata plugin installed, you can skip the second to the last command, the one right above exit
  7. Start the Docker engine (Settings > Docker > Enable Docker = Yes > Apply)
  8. Check the Docker tab to verify your containers are all listed, and then start them
mkdir -p /mnt/disks/REPLACE_ME/system/docker && chown -R nobody.users /mnt/disks/REPLACE_ME/system
mv /mnt/user/system/docker/docker.img /mnt/disks/REPLACE_ME/system/docker/
mv /mnt/user/appdata/ /mnt/disks/REPLACE_ME/
mkdir /mnt/docker_temp && mount /mnt/disks/REPLACE_ME/system/docker/docker.img /mnt/docker_temp
find /mnt/docker_temp/containers -name '*.json' -exec sed -i -r 's+/mnt/user/appdata/+/mnt/disks/REPLACE_ME/appdata/+g; s/appdata(.*):(r[ow])"/appdata\1:\2,slave"/g' {} \;
umount /mnt/docker_temp && rmdir /mnt/docker_temp
find /boot/config/plugins/dockerMan/templates-user -name '*.xml' -exec sed -i -r 's+/mnt/user/appdata/+/mnt/disks/REPLACE_ME/appdata/+g; s/(.*)appdata(.*)Mode="(r[ow])"(.*)/\1appdata\2Mode="\3,slave"\4/g;' {} \;
sed -i 's+/mnt/user/+/mnt/disks/REPLACE_ME/+g' /boot/config/docker.cfg
sed -i 's+/mnt/user/+/mnt/disks/REPLACE_ME/+g' /boot/config/plugins/ca.backup2/BackupOptions.json
exit

FAQ

My containers are running fast enough, do I need to do this?
  • Nope - if you are happy with how your system is running, there is no need to move the containers.
I just started using unraid, do I really need to go through all of these steps?
  • Nope - if you haven’t installed any containers yet, all you have to do is change the paths in the Docker settings to the unassigned drive you want to use.
Why don't you just recreate the Docker vDisk image instead of moving it?
  • Sure, you could delete and recreate the image in the new folder, but then you would need to re-download all of the images and recreate all of the containers. Moving a file is much simpler.
My appdata share disappeared after I ran this. How do I get it back?
  • When you moved the appdata folder, the share was automatically removed. You’ll need to share the unassigned drive to which you moved the folder. In the main tab, under the unassigned devices category, toggle the “Share” button for the drive you used.
Can I update the unassigned devices plugin if I moved the appdata to an unassigned device?
  • Yes. You can still update the plugin, but you need to do one of the following:
    • Stop all running containers and the docker engine, and unmount all of your unassigned drives before you run the update
    • Reboot your server after you install the update.
11 Likes

In my case I want to pair this guide with a cache drive upgrade. Current cache is 120gb ssd, upgraded drive is a 1tb ssd. Since all the app data is already on the final destination 120gb SSD, can the steps be simplified?

Something along the lines of:

  • stop all dockers and stop array
  • move the 120gb drive to unassigned
  • assign the new ssd to cache
  • run a subset of the commands above to update the docker container paths
  • restart the array and dockers

The alternate is to follow the unraid cache drive swap instruction, move appdata off the cache to the array, swap drives, move appdata back onto the (new) cache drive, then follow the instruction above. That just seems a bit circular since the data ultimately ends up on the same drive where it started.

Thoughts/suggestions?

The method you described will work. I just ran through the scenario on a test rig I have set up and it worked with no issues.

I’m not all that familiar with the Linux commands, would it just be the mkdir and mv commands that get skipped?

Ok so there are a couple caveats to my previous “it works” reply. First, you need to make sure the appdata is actually on the cache drive. Unraid doesn’t enable caching for the appdata by default, you have to manually change it. If you did, then your idea will work. You’ll still need to go through most of the steps in the walkthrough above, you can just skip step 7 since the appdata is already on the drive. If you didn’t change the cache setting, then you’re going to need to run through the whole process above after swapping the drives. The second concern is other data on the drives. I was running on a test platform with no other data on the cache drive. I’m guessing you’re not. So you’ll need to move all of the other data off the drive. Check the cache settings for all of the shares except the appdata share, and for any of them that are set to Only or Prefer, change the setting to Yes temporarily, but make a note of the original setting so you can change it back once the drive is swapped. Then, run the mover to put everything on the array. Once the mover is finished, you can swap the drives and proceed with the rest of the steps.

2 Likes

Process worked like a charm for me, Thanks for the detailed write-up @Mthrboard.

This worked fantastically for me. One thing I had to do afterwards was to look at 'Fix Common Problems as I found some issues that looked like this:

The solution is to go to your docker set up for each and edit the path to be ‘RW/Slave’

Thanks @Mthrboard for a great guide!

2 Likes

Just a note that if you look at the steps now, there appears to be a regex substitution that will copy over the slave settings in each docker image

I’m getting an error when running step 10. Any advice?
Edit: I finished the rest of the steps and it seems like it is working, but I’d guess I still want that step to run?

That’s strange, I don’t see any errors in the command you entered. The good news is that specific command just switches the appdata map to slave mode, so it can be done manually. The downside is you’ll need to change it on every container individually. Go into the container config, click the switch in the upper right to switch to advanced view, then edit the appdata map and change the Access Mode to rw,slave. Then save the config and move to the next one.

That is odd. Well, at least I can do it manually. Thanks for the guidance! I can definitely tell how much faster Plex is at loading art.

Question for you. Why the need to change the access mode when moving it to an unassigned device? And do I need to change that whenever I download a new docker?

Setting the mount to slave mode ensures the container will be able to see the files contained inside it. Any folders you map that are on an unassigned drive need to mounted in slave mode. Step 10 above changes the templates for the containers so if you ever reinstall the same container it will mounted properly. If you install a new container, you will need to update it manually.

FWIW, I haven’t changed to the slave option and haven’t had any issues with my containers. :woman_shrugging:

I probably should though.

I failed to do this when I made the change. Everything worked fine for about 3 days. Then I find that I had a run away log issue that was bringing my server to its knees and had to go back and fix all of my dockers.

I just made an account to say thank you so much @Mthrboard. The only hiccup I ran into was having to redo each Docker path, but thats acceptable for how simple the rest of this was.

Your description as why to do this too was exactly what I was experiencing and was trying to find a good way to go about doing this. Offloading my appdata to my nvme, and leaving my downloads folder on the SSD/Cache, because of the absurd IO wait I was experiencing.

Thanks again!

1 Like

Will this allow us to run docker images with the array stopped?
Given the amount of energy my SAS array takes and is still and issue of unraid not spinning down SAS drives, I want to be able to power up the server without the SAS shelve sometimes, but still be able to use some dockers and VMs,
My unraid is telling me Docker will start once the array is working… I already have my docker images in the cache drive.

No, you still need the array running to run any docker containers or virtual machines, even if those containers and VMs don’t touch the array.

I followed the instructions to a T but its not worked properly. When I startup Unraid I have to disable Docker before starting the array and then when its started I can turn on Docker but quite a few of them don’t work and the docker screen looks like this:

Could someone help please? Or at least get me back to how it was!