Impetus

Well, that whole foreshadowing thing happened a lot quicker than I thought. When I attempted to kick up https://app.plex.tv/web/app# and was greeted with a server error, an initial wave of fear flew through me. I opened the NAS’s GUI and was presented with the following error:

Opening the error up, I got to see a bit more detail:

This isn’t fantastic, as a volume is essentially a hard disk, which is essentially a big block of code that consists of the data that I have been accumulating for years now. To get a better understanding, I looked at the Volumes section of the website:

More fear. This is essentially saying that the data has been blocked out. Given I have these alert states, I wanted to do a deeper analysis on any potential error that has occurred. It’s also weird because I didn’t receive any emails about it as I have already set the system up to do this. So, I navigate the following tree to download the logs; System > Logs > ‘Download Logs’:

Since I know that this is potentially a disk issue, that’s where I’ll try looking first. So, after the logs have finished downloading, I’ll navigate to the Downloads, open the .zip file, then find the log file name ‘disk_info.log’:

This log is where I’ll be able to find if there are any errors in the hard disks:

In my case, the issue is channel 2, or Bay 3 (given that when counting in computing, we start at number 0). The ATA Error Count is at 316. We can also see this in the ‘systemd-journal.log’ file:

And the ‘kernal.log’:

The Fix

I believe it’s safe to diagnose the third disk with an error. The ata4 error that we can see is tailored more to the fact that disks 1 & 2, 3 & 4, and 5 & 6 are the replication drives in this Raid 5 array. So, the error that we can see in 4 is more of a duplication from disk 3. Since we can only see errors primarily in disk 3, this error in 4 should disappear once disk 3 has been replaced.

To fix this, the first step I took was to restart the system. Thankfully, this has temporarily fixed the system, but the underlying issue still remains:

Backing up a Plex Library

Create a backup of Plex Library

Plex’s Media Server performs many scheduled tasks and one of those tasks is making regular backups of the core database(s). These backups hold information on various aspects, including the metadata that your server downloads for your media files. When your NAS happens to come across an error of some type, this has an impact on the .deb file (software package format for the Debian Linux distribution) that is used to install the server version, and you need to actually rebuild the library. This includes re-downloading metadata information. This is very annoying and, depending on the size of your library, time-consuming. This is why we’re performing the backups that we are.

Finding the Database Backups

The location of the database backups will vary by operating system. Unless you’ve altered the location via the Backup directory advanced setting, you’ll find the backup files under the Plug-in Support/Databases subdirectory inside the normal server data directory. If your Plex Media Server settings are in their default state, then the following should apply for the file location.

Using the Finder’s “Go” menu, select Go To Folder…:

Now enter the following in the dialog box that pops up:

~/Library/Application Support/Plex Media Server/

For the different operating systems, here is Plex’s default locations.

Since I am storing my Plex database on my Netgear NAS, I have a different file location. For me, it is:

/Volumes/data/.apps/plexmediaserver/MediaLibrary/Plex Media Server/Plug-in Support/Databases

The database backups will have the date in the filename and will be named along the lines of:

com.plexapp.plugins.library.db-<yyyy>-<mm>-<dd>

and

com.plexapp.plugins.library.blobs.db-<yyyy>-<mm>-<dd>

You can see these files in screenshot from my NAS (accessed through Mac’s Terminal):

com.plexapp.plugins.library.db-2024-06-08

com.plexapp.plugins.library.blobs.db-2024-06-08

Backup the Library

These library files are the backup file for those particular days. Once you’ve found the backup you wish to use:

1. Stop/quit your Plex Media Server by navigating to your NAS > Apps > Plex: Off

2. Move the following 6 files out of the directory and store them somewhere for backup, just in case. For instance, you could copy these to your local computer’s Desktop/Downloads folder. (The “-shm” and “-wal” are temporary files and may not be there.)
   1. com.plexapp.plugins.library.db
   2. com.plexapp.plugins.library.db-shm
   3. com.plexapp.plugins.library.db-wal
   4. com.plexapp.plugins.library.blobs.db
   5. com.plexapp.plugins.library.blobs.db-shm
   6. com.plexapp.plugins.library.blobs.db-wal

3. Duplicate the database backup file (com.plexapp.plugins.library.db-2024-06-08; com.plexapp.plugins.library.blobs.db-2024-06-08) into the correct directory and then rename them to the following (by removing the final ‘—
   ’ text):
   1. com.plexapp.plugins.library.db, and
   2. com.plexapp.plugins.library.blobs.db
4. Ensure that Plex Media Server has read/write permissions to the restored database file(s) (e.g. in a Linux install, the plex:plex (user:group) needs access),
5. Start your Plex Media Server by following step 1 again (but turning the app back on).

Automate the Backup

For this process, I will be using the native Mac application Automator (workflow). The following steps are the broad strokes that I use to complete the process:

1. Get Specified Server: smb://<server IP : server port>,
2. Connect to Servers (manually choose the data directory),
3. Run the necessary shell script.

Note: Yes, I use zsh over bash due to tab autocompletes and some GUI changes that I can use. If you’re interested in zsh, I came across ohmyz.sh when I set up my Mac in 2022.

Script

Since configurations can be different, here is a copy of the scripts that I use for ‘general’ and ‘weekly’ backups.

General script

#!/bin/zsh

# Define source and target directories

SOURCE_DIR=”/Volumes/data/.apps/plexmediaserver/MediaLibrary/Plex Media Server/Plug-in Support/Databases”

TARGET_DIR=”/Users/jamesmiller/Scripts/Plex/backups/General”

# Create the target directory if it doesn’t exist

mkdir -p “$TARGET_DIR”

# List of files to copy

FILES_TO_COPY=(

    “com.plexapp.plugins.library.db”

    “com.plexapp.plugins.library.db-shm”

    “com.plexapp.plugins.library.db-wal”

    “com.plexapp.plugins.library.blobs.db”

    “com.plexapp.plugins.library.blobs.db-shm”

    “com.plexapp.plugins.library.blobs.db-wal”

)

# Loop through the files and copy them

for file in “${FILES_TO_COPY[@]}”; do

    if [ -f “$SOURCE_DIR/$file” ]; then

        echo “Copying $file to $TARGET_DIR”

        cp “$SOURCE_DIR/$file” “$TARGET_DIR”

    else

        echo “$file not found in $SOURCE_DIR”

    fi

done

# Verify if files were copied

echo “Files in target directory:”

ls -l “$TARGET_DIR”

Weekly script

#!/bin/zsh

# Step 1: Mount the NAS directory

echo “Mounting NAS directory…”

osascript -e ‘mount volume “smb://username:password@192.168.1.4/data”‘

# Step 2: Define source and target directories

SOURCE_DIR=”/Volumes/data/.apps/plexmediaserver/MediaLibrary/Plex Media Server/Plug-in Support/Databases”

TARGET_DIR=”/Users/jamesmiller/Scripts/Plex/backups/Weekly”

# Check if the source directory exists

if [ ! -d “$SOURCE_DIR” ]; then

    echo “Source directory does not exist: $SOURCE_DIR”

    exit 1

fi

# Create the target directory if it doesn’t exist

mkdir -p “$TARGET_DIR”

# Log the directories

echo “Source directory: $SOURCE_DIR”

echo “Target directory: $TARGET_DIR”

# Step 3: List all files in the source directory

echo “Listing all files in source directory…”

ls -l “$SOURCE_DIR”

# Step 4: Extract dates from filenames and determine the most recent date

echo “Extracting dates from filenames…”

dates=$(ls “$SOURCE_DIR” | grep -oE ‘[0-9]{4}-[0-9]{2}-[0-9]{2}’ | sort | uniq)

recent_date=$(echo “$dates” | tail -n 1)

if [ -z “$recent_date” ]; then

    echo “No files with date pattern found in filenames.”

    exit 1

fi

echo “Most recent date found: $recent_date”

# Step 5: Find and copy files with the most recent date

echo “Copying files with the most recent date…”

find “$SOURCE_DIR” -name “*$recent_date*” -exec cp {} “$TARGET_DIR” \;

# Verify if files were copied

echo “Files in target directory:”

ls -l “$TARGET_DIR”

Script (cont.)

If you want to follow this process outside of Automator, you can create your scripts in TextEditor and make the necessary adjustments as you develop it. To do so, I can suggest:

1. Opening TextEdit,
2. Create a new text file,
3. Formate > Make Plain Text:

4. Enter the code,
5. Save <filename>.sh,
6. Open Terminal and navigate to where the script is being stored,
7. Type in, then press enter: chmod +x <filename>.sh,
8. Finally, type ./<filename>.sh.

This will output the script and you will be able to determine the from and to destinations, initiated and established server connections, and the files that have been copied over.

If you’re interested in the Automator files, they can be accessed in this Google folder.

The Final Fix

The final fix for this problem required me to replace the disk (Western Digital Red, 6TB). This NAS allows the hot swappable option (ReadyNAS should automatically detect the new disk and begin rebuilding the RAID array), but that has always freaked me out so I will choose the other option — power down the device, change the disk over, reinsert, and boot back up.

This was the GUI after I booted it back up:

This was the alert:

This was the volume page. This is where you can also monitor the health of the disk(s):

Conclusion

I’m not going to lie. The major point of me making this post was a way for me to (finally) keep track of a finalised approach that I take to manage and fix any of these issues that I have. Do you have written processes for the issues that you come across? What about change logs? It was a process that I started about 10 years ago, but I have to admit that the process has fallen by the wayside. I do enjoy tracking these changes as they come up though. I’d love to see how you track your processes — feel free to comment below how you track things.