Rick’s Rants and Raves

Updated 9/11/2007: MediaMax, mentioned below, stopped allowing FTP connections earlier this year. My off-site solution is better described by Skippy. End Update.

OK, I have an NSLU2 with two 80GB drives attached, working swimmingly on my network. I have gone through the not-too-difficult process of making it “Unslung” so that I can thoroughly modify it to my hearts content.

After the Unslung, the backup from drive 1 to drive 2 didn’t work. The error message was incredibly vague:

11/13 02:09:04 Drive Backup: Backup fail.
11/13 02:00:00 Drive Backup: Backup start.

It looked like it was working–files were being copied from one drive to the other. A little Googling turned up this message: http://www.nslu2-linux.org/wiki/HowTo/FixTheDriveBackupForUnslungFirmware

I followed the instructions, and got it working fine.

All this attention to the backup solution reminded me that my method for taking files offsite was less than ideal–for my family photos, I burned a DVD every (ahem, cough) 6 months or so, and took them to work.

So again, I searched around for NSLU2 Offsite solutions. I found a guy who had made it work with Amazon’s S3 service. I estimated how much it would cost to store my 5GB of photos on S3, and it looks like it would be less than $15 per year. Not too shabby at all. Perhaps I’ll go that route if this one doesn’t work.

I then came across a service from MediaMax.com. Free storage up to 25GB, with restrictions only on download, not upload. Also, they have an FTP service to make scripting from Linux possible.

So, I grabbed an account, and started playing with it. One of the limitations of the FTP service is that it is one-way on MediaMax. After you transfer a file to the FTP service, it gets moved to the real storage portal. So, I have to be able to remember locally which files have already been sent. I accomplished this by removing the User execute bit from the file I just transferred. This translates through Samba as the Archive bit in Windows. So it is easy to see from my Windows system whether a file has been sent offsite or not. So I wrote a BASH script to do the job. It’s quick. It’s dirty. It has been 6 years since I wrote shell scripts at work, so my techniques are probably a little rusty. So, if you want to use my script, here is what you need to do.

1. (I assume you have an NSLU2 with one or more drives attached. I also assume you know how to telnet and edit files in vi or some other editor).
2. Unslung
3. Install bash: ipkg install bash
4. Get an account with MediaMax.com (I am not a shill, they just happened to be the first I found with the features I wanted. Its free anyway)
5. Copy my script to your system in the location of your choice.
6. Modify the variables at the top. Note that MediaMax’s FTP service has the password set the same as the username. So if your user name is “MotherGrove”, in FTP only, your password will also be “MotherGrove”. I used two separate variables just in case that changes some day.
7. Add an entry to /etc/crontab to run the script periodically.

#!/bin/bash
IFS=”
UPLOAD=/full/path/of/the/files/you/want/to/store/offsite
WORK=/full/path/where/you/want/the/log/file
SERVER=ftp.mediamax.com
USER=YOURUSERNAME
PASS=YOURUSERNAME
function call_ftp {
eval FULL=$*
echo $FULL
DIR=`dirname “$FULL”`
FILE=`basename “$FULL”`
cd “$DIR”
echo `date` $FULL Starting FTP >> $WORK/offsite.log
ftp -in < open $SERVER
user $USER $PASS
bin
put "$FILE"
quit
EOF

echo `date` $FULL FTP Exit Status:$? >> $WORK/offsite.log
chmod ug-x “$FULL”
echo `date` $FULL CHMOd Exit Status:$? >> $WORK/offsite.log
}
if (($#==1)); then
call_ftp $*
exit 0
fi

find $UPLOAD -type f -perm -u=x -maxdepth 2 -exec $0 ‘”{}”‘ \;

Limitations (or future features)

• This script does NOT mimic your directory structure on MediaMax. All the files will be dumped in your “Uploaded Files” folder. If I ever need to download these, I’ll recreate the folder structure.
• FTP is very poor at reporting errors. I don’t know with certainty that the files are actually transferring. I just spot check my account on MediaMax to be sure I see them.
• Depending on your upload speed, it might take a few days to do the first upload. This script is not smart enough to check and see if it is already running, so you might want your initial CRON entry to be once a week.
• There is not a graceful way to stop it from running. I’ll probably add that in later.
• There is no upload throttling. If this matters to you, hopefully you have a router with QOS, (or one that can be hacked to provided QOS).
• Files are stored unencrypted in your account on MediaMax. I do not know how robust their security policies. If you send sensitive information, please encrypt it before you send it offsite.

Enjoy!