Late last year I made the switch to Adobe’s Lightroom 4 from Aperture and Picasa for a variety of reasons, but a big one was photo/file management. As my photo collection got larger I found that I’d need more than my laptop’s hard drive to store them all. Despite all the things I liked about Aperture and Picasa, both had clunky file management and the burden was enough to make me consider alternatives. I’m now very happy with the workflow described below using Lightroom, Dropbox, Crashplan, and a Linux file server.

What I Want to Accomplish

I want my setup to enable the following:

  1. Mobile Photo Management: To be clear, when I say “mobile,” I don’t mean smartphone. I take my Canon 5D Mark III everywhere and like to transfer photos to my Apple MacBook Pro immediately. It’s important that I be able to do whatever I need to do no matter where I am, i.e., I don’t want to be tied to my office.
  2. Large Photo Archive: Photos (especially RAW photos) consume a lot of disk space. I want a file server to store any photos that I’m not actively working on. It has to be huge and scalable.
  3. Passive Transport: I want my files to get to my file server as quickly as possible, but I don’t want it to be an active part of my workflow, e.g., I don’t want to get stuck waiting on a FTP program to finish its batch before I can disconnect from a wifi spot or put my computer to sleep. I’d rather it just happen when the opportunity arises.
  4. One-Time, Offsite Backup: Once my files are on the file server, I want them to back up from the server only (i.e., I don’t want my laptop doing a second backup). Also, as I’ve noted before, I like offsite backup of my photos. A local copy isn’t good enough.
  5. Accessibility. When home, I want to easily access all of my photos, whether they’re on my laptop or in the archive.

Sound like a lot to accomplish? It turns out it’s pretty easy to do.

The Right Tools For The Job

I use 4 tools for the following purposes:

  1. Mobile Photo Management: Lightroom 4.3
  2. Large Photo Archive: Linux File Server (any OS will do if it meets your storage needs … more later)
  3. Passive Transport: Dropbox
  4. One-Time, Offsite Backup: Crashplan
  5. Accessibility: Lightroom 4.3

Now let’s walk through each.

Mobile Photo Management

I use an Apple MacBook Pro and Adobe Lightroom for just about all of my photo work. The MacBook Pro is really just personal preference. Any laptop with decent specs running OS X or Windows (needed for Lightroom) will work just fine.

As for Lightroom, there are endless articles online describing the strong points of what it lets you do with your photos. I won’t reiterate those points, but will simply say it’s fantastic both for viewing and processing your photos. It’s a perfect on-the-road application that lets you quickly import, tweak, and export your photos to wherever you intend to share them.

What I will focus on is how it integrates with the other needs I listed, so let’s move on to those areas.

Linux File Server for Large Photo Archive

I essentially did 2 things:

  1. Built a 5 TB RAID Array - all the details you need are on that link
  2. Configured an Apple File System service on the Linux server using netatalk and avahi

I won’t go into details of those applications as there are plenty of other articles online documenting what they are and how to set them up, but the gist is that netatalk enables the sharing of a folder on your server to OS X, and avahi advertises it on your local network so that you see it in Finder. The key configuration after installing each is:

For netatalk, in /etc/netatalk/AppleVolumes.default:

"Media" -tcp -noddp -uamlist uams_guest.so,uams_dhx.so,uams_dhx2.so -nosavepassword

And in /etc/netatalk/afpd.conf:

# Access The Storage Area On Your Server
/mnt/store/private/photos "Photos" allow:markmcb

For avahi, in /etc/avahi/services/media_group.service:

<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name replace-wildcards="no">Media</name>

<service>
<type>_afpovertcp._tcp</type>
<port>548</port>
</service>

<service>
<type>_device-info._tcp</type>
<port>0</port>
<txt-record>model=RackMac</txt-record>
</service>

</service-group>

If you get that all set up, you should see something like the following in Finder (using what I listed, you’d only see the “Photos” folder).

If that all made sense, the rest of this steps in this article are quite easy in comparison. :)

Passive Transport

First, if you’re not already using Dropbox, check it out. The concept is simple: for every device you install Dropbox on there is a Dropbox folder. If you put a file in the Dropbox folder on device A, it uploads to Dropbox and downloads to devices B, C, D, etc., automatically. In other words, it’s a folder that syncs across devices. The key here is that it does this automatically, i.e., you don’t tell Dropbox to start a transport, it just does it once a file is saved and an Internet connection is available.

All we need to do is:

  1. Install Dropbox on the laptop
  2. Install Dropbox on the Linux server
  3. Create a folder somewhere in Dropbox for our active photos
  4. Add that folder to the folders list in Lightroom

For me, #3 is ~/Dropbox/Photos/Master Workspace, and when added to Lightroom it looks like:

As you can see, I have 313 active photos in this workspace. As soon as I import new photos into Lightroom, Dropbox will start to upload them. Let’s say I take 100 photos and import them from my hotel room. As soon as I import them, the upload starts. Let’s say it’s a slow connection and 20 have finished when my friend calls and wants to meet up for lunch. With Dropbox, I just pack up my laptop and go. Whenever I open it next and get on the net, it will resume. In the mean time, those 20 photos that transferred are being downloaded to my Linux server that’s always online. Once the server gets the copy, it backs them up (more on that in the next section). It’s important to note that the backup process is a separate one. While Dropbox offers some backup characteristics, you really shouldn’t depend on it as a backup solution.

The benefit here is that, when I’m away for a few days or weeks, my photos automatically make their way to a safer place on my server. In general I like to have my laptop be a “throw-away laptop.” That’s not to say I’d be happy if I lost it, but I like knowing that data loss from such an event would be minimal or none. Dropbox helps a lot in making that true (and not just for photos).

One-Time, Offsite Backup

Dropbox has now sent the files to my Linux server’s Dropbox folder. This is one of two places my photos reside in this process. Once I am done actively working with them, they will move to the archive folder that we shared earlier.

I use Crashplan for backup. I’ve been using it almost two years now and it’s really a tremendous service for the price. One of the cool things it does is smartly determine if a file needs to be backed up, or if you’ve already backed it up before and a copy is on their servers. This is critical for the workflow I’m describing. Here’s what happens:

  1. From my remote location, I import photos to Lightroom
  2. Lightroom stores the photos in my laptop’s Dropbox folder, let’s call it D1
  3. D1 is uploaded to Dropbox
  4. The Dropbox folder on my server, D2, gets the files pushed down
  5. Crashplan is watching D2 and immediately starts backing up the files to the offsite backup destinations I’ve designated.
  6. Crashplan is also watching the archive folder, A1. In the next section we’ll use Lightroom to move files from D1 or D2 to A1. The nice thing Crashplan does is realize that the files being moved have already been backed up and simply updates some file pointers.

That last step is HUGE if you work with RAW files like I do. When you need to transfer 500 RAW files at 30+ MB each, the last thing you want to be doing is chewing up more bandwidth than you have to. Granted, you’re already moving the files twice: once through Dropbox, and once through Crashplan. However, if you started the process remotely it’s not bandwidth you’re going to miss, i.e., all or some of this happens when you’re away and is generally finished when you sit down at your home computer to work. I really, really, really love this aspect of this workflow.

Accessibility

The last point I’ll touch on is accessibility. Once you’re home and ready to do some serious post-processing, you want to quickly get to all of your files regardless of where they’re physically stored. Lightroom 4 is a dream in this regard. In the same way we added the Master Workspace folder above, all we need to do is add the Master Archive folder on the Linux server that we made available via Finder. Once you’ve added the folders, your catalog acts as the brain and even if you forget to mount your shared drive, Lightroom give you intelligent feedback:

Lightroom shows question marks indicating it can’t find the folder. If you forgot to mount a drive, simply open finder and click on the shared drive to “mount” it.

Master Archive" unmounted.

Master Archive" unmounted.

And that’s it. Now you can click on any of those folders and work on them as if they were all stored on your computer.  If you drag a photo or folder from the Master Workspace to the Master Archive, then files move to their new home on the server and off your laptop and out of Dropbox (this is really good since even a “pro” Dropbox account only stores 100 GB of data).

Wrap-Up

As with anything this complex, there are a bazillion ways to do it. I have tinkered with quite a few setups and this one is where I ended up. I really like the flexibility, the saved bandwidth, the immediacy of backup, and the ease of access. I hope this helps someone, and if you’ve got variations of this, I’d love to hear about them.