TrueCrypt + Mac OS X + Advanced Format Drives = True

I bought a new hard drive from newegg.com the other day. It’s a rather nice Seagate Expansion STBV3000100 with a whopping 3 TB of storage (current price: $119). It’s also got a whopping 4 KB large sectors instead of the standard 512 byte sectors. This is apparently called the Advanced Format. It’s a fancy name for what seems like such a small difference, but it is apparently a bigger change than you might think since those 512 byte sectors appears to have been with us since the dawn of hard drives back in 1956!  Talk about staying backwards compatible forever…

But be that as it may, all my computers (Mac, Windows, Linux) seem to handle this change just fine, but TrueCrypt was less happy when I tried to initialize the drive with it. TrueCrypt is a great disk encryption app and I highly recommend it.  I run it on all my computers (Mac, Windows, Linux) and I use it to encrypt most of my external hard drives — especially those I tend to travel with.  Sadly, its update schedule is not especially frequent and the most recent version (7.1a) has been out for quite some time now. It still works great, even under Mac OS X 10.8 Mountain Lion, but it does seem to carry some rather old fashioned ideas about what works and what doesn’t work under OS X. In particular, it’s rather set on the idea that anything but 512 byte sized sectors are ganz verboten when it comes to OS X. Or in other words, unsupported. In fact, it is so sure of this that it won’t even bother trying it out; it’ll just post a big juicy alert saying:

Error: The drive uses a sector size other than 512 bytes.

Due to limitations of components available on your platform, partition/device-hosted volumes cannot be created/used on the drive.

Possible solutions:
– Create a file-hosted volume (container) on the drive.
– Use a drive with 512-byte sectors.
– Use TrueCrypt on another platform.

The error message is very well written and quite informative, but it is also quite wrong. It may have been true once upon a time, but at least not since Mac OS X Snow Leopard came out in 2009 according to Wikipedia’s article on the Advanced Format.  So to try this out, I downloaded the latest source for TrueCrypt 7.1a. I then dug through the source and removed all the 512-byte restrictions that I could find and — after some initial hair pulling to get the build environment right — recompiled.  And lo, it ran!  Not only that, it accepted my AF hard drive and started reading & writing to it without a care in the world. Success! Or at least so I thought…

Merrily, I started copying the files from my old 2 TB drive onto it. The copy process ran all night and would probably have run for another day or two too if I hadn’t noticed something odd. For some reason or another (call me paranoid, I don’t care), I decided to verify a few of the files that I had copied and lo again: They did not match! Grumble, grumble, grumble. I stopped the copy process and looked closer at them. Some of the copied files’ data matched the originals and some didn’t, but there was no rhyme and reason to it.  Most of the the corruption appeared to happen on multiples of 4K byte blocks, but there were a few cases on 512-byte boundaries too, maybe < 10%.  Worse, re-reading the files yielded different results each time! WTF? Weren’t 4K sectors supported under Mac OS X after all or was there some other glaring bug or limitation  that I  had missed?

The next several days was spent deep in the TrueCrypt code trying to find the responsible culprit. It was all somewhat complicated by the fact that the TrueCrypt app spawns several subprocesses that each runs several threads, so it’s somewhat hard to control under the debugger. There’s also the case of the data path, which is slightly convoluted under Mac OS X. In order to present a decrypted hard drive to the OS, TrueCrypt uses a subprocess that will implement a FUSE file system with two files: “control” and “volume.dmg”. These are mounted hidden on /tmp/.truecrypt_aux_mnt1. From what I can tell, “control” holds miscellanous details about the encrypted hard drive for TrueCrypt’s internal use while “volume.dmg” represents the decoded data from the hard drive. But that in turn needs to be mounted as another file system in order to get access to the real files and this is accomplished by TrueCrypt using hdiutil to “attach” the .dmg file, which will create a new /dev/disk pseudo-device, which in turn will get probed and mounted by the standard file system utilities.

I’m sure I’ve lost most if not all of my non-techie readers by now and the rest of you are probably having hard time staying awake, but just for the record, the culprit turned out to be the EncryptionThreadPool that attempts to parallelize the work by dividing up the data to be encrypted or decrypted into as many fragments as there are CPU cores on the host computer and letting as many separate threads process each fragment simultaneously. It did a great job at this, but when dividing the data to be en-/decrypted, it used the wrong size factor which caused the fragments to be overlapping and the threads to be stepping on each other’s toes. What size factor was used? What do you think? Yup, 512 bytes was hardwired into the code instead of using the host drive’s sector size. A leftover from before variable sector sizes got added to TrueCrypt, I’m sure. Simple to fix in any case.

A little copy & paste and a quick recompile yielded a new TrueCrypt.app. And lo, take III! The files now copy with perfect fidelity, each one an identical to its original. The world was right again — and more importantly, I was finally able to actually use my nice new hard drive the way I wanted.  Now, I could have used some other disk encryption software, but none of them is as portable or free as TrueCrypt. I could also have formatted the drive unencrypted and then used a file container with TrueCrypt to hold the encrypted file system, but that would mean going through the file system driver twice each time I want to access a files. Yech. So instead I spent several days digging through random code until I got my favorite tool working just like I wanted it to. I admittedly swore over my own idiocy many times in the process, but I guess it really does pay to be stubborn sometimes…

OK, long story over. Here’s finally a link to the precompiled binary if you’d like to try it yourself too: http://bit.ly/tc71alen. You’ll need to download OSXFUSE and install that too in order to run it. It comes with a patch file in diff format if you want to build it yourself too, see the enclosed ReadMe.txt file for details.