rsync+hfsmode is a version of rsync which recognizes the HFS+ resource forks and Finder metadata in Mac OS X files, and properly copies them to remote filesystems. The destination can be any OS and filesystem that supports rsync, so you can use rsync+hfsmode to safely archive all of your Macintosh files[*] to servers running FreeBSD, Solaris, Linux, et cetera.
Newer versions of standard rsync (3.0.0+) will copy resource forks and Finder metadata between two Mac OS X systems (only). If that suits your requirements, by all means use that instead — rsync 3 also adds many other useful new features. Otherwise, read on.
To build rsync-2.6.3+hfsmode-1.4a1
Requires Xcode [4.1 GB].
Downloading the proper rsync+hfsmode binary from the links above
is recommended for non-developers.
bash$ tar xzf rsync-2.6.3.tar.gz bash$ patch -p0 -u < rsync-2.6.3+hfsmode-1.4a1.diff bash$ cd rsync-2.6.3 bash$ LDFLAGS="-framework CoreServices" ./configure bash$ make proto bash$ make man bash$ makeNOTE:
make man will fail, unless you
happen to have yodl2man installed. Don't worry about it.
Mac OS X uses the HFS+ filesystem. HFS+ files are often composed of a data fork, a resource fork, and Finder metadata. The data and resource forks contain what you normally think of when you think of file information: data, program code, etc. Finder metadata includes information like file type and creator, comments, modification dates, locked and invisible status, and Finder colors.
Traditional UNIX filesystems only store a single stream of file data (the HFS+ equivalent of the data fork). Mac OS X (or Darwin, more precisely) is a genuine BSD UNIX, but with a nontraditional UNIX filesystem. Because of this, standard UNIX tools can only see certain portions of OS X files.
The difficulties caused by HFS/HFS+ aren't new. Early Mac users of BBSes and the Internet had the same problems when uploading files from a Mac for storage by other operating systems. Since the foreign OS had no way to store the additional HFS data, the uploads would be incomplete and useless. To solve this problem, Apple and others invented serialization formats that collected the full set of file information into a single data stream. Examples (some with varying and/or additional design goals) include AppleSingle, AppleDouble, MacBinary, BinHex, and Stuffit.
Apple released Mac OS X in 2001, and suddenly decades of software written for traditional UNIX systems could be used on Macs, but some of that software would need to be updated before they would be able to work with all of the components of HFS+ files.
One such standard UNIX tool is rsync, an excellent file synchronization utility which is also great for use as filesystem backup software. Rsync builds and runs without errors on OS X, but because it is unaware of resource forks and other metadata, it creates incomplete (and therefore corrupt) backups. [UPDATE: Newer versions of rsync (3.0.0+) are now aware of resource forks and metadata, but still cannot transfer them to non-Mac OS X systems.]
rsync+hfsmode is HFS+ metadata-aware. Resource forks and Finder metadata are assembled on the sender into a synthetic file in standard AppleDouble format before being sent to the destination.
This method preserves disk space on both sides, with zero redundant data and only a small amount of overhead per file [*]. It works with any destination filesystem and operating system (tested primarily with Solaris and Linux), and even with older or unpatched versions of rsync.
This is currently a one-way, backup-only process. To restore from
the backup to an HFS+ filesystem, reassemble the AppleDouble files
with an Apple utility named FixupResourceForks
(see simple instructions below).
rsync+hfsmode names AppleDouble files according to the Netatalk naming scheme. This means that if your remote filesystem is served over Netatalk to other Macs, reverse conversion is done automagically. It's also convenient for the rest of us.
For a source file named filename, the
destination filesystem will store two files:
filename, containing the source file's data
fork, and ._filename, containing the resource
fork and Finder metadata, in AppleDouble format.
Apple's /System/Library/CoreServices/FixupResourceForks
will reassemble the two component files into a single HFS+ file.
bash$ ls -a
._filename
filename
bash$ /System/Library/CoreServices/FixupResourceForks filename
bash$ ls -a
filename
FixupResourceForks works recursively, so to reassemble
a large number of files just specify the directory name
(or ".") in place of filename,
above. FixupResourceForks will reassemble each pair of
files beneath that point.
Backups are important, test them regularly!
Using rsync option --hard-links and --hfs-mode
together would cause file corruption on the destination filesystem. The
current version of rsync+hfsmode will print an error message and exit if
you try. Future versions might resolve this problem more elegantly.
To back up your home directory:
bash$ rsync --archive --delete --verbose \
--hfs-mode=appledouble \
--delete-excluded \
--exclude=.Trash \
--exclude=Cache \
--exclude=Caches \
/Users/username fileserver:/backups/username
To back up your whole machine:
bash$ rsync --archive --delete --verbose \
--hfs-mode=appledouble \
--delete-excluded \
--exclude=/dev/fd \
--exclude=/.Trashes \
--exclude=/Network \
--exclude=.Trash \
--exclude=Cache \
--exclude=Caches \
/ fileserver:/backups/machinename
Marion Bates has also written an article explaining how to use rsync+hfsmode in a backup strategy.
FixupResourceForks is very simple.--disable-times will force checksums for all files,
which could be a good idea when for a backup plan, but nothing currently
exists to force checksums for synthetic files only.Let me know if you have any problems (or great successes) with rsync+hfsmode: reynhout@quesera.com