first released: 12Sep2004
last updated: 30Jan2012
rsync+hfsmode is a version of rsync
which recognizes the HFS+ resource forks and Finder metadata in
Mac OS X files, and properly copies them to remote filesystems.
The destination can be any OS and filesystem that supports
rsync, so you can use rsync+hfsmode to safely archive
all of your Macintosh
to servers running FreeBSD, Solaris, Linux, et cetera.
Newer versions of standard rsync (3.0.0+) will copy resource forks
and Finder metadata between two Mac OS X systems (only). If that suits
your requirements, by all means use that instead — rsync 3 also
adds many other useful new features. Otherwise, read on.
rsync-2.6.3+hfsmode-1.2b2 (STABLE on PPC or Intel/Leopard, BROKEN and WON'T BUILD on Intel/Snow Leopard+)
rsync-2.6.3+hfsmode-1.4a1 (ALPHA! this will supersede 1.2, but is presently for TESTING ONLY!)
Rsync source is available from rsync.samba.org
To build rsync-2.6.3+hfsmode-1.4a1
Requires Xcode [4.1 GB].
Downloading the proper rsync+hfsmode binary from the links above
is recommended for non-developers.
bash$ tar xzf rsync-2.6.3.tar.gz
bash$ patch -p0 -u < rsync-2.6.3+hfsmode-1.4a1.diff
bash$ cd rsync-2.6.3
bash$ LDFLAGS="-framework CoreServices" ./configure
bash$ make proto
bash$ make man
will fail, unless you
happen to have
installed. Don't worry about it.
A Short History of the Problem
Mac OS X uses the HFS+ filesystem. HFS+ files are often composed
of a data fork, a resource fork, and Finder metadata. The data and
resource forks contain what you normally think of when you think of
file information: data, program code, etc. Finder metadata includes
information like file type and creator, comments, modification dates,
locked and invisible status, and Finder colors.
Traditional UNIX filesystems only store a single stream of file
data (the HFS+ equivalent of the data fork). Mac OS X (or Darwin,
more precisely) is a genuine BSD UNIX, but with a nontraditional
UNIX filesystem. Because of this, standard UNIX tools can only see
certain portions of OS X files.
The difficulties caused by HFS/HFS+ aren't new. Early Mac users
of BBSes and the Internet had the same problems when uploading files
from a Mac for storage by other operating systems. Since the foreign
OS had no way to store the additional HFS data, the uploads would be
incomplete and useless. To solve this problem, Apple and others
invented serialization formats that collected the full set of file
information into a single data stream. Examples (some with varying
and/or additional design goals) include AppleSingle, AppleDouble,
MacBinary, BinHex, and Stuffit.
Apple released Mac OS X in 2001, and suddenly decades of software
written for traditional UNIX systems could be used on Macs, but some
of that software would need to be updated before they would be able
to work with all of the components of HFS+ files.
One such standard UNIX tool is
rsync, an excellent file synchronization utility which is also
great for use as filesystem backup software. Rsync builds and runs
without errors on OS X, but because it is unaware of resource forks
and other metadata, it creates incomplete (and therefore corrupt)
backups. [UPDATE: Newer versions of rsync (3.0.0+) are
now aware of resource forks and metadata, but still cannot transfer
them to non-Mac OS X systems.]
rsync+hfsmode is HFS+ metadata-aware. Resource forks and Finder
metadata are assembled on the sender into a synthetic file in standard
AppleDouble format before being sent to the destination.
This method preserves disk space on both sides, with zero
redundant data and only a small amount of overhead per file
It works with any destination filesystem and operating system
(tested primarily with Solaris and Linux), and even with older
or unpatched versions of rsync.
This is currently a one-way, backup-only process. To restore from
the backup to an HFS+ filesystem, reassemble the AppleDouble files
with an Apple utility named
(see simple instructions below).
Filename mapping details
rsync+hfsmode names AppleDouble files according to the
naming scheme. This means that if your remote filesystem is served
over Netatalk to other Macs, reverse conversion is done automagically.
It's also convenient for the rest of us.
For a source file named
destination filesystem will store two files:
filename, containing the source file's data
._filename, containing the resource
fork and Finder metadata, in AppleDouble format.
Reassembly of restored files
will reassemble the two component files into a single HFS+ file.
bash$ ls -a
bash$ /System/Library/CoreServices/FixupResourceForks filename
bash$ ls -a
FixupResourceForks works recursively, so to reassemble
a large number of files just specify the directory name
.") in place of
FixupResourceForks will reassemble each pair of
files beneath that point.
Backups are important, test them regularly!
Using rsync option
together would cause file corruption on the destination filesystem. The
current version of rsync+hfsmode will print an error message and exit if
you try. Future versions might resolve this problem more elegantly.
To back up your home directory:
bash$ rsync --archive --delete --verbose \
To back up your whole machine:
bash$ rsync --archive --delete --verbose \
Marion Bates has also written an article explaining
how to use rsync+hfsmode in a backup strategy.
Possible Future Enhancements
- Port rsync+hfsmode to rsync 3.
Many have requested this, and it's a great idea.
- Add bidirectional functionality.
Nice to have, but manual (or shell scripted) reassembly with
FixupResourceForks is very simple.
- Check for file naming collisions with synthetic
With any process that generates synthetic files, there is a
danger that the chosen filename will collide with a pre-existing
(but unrelated) file. Using a standard naming scheme reduces the
risk, but checking would be reasonable and might be helpful.
- Force checksums for synthetic AppleDouble files.
Synthetic files don't have meaningful modification dates, so checking
for changes by comparing date and size won't always work. Also, some
Finder metadata changes do not update the HFS+ file's modification date.
To be safe, we could force rsync to compare checksums on all synthetic
files. The added expense can be significant, however.
--disable-times will force checksums for all files,
which could be a good idea when for a backup plan, but nothing currently
exists to force checksums for synthetic files only.
(One sketchy way would be to send a known-incorrect size to the remote
system in the flist, so the file would fail the initial date-and-size
comparison and fall back to checksumming. Can't send a faulty date
instead, since that is independent metadata that will be used by the
remote side even if the files match checksums.)
- Handle rsync --hard-links switch more gracefully.
(instead of disallowing the switch in combination with --hfs-mode)
Let me know if you have any problems (or great successes) with rsync+hfsmode: