Sounds to me like the product of many years of evolution with many developers working on it, and all that at a large company. Working at a large company myself, I see how these messed up pieces of code / specifications come into being every day.
Exactly. What version of Photoshop dear HN reader was your first? Think about all the features that have been added since then and how hard it must be to maintain even a modicum of backwards compatibility.
Another thing to consider: a file format as popular as PSD probably had a great deal of interest from competitors. Sometimes file formats get obfuscated simply for the purpose of breaking the other guys' import filters.
I've seen this behavior in the file format of an extremely popular application. IIRC, in one section, there were magic bytes that told you the alignment of other blocks that told you how to look up the version of the algorithms that you were supposed to use to unpack the binary data blobs that held resource data. By jiggling around the various forms of indirection, it was possible to make the file formats nigh indecipherable from version to version, while still maintaining backwards compatibility.
Naturally, the parsing code was a total nightmare.
This doesn't sound unlike the Office file formats. Apache's POI project named its Office binary reader/writers HSSF and HWPF, acronyms for Horrible SpreadSheet format and Horrible Word Processing Format respectively. IIRC, they are basically serializations of the in-memory structures used by Office. This choice was made when a powerful Windows machine had less than four megs of RAM and manipulation of document data by a 33 mhz processor could take significant time. Of course, like Windows, Microsoft chose backward compatibility over forward movement -- a wise economic choice, at least in the short term.
