This is a really tough problem. I worked on something like this extensively in 2005. Ultimately, the hardest part was syntax highlighting on the current line, because you had to monitor input to figure out when a "token" had been completed, and then do the corresponding magic. Dealing with the idiosyncrasies of changing colors and maintaining cursor position makes this a real bitch.
Turning off syntax coloring on the active line definitely makes the problem easier. Wish I would have thought of that. Strange how you can spend days on a problem but never think of the "easy" solution.
I looked at EditArea before building the AppJet Code Editor (ACE). (See our in-browser code editor with syntax highlighting at appjet.com).
The problem with EditArea is that the overlay method causes visible ghosting when you insert newlines, even on a fast computer. Also not being able to see highlighting on the current line bothered me.
I based the ACE implementation on http://marijn.haverbeke.nl/codemirror/story.html only simplified the parser and added some features like paren/bracket/brace flashing. The basic idea is to use design mode but keep careful track of the DOM of the design body and cursor. The big challenges then become things like undo support (which we still haven't implemented yet), but the major advantage is that you get a fast and native feel, which is important. We plan to open-source ACE pretty soon.
Turning off syntax coloring on the active line definitely makes the problem easier. Wish I would have thought of that. Strange how you can spend days on a problem but never think of the "easy" solution.