tim | TMI: or, toward a blog about Rust hacking (Reply)

In general, I don't blog about work. That's not a matter of policy, just that the topics that move me enough to cause me to sit down and write for a couple of hours tend not to be work-related. You might say that I should question the line of work I'm in because of that -- and believe me, I have. On the other hand, I do stay up late working because I have to know the solution to this problem rather than because there's a deadline (deadlines?) on a regular basis. And when I do find out the solution, I'm usually too tired to write about it. So there's that.

There are plenty of really good reasons to document what I do as I go along, though, and all of those reasons are centered around me rather than other people. Thus I'm going to start trying to write for an audience of one. But if I write here, in my public blog, rather than in a private text file, I'll be forced to write clearly enough that others might have a chance at understanding it -- which means I'll probably be able to understand it later myself, even if no one else ever does. So there's also that.

Some of my colleagues do a really great job writing beautifully detailed, explanatory posts, but I'm not going to try to do that, because it's just too intimidating. Instead, I'm going to write as close to every day as possible (though I won't beat myself up too much if I miss a day), as much as possible, and as uninterestingly as possible. You have been warned.

Today, I went back to working on implementing classes in Rust, as I've been doing for more or less the past four months. (The last two days of last week, I took some time to do bug triage and to fix what I thought was going to be an easy bug (to give myself that key little dopamine-surge of accomplishment) and turned out not to be.) When I left off working on it before, I was in one of those truly gruesome states where you're trying to add support for a new feature -- in this case, the ability to cast a class to an interface type -- but it breaks everything and you don't know why. In this case, I was trying to unify the code that handles implementations of interfaces (existing code), and classes that implement interfaces (new code), so I guess I did know why it broke (I messed up how interfaces got typechecked), but not how to fix it.

Rust still has a pretty slow edit-compile-run cycle (my understanding is that this is at least partly due to how we compile polymorphism by monomorphization, causing a certain amount of code blowup that overwhelms the LLVM back-end), and when you debug-via-printf (and is there any other way to debug?), that's really frustrating. My workflow, such as it is, looks like "add a one-line debug statement, recompile, wait 5 minutes, read the output and realize I need to add one more one-line debug statement somewhere else, recompile, wait 5 minutes..." It would be pretty boring to give you the blow-by-blow, but that's how I spent this afternoon. Starting out, I knew the bug was related to how Rust does separate compilation. As part of its library files, the Rust compiler writes out metadata needed to typecheck calls to functions in different crates (Rust's unit of separate compilation). If you know the GHC internals (of course you do!), it's analogous to GHC's "interface files". That metadata is in a binary XML format called EBML. For technical reasons I don't quite understand, there is no debugging pretty-printer for the Rust EBML library, and as with any parser/pretty-printer setup, it's easy for the two to get out of sync. That's what was happening today -- due to a bug I introduced, of course -- and given my lack of ability to just dump out the metadata that was getting written for the crate-that-was-getting used, as well as the metadata that was getting read by the crate-that-was-using, I could see no alternative except inserting lots and lots of debug printfs (imagine writing an HTML generator without being able to read the HTML it was generating directly).

Eventually, hitting the code with enough printf-debugging hammers led me to realize that I was printing out a type with a tag around it, but the parser expected the tag to not be wrapped in a tag. That is, when I write class A implements B, or impl A of B for C (an explanation of the syntax), any other crate that imports A needs to know what interface (B in this case) A implements, in order to do typechecking. So, the parser and prettyprinter just had different ideas about the representation of the fact "class or impl A implements interface B": it was going out as something that looked like

<tag_impl_iface>B</tag_impl_iface>

, when the parser expected to just see B. That was all, but the error message was, of course, singularly unhelpful -- just to the effect that the literal string "

<tag_impl_iface>B</tag_impl_iface>

" wasn't a type; of course it wasn't, but I couldn't see that since the string wasn't part of the error message! (Or rather, it was, but as the hex version of the binary representation of the document in EBML. Got that?)

I'm not sure what the moral of the story is, except that we need a pretty-printer. I fixed the parser to expect the extra tag, and then I returned to the excellent state where all the test cases pass except for the one test I'm working right now, and for that one I get a new and different error message. But that's for tomorrow.