The fall term always gets a little overwhelming, as classes get into session and lectures need to be written, problem sets graded, exams created. I have an amazingly long backlog of things about which I want to write at this point but which I have not yet had the time to even get started. To top things off, my life appears to be changing forever. Most of these pieces get written on weekend afternoons while Lilah is napping. But the days of napping appear to be coming to a close.I understand intellectually that this is likely, after all, few 10 year olds nap, but I had never really stopped to think about the effect on my life. It’s not all bad; being able to pondering going out to do something with Lilah in the afternoon could be quite fun! But it will definitely gobble up my quiet afternoon writing time. But, today, after a late Halloween night and a no-doubt sugar-induced-early-morning wakeup, Lilah is currently snoozing away and I am going to now type as quickly as possible. Ready? Go! (Halloween? Yes, I started writing this almost a month ago, giving a perfect demonstration of the point I am trying to make.)
Back at the end of August I asked everyone to review my paper on Titan fog, and, to my surprise, many people took the task extremely seriously. The paper was discussed in classes and in on-line forums and was stared at by many eyes. If you recall back in August one of the reasons for attempting this open review was the fear that having only a single official reviewer leads to a huge random factor as to whether or not you will get anything useful out of the process. In this case, I have to say that the official review was pretty difficult to decipher. The reviewer commented on a few typos, complained about the location of the references, and said that the paper was generally incomprehensible.
Incomprehensible? Now, I will admit to having written papers that are incomprehensible before (how about this one; I can barely understand it myself 20 years later), but I actually thought that the paper was pretty clear. What’s more, of the many comments I had gotten from outside the official review process, no one had quite said “incomprehensible.” So what was going on here?
I reread the paper several dozens of times, and reread all of the comments that I had gotten, and realized, I think, the source of the problem. I think I was much too terse in my explanation of what I had actually done. Sure, I discussed fog and its discovery in gory detail. But I perhaps did not do a great job of describing how I really sorted through all of the data to find fog. It’s a pretty crucial step. If you don’t provide enough details in your paper that someone else could come after you and reproduce precisely what you did, you have failed an important point of having a paper in the first place.
One reason for describing all of this poorly was that the real process was actually quite different from the way I attempted to describe it in the paper. The real process consisted of this: I was looking at a bunch of pictures of Titan and said “Whoa; what the heck is that?” That turned out to be fog. I suspect that many discoveries are made that way, but if you read scientific papers you will rarely learn that fact. If you read my paper, you will find something like “Fog is very important so one fine day we decided to go look for fog on Titan. And we found it.”
OK, partially this description is true. After the first few times of accidentally seeing the fog we did, one fine day, systematically search through the entire data set. But that description was all pretty muddled.
The solution was a nearly complete rewrite of the paper. Had I just gotten the official review I would have fixed the typos and reworded a few things here and there and wondered what the heck the reviewer was talking about, but with the strength of the large number of comments, I could really tell what people were seeing and reading and I could make it significantly better. At least I think it is. But don’t take my word for it. Remind yourself of the first version, here. And now go read the new version here as it is about to appears in this week's Astrophysical Journal Letters. You still will not read the new version and realize that the real way we found fog is that we stumbled on it accidentally, but you will at least, I think, have a better idea of precisely what we did and how we did it. Want to go find fog yourself? I think the roadmap is now significantly more clear.
My conclusion from this experiment? I can’t tell you whether this system will always lead to such dramatic improvements in the quality of a paper, but in this particular case there is no doubt that when you read the two versions of the paper and you note any improvements almost all of those improvements came from the open review, rather than the official review. All of the comments that were sent to me were incorporated in one way or another. And for that, I would like to say a hardy THANK YOU to everyone.
But wait, there’s more!
Fresh on the heels of the Titan fog paper, I have submitted a paper to the Astronomical Journal called “The size, density, and formation of the Orcus-Vanth system in the Kuiper belt.”
This paper, I will admit, is less accessible than the paper about fog on Titan, yet, still, would you give it a read? It’s been posted online for a week and one reader already pointed out a rather stupid math error (thanks Alan Martin) of the sort that creeps into papers when you work on them one hour a week for 3 months (the error is still there, until we fix it in the next round of reviews, so feel free to go track it down and marvel at how stupid I sometimes can be).
Normally I would spend a few pages here telling you what the paper is about but, conveniently, I did that last spring, when we were searching for an appropriate name for the moon of Orcus. Go back and reread the post about coming up with names for the moon of Orcus, where I talk about the strange characteristics of what we now call Vanth. And, with a bit of continued Lilah napping over the weeks to come, stay tuned for thoughts about searching for the real Planet X, why I hate the 5 dwarf planets, and strategies for Lilah-weekend-nap-inducement.
And look! Lilah is done with her nap, and ready to start in on last night’s candy. Back to the sugar frenzy. (and with that, Lilah was awake, and we were off, and now it is a month later and Lilah is settled into a post-Thanksgiving nap and I finally have a spare moment to finish and post. Classes end next week for the year, so I look forward to a bit more time for reflection soon. Stay tuned.)