Comments on Mike Brown's Planets: The problem with science

sorry, anon; my in box doesn't show any other comm...

2009-11-12T13:44:05.843-08:00

sorry, anon; my in box doesn't show any other comments from you. are you sure it got transmitted?

I guess I'm confused: why isn't what the competing...

2009-11-12T13:24:57.602-08:00

I guess I'm confused: why isn't what the competing group did a clear case of academic misconduct? Why aren't lots and lots of committees from the various institutions, publications, etc. looking into this? And why was my previous comment on these matters not published here? Surely you're not afraid of controversy or you wouldn't have mentioned this in the fist place?

Nice article, and in my view you did the right thi...

2009-10-28T14:11:13.682-07:00

Nice article, and in my view you did the right thing. Science shouldn't be about avoiding conflict or smoothing egos. There's too much real work to do.

A (partial) solution to these issues would be to have improved standards of publishing. The internet makes it very feasible to publish source data, analysis scripts, simulation code, etc. as part of the final paper. It seems we're still stuck in the pre-computer age, where all of the data and analysis are hidden from view. You can't really blame referees/editors from not catching these errors, since they have limited information to work with. So much is taken on faith. Yes, eventually the truth will out -- but it's an inefficient process.

If both teams published all of their imagery and tagged cloud locations, it would (a) put this disagreement on a more factual footing, (b) improve the quality of reviewing, and (c) dissuade people from publishing known-shaky analysis in the first place.

What we need is an arxiv on steroids. (And the end of proprietary publishing of publicly-funded work, but that's another topic.) Is anybody working on this?

Nice piece! Seems to me you put your finger on a m...

2009-10-03T16:03:20.750-07:00

Nice piece! Seems to me you put your finger on a most important point. One of the problems with science as a way of knowing is a frailty in the human spirit—old testament mixed with modern empiricism.

Thanks for sharing this story. I had a couple of s...

2009-10-03T13:12:12.768-07:00

Thanks for sharing this story. I had a couple of similar experiences with yours regarding science publications, though I work in a different sub-field of astronomy (extrasolar planets). I once refereed a paper for ApJ, for which a "detailed" mistake result in totally flawed conclusion. The authors withdrew the paper by actually agreeing with my report. Then they published their paper essentially unchanged in another journal!

Hurray for upstart planet hunters! That could only...

2009-10-01T14:26:37.430-07:00

Hurray for upstart planet hunters! That could only be a good thing!

Unfortunately, going back to the Kuiper Belt may n...

2009-10-01T14:09:58.262-07:00

Unfortunately, going back to the Kuiper Belt may not end the headaches. Latest word is there is are several upstart "planet hunters" looking for objects out there and they're no easier to deal with than the Titan people. In fact, they may be even more petty since some of them are hard core Plutophiles. Looks like going from the frying pan into the fire.

JohnD -- That is my point exactly; of course my he...

2009-09-29T15:43:41.348-07:00

JohnD --
That is my point exactly; of course my heart is overruling my head (or at least overruling my sense of politeness). In this particular case, though, it is not a question of making predictions to see which list is correct. It's simply a matter of looking at the actual data from the past and seeing which is correct. There, at least, there is no dispute. Their list contains errors. The question then comes: how much does that matter to the paper? That's where the difference in opinion lies.

Mike, Could it be that you have let your heart ove...

2009-09-29T15:31:37.738-07:00

Mike,
Could it be that you have let your heart overrule your head here? Retinr to scinece, and see if the different lists of Titanic cloud sites allow different conclusions to be drawn. Do testable predictions arise from those, that can discredit the other list, or dare I say it, yours?
JOhn

I kind of agree with Raven. I'd advise anyone juni...

2009-09-29T12:16:35.044-07:00

I kind of agree with Raven. I'd advise anyone junior to talk it though with a postdoc or advisor first, to see if they at least agree with you. But if you know you're in the right, and otehrs agree, then why shut up? While some might disagree with you, or think you're an arrogant arse, most people will respect you for it. As long as you're polite and professional about it you'll be fine.

Raven -- I'm with you. But I would also say be sur...

2009-09-29T12:16:21.865-07:00

Raven --
I'm with you. But I would also say be sure to choose your fights. Retreating one day to win the larger war is not necessarily unwise. That doesn't mean ignoring the problem. There are other ways of dealing with erroneous papers, such as simply never citing them and not considering the erroneous results as correct.

Myself, I've always been a bridge burner, but I consider that a personality flaw rather than much of a virtue!

My recommendation is just the opposite. How can yo...

2009-09-29T11:59:43.004-07:00

My recommendation is just the opposite. How can you justify doing something yourself and then tell others, just because they may have less career recognition than you, not to do it? Grad students and post docs shouldn't just go along with nonsense that passes as science. I say, burn the damn bridges. If you know you're right, stick with your position and you will be vindicated with time and more data. It would not be career suicide. What it would be is courage. As a grad student, I refuse to live by some double standard where you can get away with something while I can't. You're asking people to give in to intimidation and that's just plain wrong.

Eliat -- You're absolutely right. I can get away w...

2009-09-29T08:38:53.758-07:00

Eliat --
You're absolutely right. I can get away with burning bridges much more than I could 15 years ago, for example. My advice to a student or postdoc: don't do it. In the end you will be punished for pointing out major mistakes of others and it could take years (or a field change) to recover. Sad, cynical advice, yes, but that, again, is the problem with science.
Mike

Thanks for this. I wonder, though, how you woul...

2009-09-29T06:23:31.022-07:00

Thanks for this.

I wonder, though, how you would advise a person in your position who has not achieved the career status that you have. In a way, you are free to be bold like this and defend the purity of science, perhaps burning a few bridges along the way. But what would happen if a postdoc did something like this and pissed off a prominent scientist in an important group. Wouldn't that be career suicide? At the very least it would shrink the pool of future potential collaborators and jobs.
What advice, if any, do you have for more junior scientists who are faced with the dilemma of the integrity of science versus the urgency to publish?

"But I had never come across a paper where the mis...

2009-09-29T03:45:25.071-07:00

"But I had never come across a paper where the mistakes were pointed out before the paper was submitted for publication and the authors had not fixed them."

Do you live with a bag over your head?

That shit happens all the time. And it can't be your environment, because I know at least one person who has done that before going on to a job in your department.

If you don't correct these mistakes, who will? A grad student who puts their life on hold and lives in poverty for four years only to find the database on which their research is based is flawed? If you flee Titan for the Kupier belt, who is gonna keep the bastards honest?

I think you did the right thing by talking frankly...

2009-09-28T11:25:04.117-07:00

I think you did the right thing by talking frankly about it in your paper. Particularly if you'd discussed your concerns with them before! It's the least ass-like approach IMO. This is how science works isn't it?

Aah, so THAT's what the drama was all about - and ...

2009-09-28T07:39:29.280-07:00

Aah, so THAT's what the drama was all about - and I had feared, following your tweets over the weekend, that you had now proven that Mercury wasn't a planet either ... :-)

Where is your new paper published? I couldn't find it via either ADS or arXiv (and your homepage lacks a list of publications, it seems).

Even for those not exactly interested in details of Titanian meteorology the pair of paper might make quite interesting reading at some meta-level, given the different approaches.

Aww crap I waited too long to email -- I was going...

2009-09-27T23:27:17.044-07:00

Aww crap I waited too long to email -- I was going to ask if you need/wanted a volunteer, I'm in LA and found your posts and papers(at least the ones you put up here and on arxiv) fascinating and I was hoping there was some way that I could help. Well may be there's some other thing I can help you explore that isn't named Titan, but still has interesting weather. Expect an email/letter soon.

-Brent Markus

I have a colleague whose motto regarding publicati...

2009-09-27T22:23:19.009-07:00

I have a colleague whose motto regarding publications is that "It is better to be first - and to be wrong." That way you get more citations, and we all know that citations are the standard by which a paper is judged. I'm willing to bet you find that philosophy somewhat... distasteful?

Looking on the bright side, I doubt that you'll upset too many school children about this particular icy body.

Sadly, it's NOT difficult! the big mistake the oth...

2009-09-27T21:46:53.846-07:00

Sadly, it's NOT difficult! the big mistake the other guys made was to try to do it in some intelligent automated computerized way with no sanity checks. We went for the dumb look-at-every-picture-Cassini-has-taken approach and simply found the clouds. Dumber is sometimes better.

Mike - Wow, what a train wreck. Is there a simple ...

2009-09-27T21:42:06.567-07:00

Mike - Wow, what a train wreck. Is there a simple explanation why it's difficult to do this analysis?