Science and Research; Then and Now
There’s been a lot of talk about the need for disruption in the way that science and research are done. Traditionally, a small group of people (or even just one person) would work for a while on something on their own and then publish it in a conference or journal when it was somehow ‘complete’. This publishing was done mostly on paper.
The modern era has brought a huge amount of technology that could improve this process, but in practice all it seems to have done is make it possible to download a PDF rather than having to find the actual physical paper. In fact, crappy policies by conferences, journals and professional organizations have actually made even this advance inaccessible in many cases. (Matt Welsh has a great blog post about Research Without Walls which calls on researchers to not agree to submit or review work that will not be publicly available online.)
Further pointing to the fact that we’re not taking enough advantage of technology is the success of the polymath projects in leveraging a distributed and open group of people to solve hard math problems by letting them easily collaborate on ideas for making progress. Certainly, this shows we can do better. People are solving hard math problems with global-scale collaboration and we’re still having arguments about whether it’s OK for organizations to be able to hide publicly-funded research behind paywalls.
Disruption
Michael Nielsen has a great piece in the Wall Street Journal (which I think should be open to anyone) that goes into some details and also describes some of the hurdles that such a transition might face. The core message is that the incentive system we have now is broken and doesn’t encourage people to share their results, their data or generally collaborate very well. Instead, all it incentivizes people to do is to produce publications which in turn help their reputation and likelihood of getting funding.
His point seems right on, but managed to come across with a tone that makes me want to dig in deeper and ignore his advice. Sentences like “There are other ways in which scientists are still backward in using online tools,” simply make me want to scream. It’s not like scientists are actively trying to hide in a hole. We’re doing the best that we can with limited time to figure things out and responding to the reward structures we have.
Worse still, his tone is easy to pick up on and blogs that focus less on research are able to nearly parrot it back. This GigaOM post is a good example. It winds up pulling the same strings in me as the calls for the general public and congress to review what science is done in this country.
That being said, I agree. We’ve made crap for progress even in my field of computer science. I think that we do better than most with there being viable venues for people to present their work every 6 months or so (more often in areas like databases where VLDB now has monthly submission deadlines). Further, it winds up being something like 6 months between submission and presentation. That means even if it’s accepted right off the bat, it can take a year from ‘finishing’ something until it’s presented.
At the same time we still struggle with actually ‘publishing’ results in the good sense with data and open access to the papers. USENIX is amazing and lets you freely distribute your work, posts it online for free and even posts the videos of presentations online for free. Other organizations—ahem, ACM and IEEE, ahem—have been less forward-thinking.
That’s just dealing with bringing the old model of publication into the new area where faster publication schedules and wider dissemination are possible. It doesn’t do anything to address new forms of collaboration.
Resolution?
Sadly, while Nielsen does go on to explain some easy fixes that will at least aim to provide open access to data and papers, namely mandating it as part of grant approval, he offers very little to address getting to the real-time, cross-group collaboration which he starts talking about with the polymath project.
Really, all he has to offer is this:
Grant agencies also should do more to encourage scientists to submit new kinds of evidence of their impact in their fields—not just papers!—as part of their applications for funding.
The scientific community itself needs to have an energetic, ongoing conversation about the value of these new tools. We have to overthrow the idea that it’s a diversion from “real” work when scientists conduct high-quality research in the open. Publicly funded science should be open science.
I think it misses the broader issue which is that it’s hard to be a scientist or researcher today and the result is that we instinctively cling to any edge we have. The consistent cutting of higher education’s budgets and similar cuts in industry, Intel radically cut it’s collection of research labs in the last 2 years, have left far fewer true research positions available. When resources are scarce, it’s hard to convince people to share.
Sometimes we don’t share data because we think we can benefit from it if we hoard it. Other times we don’t collaborate because we worry about how credit will be divvied up. It’s entirely possible that these are actually non-issues. In fact, I think this is likely the case. Nonetheless, right now it’s dangerous to step out into this world since we don’t know the answers.
However, I think most of the time we don’t collaborate or share, it’s not out of malice or selfishness, but rather that it’s extra effort to share. I think that Nielsen underestimates the difficulty in releasing curated code or data sets. It’s not just a matter of posting a file to a web server. Even more so, researchers see little or no immediate benefit from such sharing.
Perhaps a first step would be creating a way for researchers to share some details of what they’re doing in exchange for people providing feedback and suggestions publicly. This model is already used to some degree when grad students and faculty give talks about their work in progress to closed audiences. Even so, I think it’s done too little and too late.
My Conclusions
In the end, I think that the goal of open access seems like something that’s relatively easy to obtain and will at least modernize the traditional publishing mode. The more ambitious goal of broad collaboration to make progress more quickly is tantalizing, but I think merely yelling at scientist to believe in it and do it is the wrong way to get there.