I love the idea behind Digital Humanities Now, and find the responses on the twitter stream about it both informative and interesting. The main concern has been with the mechanism for publishing. How is the stream of information, articles, ideas, posts, etc. being sorted? How do the editors work? Is it popularity based? How will the algorithm allow things to float to the top? All great questions.

But, nobody is asking what I view as an equally important question in open peer review. How can we make peer review work better? And, specifically, how can we make open peer review work better? Indeed, at least part of the value here is in openness. But don’t some of the same concerns we might have about the editorial stream also work on the peer review side. If the peer reviewers are drawn from a small coterie of individuals who are  most engaged, we will surely get a process that reflects their particular interests. Of course, this happens with all peer review right? But, if peer review as it is now constituted introduces some editorial bias, shouldn’t we imagine an open peer review process that works against such bias?

More to the point, how can we valorize peer review as critical to the entire endeavor of scholarly publication. We produce knowledge not as lone individuals, sequestered in archives. We produce knowledge within broader communities. Digital humanities has extended and built those communities, threatening all the traditional boundaries. Yet, too often, our concerns focus on the most obvious ways we contribute to those communities: by being individual ‘thought leaders’ and scholars. How about the more modest but vital ways that we work in building knowledge as collaborators, reviewers, and evaluators.

Hence,  I make a  modest suggestion.

I recommend, to our friends at Digital Humanities Now and Press Forward, as well as other open peer review publishers, that they consider some form of incentive, perhaps badges, as a way to a) incentivize peer review, b) diversify the pool of peer reviewers, c) create a sort of validated process for peer reviewers’ CVs that is recognizable (and, thus, tradable.)

Think about it. I have 3 hours a week to do professional service. I have many ways to perform this service. How, especially as I think about tenure and promotion, would I prioritize those choices. Presently, my choices are based on a time/cost calculation. I should do something that advances my argument for promotion/tenure. Thus, those activities that are most easily recognizable to my colleagues might make there way into service. Participating in open peer review is undeniably a good thing, but it will not necessarily be recognizable to my colleagues. In fact, they might resist my counting this for anything. Indeed, such folks already view blogging with suspicion because it is publishing without validation. Those same folks would view open peer review with equal suspicion. Imagine the criticism: peer review before you’ve established your reputation. Anyone can engage in this type of service. … yada, yada, yada …

Points and badging or some clear acknowledgement of value-added to this process strikes at the heart of that argument. It states clearly that a person has added to the process of producing a validated scholarly work. It is a clear marker of effort, investment, and engagement. It can be put on a CV and compared.

Likewise, there is another aspect of a badging system that might work to build the “reputation” of the endeavor (DH Now but an open publishing really) beyond the DH community.  Namely, by providing incentives, it widens the pool of potential contributors. It opens the process further. It builds a broader investment in the outcomes and process itself. It strikes directly at one of the sharpest and best critiques of any peer review–too narrow, not enough qualified participants, etc.)

I’ve written this as a stream of consciousness in response to a twitter conversation with Tom Scheinfeldt and Sharon Leon. So, it has all the errors you’d expect. But, I am convinced that there is the kernel of a good idea here for digital publishing.

I would describe that core this way. We believe that open peer review is the future. We often emphasize ‘open’ as key to succeeding as we should. However, we also need to emphasize the ‘peer review’ part. How do we make this aspect of the equation as critical a part of the formula as the front-end stream of inputs. I would argue that we should elevate and extend this important function. We could do this, for example, by badging. (And, I am sure there are much cleverer ways as well, including creating algorithms that measure the impact of the peer reviewers.)

Regardless, open peer review surely will work best if we emphasize and prioritize the “review”  portion of the equation, making it as important in its rewards as it is in our rhetoric.