Slate has an interesting article on academic blogging entitled "Attack of the Career-Killing Blogs." The piece is largely pro-blog (at least the academic variety) arguing that "academic blogging represents the fruition, not a betrayal, of the university's ideals."
One of the quotes that most interested me was from John Holbo, who argued that the problem with blogs is the lack of peer-review, stating, "It is utterly absurd to propose giving someone credit for activity with no barriers to entry."
I'll have to disagree with his claim that it is absurd; in fact, the absurdity lies in the idea that credit may only be given to things that have barriers to entry. He has (apparently unintentionally) hit upon one of the primary problems of academic publishing: that the barriers are more important than the ideas. Ideally, the quality of the arguments should be judged by, er, the quality of the arguments, not how hard it is to get published.
Take, for example, the PMLA (Publication of the Modern Language Association). Publication in PMLA is relatively prestigious, since the article will be widely distributed to all MLA members. Grad students would love to have their High Theory piece published in PMLA. As a result, the barriers to entry into PMLA are very, very high. You'd think, then, that articles in the PMLA would be some of the most influential in the field, wouldn't you?
But, no, they're not. Don't get me wrong -- PMLA publishes lots of very, very smart stuff. But no one cites a PMLA article in their papers. I've never cited one in any paper, whether orally delivered in a conference or published. I've done lots of editing (both medieval and modern), and have never seen the PMLA cited. Presumably someone, somewhere, has cited the PMLA. The stuff in the PMLA is often smart, but it doesn't have an influence on par with its distribution.
In fact, although tenure and promotion committees rely on peer reviewers to vet the arguments of individual articles (T&P committees have neither the time nor the field expertise to do so themselves reliably), barriers to publication tend to be as political as intellectual. When you reach a certain point in your career, scholarly barriers to publication fall away and are replaced by the limitations of time.
Take my own case. I have three book projects at the moment, in various levels of completion. One is sitting on my desk in camera-ready form ready to go to the printer, one is in the process of having the individual articles edited, and for one the deadline for articles hasn't yet arrived. In all three cases, I was approached by someone else asking me to do the project with them. The only barrier is that of time (a 4/4 load doth not lend itself well to editing three books). The only articles I am working on at the moment were all requested, either as a request for a conference presentation or a request for a submission. The barriers to entry are all on the level of reputation -- once someone knew my reputation, they asked me to do some work for/with them.
Let's say someone publishes in a blog (or perhaps just on his own website) an article he has written. Let's then go on to say that the article, which is not peer-reviewed and is entirely self-published, becomes very influential. Should the scholar receive no credit for that?
Ideally, we would get credit for the quality of our ideas rather than the percieved barriers for publication. Of course, in the world of the practical, Holbo is right. My blog has more readers every day than the sum of readers of my articles in widely-distributed publications like Anglo-Saxon England and The Old English Newsletter. Yet I have no plan to put my blog on any T&P application, even under service; I recognize that although Unlocked Wordhoard is a key component to what I see as my academy duty to be a public intellectual, the academic community as a whole sees it as irrelevant.
Holbo's statement is absurd, and the absurdity is that it is true.