Making Digital Scholarship Count (2)

In my previous post in this thread, you may have noticed that I used the term “digital work” rather than “digital scholarship.” My choice of words was in no way accidental. Digital work encompasses everything historians do in the digital realm–scholarship, teaching, and service. “Digital scholarship” is a precisely defined (or should be precisely defined) subset of “digital work.”

Thus, if we are going to argue that digital work should count in higher education, we need to define how this work counts in each of the three main domains of scholarly activity. This was something I had to wrestle with myself not too long ago when I came up for tenure here at George Mason. If you care to read my tenure statement, you can see how I parsed the digital work that I do across the categories. Thus, for instance, I claimed my work on large digital projects such as World History Sources and Women in World History as “applied research on student learning and technology” rather than as the equivalent of the book I published in 2006.

In my formal presentation to my department, I argued that these projects were roughly equivalent to a collection of essays that one might edit. After all, these two projects included dozens of essays (and hundreds of website reviews) by scholars of World History that I and my colleagues had to solicit, edit, and then publish. And taken together, these two projects won the American Historical Association’s James H. Robinson Prize, so they must have had some merit.

That I even had to make the case that they were roughly equivalent to something we already know well is evidence of how difficult the problem is.

Before we can even begin to claim that something called “digital scholarship” should count in the research domain of our professional lives, we would do well to define exactly what constitutes “scholarship.” Here, I think we have an easier task. In almost any discipline one cares to name scholarship has the following characteristics: It is the result of original research; it has an argument of some sort and that argument is situated in a preexisting conversation among scholars; it is public, it is peer reviewed; and it has an audience response.

There are exceptions, of course. A novel, a collection of poetry, a work of art, or a piece of music may all count as scholarship in certain contexts. But by and large, the characteristics I’ve described hold for most forms of scholarship.

This means that for digital scholarship to be scholarship it has to have all of these characteristics. But, as Stan Katz said in an interview I did with him for this series, digital scholarship “is a new form of scholarship.” In other words, we need to stop trying to pound the square peg of digital scholarship into the round hole of analog scholarship.

We’ll return to this crucial issue later. But for now, I think it’s easier to define what digital scholarship isn’t than to define what it is–especially because as we’ll see, it is an inherently moving target.

I think we would all agree that a course website or a series of lectures created in one’s favorite slideware program do not constitute scholarship. They may well be very scholarly, but on any campus I can think of, this sort of work falls clearly and unequivocally into the teaching domain.

Where it gets trickier is when we consider digitization projects–whether small in scale, or massive, like the Perseus Project or the Valley of Shadow. Each of these excellent and heavily used projects offers scholars, teachers, students, and the general public unique access to their content. But, as Cathy Davidson at HASTAC told me in an interview for this series, “the database is not the scholarship. The book or the article that results from it is the scholarship.” Or, I would add, the digital scholarship that results from it. In other words, I’m not willing to limit us to the old warhorses of the book or scholarly article.

I also want to emphasize that I have tremendous respect for the scholars and teams of students and staff who created these two projects–both of which I use often in my own teaching. But I also have to say that I don’t think either project can be considered “scholarship” if we use the definition I’ve proposed here.

Why not, you might well ask? The reason is fairly simple in both cases. Neither project offers an argument. Both are amazing resources, but neither advances our understanding of particular historical questions. They make it possible for that understanding to advance in ways that weren’t available before, but as Davidson says, it is what results from a project like these that is the scholarship. Thus, for instance, the article published by Valley of Shadow creators William Thomas and Edward Ayers —An Overview: The Differences Slavery Made: A Close Analysis of Two American Communities,American Historical Review, 108/5 (2003)–rises to the level of scholarship in our working definition.

In my next post, I’ll try to get to the heart of the matter–what would digital scholarship actually look like?

19 replies on “Making Digital Scholarship Count (2)”

  1. I like how you are proceeding with this series. The distinction between digital work and digital scholarship was really useful. It was smart to talk about what digital scholarship isn’t. The cogent definition of scholarship was spot-on. Still, I wondered if you left out an element. It seemed like it was implied in your reference to “original research.” What I am thinking about is the production of new knowledge.

    This got me thinking about the link between scholarship and technology. If they have anything in common, it seems like the quest for innovation is the hallmark of both. So what does this mean for digital scholarship? Does it mean digital projects must be innovative and produce knowledge that is fundamentally new in order to satisfy the criteria for tenure? If so, this seems like a fairly high, but necessary, bar for tenure.

    There is another other tension I am wondering about. The creation of new tools and technologies are often the goal of digital projects. Yet these tools and technologies are fairly neutral. They don’t make an argument; they serve a purpose. The best example I can think is Zotero. Zotero is innovative. It is the product of a lot of hard work. I like to think that it is in conversation with debates in the field of citation management. The user groups and forums must count as some form of peer review. There has been a tremendous response from thousands of users. For all these reasons, it seems like Zotero is the very kind of thing that should have helped someone to obtain tenure.

    Still, Zotero does not advance a historical argument. As you put it, Zotero is not something that “advances our understanding of particular historical questions.” It is because of this missing piece–if I understand you correctly–that Zotero can never count as digital scholarship. Instead, to paraphrase one of your interviewees, it is the secondary sources that result from using Zotero that will count as scholarship. Then again, maybe Stan Katz is right and I am mistakenly trying too hard to make digital scholarship conform to our current standards for historical scholarship. I look forward to your thoughts on all of this.

  2. This post touches directly on a struggle I’ve been having with undertaking a digital history project on as part of my master’s thesis in American Studies. I think my professors and I have concluded that the project itself (a mash-up of a WPA state guidebook with a wiki and Google Maps) is not sufficient to complete the thesis, although it is sure to be a lot of work. It would be a resource akin to Valley of the Shadow…digitizing source material and making it interactive. I’ll probably end up setting up the site for independent study credits and then undertake a study of it use for the actual thesis. I’m not sure I’m completely satisfied with that solution.

  3. I like what you’ve done so far, but I’d like to see you push the envelope a bit on what constitutes “scholarship” in general. For instance, I think this blog post should be considered scholarship, and should be counted as such by the academy.

  4. That’s a good point, Jeremy. Here’s Mills’ definition of scholarship: “It is the result of original research; it has an argument of some sort and that argument is situated in a preexisting conversation among scholars; it is public, it is peer reviewed; and it has an audience response.” That kind of sounds like a blog post, wouldn’t you say?

  5. Derek, I don’t think so. Blog posts often have nothing to do with original research; they often spark conversations with the general public rather than “scholars” per se; and they meld peer review and audience response into one.

    Basically, like it or not, blog posts are ephemera rather than scholarship. I see the internet as a means of bringing ephemera back into what I see as their proper place at the forefront of scholarly activity; I’d be willing to bet Mills disagrees with that.

  6. Perhaps this particular blog post is more like scholarship. It certainly has an argument that is “situated in a pre-existing conversation among scholars.” Just so I’m clear, do you see peer review as something that happens *before* a piece of work is made public and audience response as something that happens *after* a piece of work is made public? If so, then I would agree that blog posts lack peer review. There’s no vetting ahead of time.

  7. Mills,

    You have situated your own work as a case in digital scholarship very effectively. The idea that Women in World History is comparable to an edited collection of essays makes a great deal of sense. Collaboratively produced web resources involving original research such as digital exhibits that provide deep layers of analysis seem to fit this model as well. Other digital exhibits or physical exhibits for that matter, reflecting the research and analysis of one or two curators, may be considered comparable to a peer reviewed scholarly article.

    The situation is complicated by departments that include a formula for counting synthetic rather than discovery works as scholarship. In a department that includes textbooks among the works that are “counted” as scholarship for promotion, what is the distinction between a textbook and the digital productions related to a TAH grant project? What is the difference between a collection of edited primary documents or oral history transcripts and an online collection of primary documents?

    The question is further complicated by departments’ definition of “historical” scholarship. Is the study of how museums or educators explain and represent aspects of the past (digital or otherwise) “history”? Are peer-reviewed works about applied digital history and the impact of digital resources on history education historical scholarship? Would we consider a peer-reviewed, research-based analysis of the impact of Vertov, a Zotero-based application designed to “preserve the orality of oral history”, on the field of oral history as historical scholarship?

    In this period of change, the best strategy may be to have an open-ended clause in departmental by-laws that allows for “other works as agreed upon in writing by the individual faculty member and the promotion committee,” perhaps subject to the review of a dean or a college promotion committee. Departments may well want to reference their own categories as well as statements on scholarship put forward by the AHA, OAH, and other groups in negotiating these agreements.

  8. To everyone who has commented thus far on this series–thanks very much. I’ve been slow in responding because I was in Mexico at a symposium on “the future of digital history” sponsored by the NEH and the National Autonomous University of Mexico.

    So, since I’m behind, I’m responding to everyone at once here. That means a long response, but at least it’s a reasonably coherent one:

    PhDinHistory–I like the idea of new knowledge/tools, etc., but don’t think that’s entirely necessary. So often, scholars do something new with something old–use someone else’s methodology to accomplish something very interesting. So, something brand new would be excellent, but not required in my view. And on peer review, see my next post in this series. But as to Zotero, I don’t know how I feel about it counting as “digital scholarship” in history. It certainly would count in other fields, and so maybe we should count it as well, but I’m less convinced that history departments ought to. Universities, absolutely, but history departments? I’m just not sure.

    Tim— I’m familiar with your project from a post I wrote last year. I think you and your advisor are correct that it is not quite up to the level of digital scholarship. The question I would have about that particular project is what can we learn from the mashup you did that we couldn’t learn otherwise (or without a tremendous amount of grunt work with pencil and paper)? It’s those conclusions that to me would be scholarship.

    Jeremy— My own view is that this blog is scholarly (or at least I try to be most of the time), but only small bits and pieces of it would constitute scholarship in the way that I have defined it. For instance, I have written a number of posts that would constitute the scholarship of teaching and learning in history–drawn from classroom research. Many of these could have appeared in longer form as articles in various SoTL journals. I simply chose to publish them here rather than in the conventional article format. Then they were subjected to peer review after publication, so I would consider some of those posts as scholarship (see my next post on peer review). But overall, I would be more inclined to place this blog in the category of “service to the profession”. It is one of the ways I continue to be an “active scholar in my field” (one of our expectations here) and is quite different from the normal definition of that idea (giving papers, attending conferences). On the other hand, one could argue that blogs like mine that eschew the random posts on things like how well my kids ate their vegetables or my views on the upcoming election, rise at least to the level of a book review. Here, though, I’m not entirely sure.

    Marjorie–You are absolutely right that my argument here is complicated by the specific language used by departments when it comes to tenure and promotion standards. In other national contexts (the UK for instance) that language is often dictated from central authorities rather than individual departments which can be even more problematic. One of the reasons I began this series of posts was in hopes of generating more discussion among our colleagues as to what that language ought to look like. Your comment is a useful first step in that direction.

  9. I’m skeptical that the format and technology involved in “digital scholarship” means we should toss out older (“analog”) distinctions that are heuristic in nature. I’m not entirely immune to an argument that older distinctions derive from the standard sociology-of-knowledge structures, but people have been playing around with the categories for a bit longer than the internet has existed. I guess I’d prefer a somewhat more concrete discussion. A few thoughts:

    1) If you haven’t yet read Ernest Boyer’s Scholarship Reconsidered (1990) for one look at different types of scholarship, you need to. He broadens the notion of scholarship, but he still derives the categories he uses from a heuristic argument.

    2) One way of thinking about non-monographic projects (such as Tim describes) is to think about the way public history programs work and what “counts” in that context. Curated projects online and in a museum space are both public history. And I’m sure anyone who works in a museum will happily tell you all about the behind-the-scenes work in both environment. (Lighting; space design; graphics design; patron idiosyncrasies; server management; budgets; etc.) Again, I think that bin (public history) already exists. For similar reasons, we already have the heuristic bin of “public intellectual” to handle blogs. Russell Jacoby’s call to academics is still relevant…

    3) It may be more useful to talk about “intellectual production” patterns than “digital v. analog.” I’m not going to go all David Noble on the world, but there are already substantial divides in academe on who works under what conditions. The existence of online scholarship and massive computational resources complicates that picture but is coming on top of substantial changes in the past half-century. I think I’d prefer to keep the discussion concrete: in what sorts of ways is historical scholarship changing, and for whom? In some ways, there is a huge set of resources for GIS and social-science history that the pioneers of cliometrics would have drooled to have 40 years ago. (I could probably now replicate the quantitative work for my dissertation–stuff that kept me up until 2-3 am for weeks submitting batch program files to the Penn social-science mainframe–in a matter of a few days.) There’s also the way that people are rethinking the public-private divides in scholarship. Other disciplines are ahead of history in this (think about ArXiv in physics and eprints versioning debates), but a significant minority of historians are headed in this direction. Keep in mind Michael Bérubé’s discussion of the spectrum running between “raw” and “cooked” blogs (and entries) as an indication that the public nature of online scholarship has changed the nature of academic work habits.

Comments are closed.