Posts Tagged ‘assessment’

Playing With History

Thursday, April 29th, 2010

[9:30] Today and tomorrow I’m at the conference Playing With Technology in History at Niagara-on-the-Lake, Ontario. Day one is an unconference focused on the edges of the envelope in humanities computing. The sessions during the day include things like wearable computers, serious games, MakerBots and CraftRobo, barely games, walkabout applications for phones, along with good old fashioned issues like metrics for assessing student learning.

[11:00] I spent the first morning session in a session on making (see post on conference website). I’m particularly interested in this approach to history both because of what I’m writing in my book on teaching history in the digital age and because my teaching is more and more emphasizing turning my students loose on the past to create history in ways we haven’t thought through.

[12:00] In a session on the Great Unsolved Mysteries of Canadian History project. This is a project I’ve been using for years in my introductory history courses — the Who Killed William Robinson case — as a way of introducing my students to historical research in an engaging and rigorous way. Even though the Robinson case has nothing to do with Western Civ (the course I use it in), it introduces students to the difficulties of historical research, particularly working with documents that just aren’t very clear as to what they mean or don’t mean. We did an exercise where we did what I ask my students to do and then discussed what that meant to us as educators–what we learned from trying to learn like our students and how our expert knowledge about history, as opposed to these particular moments in history, helped us with the exercise. For me it was a lot of fun to spend some time working through a digital resource I have been using for so many years.

[12:20] Why I don’t tweet…The previous paragraph is 958 characters.

[1:30] Went on walkabout around Niagara-on-the-Lake with an iPhone researching a mystery from the war of 1812. This application (still in beta), created by our conference host Kevin Kee, is just the sort of thing Tom Scheinfeldt, Josh Greenburg, and I envisioned something like four years ago in the days before the new generation of smart phones. Ours was going to be “Stop Booth” and would give you a chance to traverse the historical/geographical space of D.C. in an quest to save President Lincoln from his assassin, but a combination of technological limitations and a lack of funding kept us from ever pursuing this idea. It was really exciting to see Kevin’s history quest through town on an iPhone and to imagine all the ways we’re going to be able to take advantage of this platform as humanists.

[3:00] Sat with Bill Turkel to see how RFID tags could be used in humanities applications. He demonstrated a simple (for him) program that would allow an RFID reader to gather data from a tag, then link it to a database of historical information. One idea I had from that demonstration would be to create a “magic wand” that had a reader in the tip that would allow students to wave the wand over an artifact or a bank of photographs to gather information about the thing being examined. If the readers had a greater range, something similar could be done with historic sites–students could wander through the site and as they passed tags, historical content could pop up on their phones. What makes this different from just having a GPS application is that they would have to actually pass close to the object with the RFID reader to get credit for completing some sort of quest in the site.

The big question for all of us at this conference is how all the “play” we are talking about can be connected to the serious purposes of teaching and learning. I’m a believer that there are direct connections, but I also am hard headed enough to insist that those connections be made explicit through data (qualitative or quantitative) that demonstrate how certain kinds of learning takes place during or as a result of play.

Tags: , , , ,
Posted in Posts | No Comments »

Why Assessment Gets a Bad Name

Thursday, December 17th, 2009

Regular readers of this blog will know that I am actually quite supportive of the whole idea of assessment in higher education. I am convinced that we need authentic forms of longitudinal assessment of learning in all of our programs, especially undergraduate programs, that provide us some sort of reasonable picture of whether our students are learning what we want them to learn and whether they are getting better or worse at it. In this way we can have some sense for whether we are doing the right thing for our students.

Without such assessments we are forced to fall back on either (a) the nods and smiles of our students that are supposed to tell us that they “got it” today in class, (b) their performance on tests and essays that we give them that may or may not be tied to departmental learning objectives, or (c) end of semester student evaluations that, of course, are no measure of learning.

However, the experience we are having right now in my department is a perfect example of why faculty members want to run screaming from anyone who utters the dreaded word “assessment.” You may find this difficult to believe (or maybe you won’t), but we are currently having to undergo five separate assessments of learning in our undergraduate program. How can it be that one department could have to engage in five separate assessments simultaneously? I’ll try to explain…

  1. We have our own assessment (one I helped design) that goes like this: All History majors must take History 300 (Historical Methods) and History 499 (Senior Research Seminar). Each semester we select a random sample of final papers from History 300, put them in a file, and the when those students complete History 499, we pull their final research papers. Then we convene as a group and score each pair of papers on a rubric of historical thinking skills to see if (a) our students are learning what we hope they will learn and (b) if, as a group, they are making progress over time. This is an on-going assessment of learning in our major and one we subject ourselves to.
  2. Several years ago our Provost created an Academic Program Review process that, for us, began in 2008, and will continue every other year, apparently forever. This particular process uses a software platform called the “Weave”. Please don’t ask me to tell you how it works. My colleagues and I figured it out in 2008, but it has already been updated several times and apparently now works entirely differently. I won’t tell you the adjectives used to describe the Weave by a colleague in Cultural Studies (this is, after all, a family blog).
  3. We are now in the first full phases of our decennial reaccreditation by the Southern Association of Colleges and Schools (SACS). For now the SACS process is all about making sure we collect credentials and syllabi from our faculty and about creating a Quality Enhancement Plan (QEP). I am part of the QEP steering committee for the University and the end result will be quite good. Getting to that result is going to be painful at times, but the benefits for our students will make it worthwhile.
  4. We are undergoing an assessment of the general education curriculum. I haven’t been able to determine the mandate for this particular assessment, but for now I’ll refer you to my previous post for some insights into my thinking about general education. The short version is that I think distribution requirements are a good thing. Mandating particular courses (and thereby stifling student choice) is a bad thing.
  5. The State Council for Higher Education for Virginia has mandated an assessment of how all colleges and universities in the state are helping our students become better writers. For this particular assessment a group of seven or eight faculty were asked to come together to use the rubric we use in our assessment of historical thinking skills (see #1 above) to assess student writing. I had to leave that meeting well before it was over, which is probably why I’m still unclear how a rubric designed to measure historical thinking can be used to measure writing. Moreover, I’m unclear how having each academic department in the University measure writing with their own rubrics will yield data that can be aggregated in some sort of meaningful way. But maybe that’s just me…

Okay, got all that? What we have here is one assessment generated by the department, two assessments coming from the Provost’s office, and two from outside agencies with some level of supervisor authority over us.

As much as I seem to be complaining (because I am) that we have so many assessments going on at once, I want to reiterate that I am sympathetic with the need for each one of these. I can’t argue with the need to know all the things that these different assessments are after.

But–and I think this is a very important but–the last time I checked, faculty members were first and foremost supposed to teach their students and second were supposed to produce high quality research. Of course, we also engage in lots of departmental, college, and university service (not to mention community service). Even with those mandates, we must make time for at least some assessment, but five assessments? All at once?

Tags: , ,
Posted in Posts | 2 Comments »

Quantifying the Humanities

Wednesday, January 7th, 2009

The rising importance of metrics for evaluation in higher education has more than a few of my friends and colleagues on edge. What will it mean, for instance, when colleges and universities see the same sorts of assessment data generated for the humanities that already exist in K-12 education? Will we see graduation exams in History or English? How does one quantify the many years spent researching and writing a book of history? How will these data be used?

While I think college faculty are right to ask probing questions about the quantification of their efforts in the classroom and in their research, I think it’s wrong-headed to assume that any and all attempts to quantify educational or scholarly endeavors are somehow an evil conspiracy to undermine our academic freedom and integrity.

For instance, read Jennifer Howard’s very interesting article in the Chronicle of Higher Education from October 10, 2008 (“New Ratings of Humanities Journals Do More Than Rank — They Rankle“). For those of you without online access to the Chronicle, the story begins:

A large-scale, multinational attempt in Europe to rank humanities journals has set off a revolt. In a protest letter, some journal editors have called it “a dangerous and misguided exercise.” The project has also started a drumbeat of alarm in this country, as U.S.-based scholars begin to grasp the implications for their own work and the journals they edit.

I would submit that one implication is that academic c.v.s will be much easier to make sense of. This past year I was on a committee in our Center for Teaching Excellence charged with helping nominees for a state award navigate the process. My two charges were in the Psychology Department and although I know nothing about the relative merits of various Psychology journals, I could quickly see which of their articles was in the more difficult-to-publish-in journals. Why? Because academic journals in Psychology publish data on the acceptance rates of articles. It was therefore obvious to me at a glance that an article published in a journal with an 11% acceptance rate was probably more notable than one in a journal with a 78% acceptance rate.

Only in the humanities have we been so resistant to any sort of quantification of results. Almost every other major disciplinary category — sciences, engineering, health sciences, social sciences — rates and ranks almost everything they do. And in many of these disciplines college graduates are already subject to de facto graduation examinations administered by various licensing boards. So what makes the humanities so special?

Because I don’t think we are special enough to get a pass on quantification of effort, I was pleased to receive the announcement today that the Humanities Resource Center Online has gone live. A project of the American Academy of Arts and Sciences with some collaboration from organizations such as the National Endowment for the Humanities and the American Council of Learned Societies (among others), the HRC offers one-stop shopping for data on the humanities in the United States, much of it set in a global framework.

Want to know how much money was being invested in the humanities in a given year? Want to know about the academic preparation of high school history teachers? Want to know more about the participation of underrepresented groups in graduate programs in English? It’s all there. I applaud the work that has gone into this website and hope that as the years go by more and more data will be deposited there.

Why? Because I’m a historian and I believe in the value of evidence in arguments. The data on this site will make it possible to have much more informed conversations about what is happening (and just as importantly, what is not happening) in the humanities. So, for instance, when we complain that scholars in the humanities are underpaid relative to our peers in other disciplines, now we have the data to prove it. Or when we wonder why our majors seem to be less ethnically diverse than the rest of our student body, we can see how our local findings compare to national data sets.

All in all, I think the current iteration of this project is a great start and I look forward to its further elaboration in the years to come.

Tags: , ,
Posted in Posts | No Comments »

Tests we don’t need, but probably ought to prepare for

Friday, March 10th, 2006

The once and former president of Harvard, Derek Bok, published an interesting op ed piece in the Washington Post last week. Bok takes notice of the growing demands for assessment tests in higher education and neatly summarizes the reasons why they are a bad idea–at least in some disciplines.

I’m all for assessment in disciplines where there is an agreed upon body of knowledge needed for basic professional mastery. For example, I want my nurse to know that I have two kidneys and one liver or the engineer who designed the airplane I’m flying in how much weight that newly designed wing can stand.

But what should every English major know about poetry? What should every history major know about the past? What should every art major know about art?

It’s easy to dismiss the idea of such testing. But that would be much like the fabled ostrich with its head in the sand. Testing is so politically expedient that if we try to pretend it’s not going to happen, it will likely sneak from behind and bite us.

So, what should happen in the humanities? I’m with Bok on this one. Scholars need to–must, in fact–develop assessment tools that can demonstrate that our students have really learned something worth knowing. We have to come to clearer agreements on what that learning would look like and then show how, without multiple choice tests, we have assessed that learning over a trajectory of years. And, when we fail to meet our own benchmarks, we have to show what interventions we’ve implimented to address some lack of success among our students.

I think digital media offers great promise for helping with this assessment process. In future posts I’ll outline some of the ways I think that can happen. For now, I’ll go try to pull a few colleagues’ head out of the sand.

Tags: , , ,
Posted in Posts | Comments Off