October 27, 2005
Open Classrooms
One of the central tenets of faculty practice is that we generally keep what happens in our classrooms to ourselves. Beyond the occaisonal visit by a colleague and the anecdotes we love to share about our students, only rarely do we open our classrooms up to public inspection. Given our mania for peer reviewed publications, you'd think we would just as anxious to have peer reviewed teaching. It seems to me that the same benefits would accrue. Our colleagues could point out the strengths and weaknesses of what we are doing, could examine the underlying assumptions of our teaching in a particular course, and could comment on the wider significance of the results we are achieving with our students. Of course, for something like this to work, we'll need the same kind of attention to detail and professionalism that our colleagues devote to reviewing one of our articles or manuscripts. And for that to happen the same sorts of informal rubrics of evaluation that we use when acting as peer reviewers for a journal or a press will need to evolve.
Some historians have caught the open classroom/peer review of teaching bug already. See, for examples, the World History and Age of Reform course portfolios posted by Amy Burnett of the University of Nebraska on the Peer Review of Teaching project website. Or Christopher E. Mauriello's Transnational History of the 1960s at Salem State University. The American Historical Association posted one by me that I produced in 1999 (hence the frames...frames were cool in 1999) and another by Bill Cutler at Temple University several years ago, but to date there is no central clearing house for examples of course portfolios in history. Would that there were...
Following the publication of my portfolio on the AHA website (which is currently down because their webserver is in South Florida and got whacked by Hurricane Wilma), I wrote a piece for Perspectives on transparency in teaching. The responses I received to this article were fairly evenly divided between those who congratulated me on opening up my classroom and those who chastised me for doing so. The chastisers' complaints largely focused on what they saw as a bad precedent being set by my willingness to ask for substantive and open peer review of my teaching. These complaints are, it seems to me, an indicator of how far we have to go before we can really expect the kind of peer review of teaching that we demand of our scholarship of discovery.
Posted by mills at 11:39 AM
October 25, 2005
Visualizing Information
Of late my graduate students have been wrestling with what constitutes "digital scholarship" in history. In our discussion we talked a good bit about how information that is not quantitative can be displayed online in ways that take advantage of the capabilities of digital media. Among the examples I showed them are several that have some real potential for the representation of research in the scholarship of teaching and learning or even for analysis of information gathered in such research.
The first comes from the Moodographer website. If you don't know Moodographer, their site description explains:
Moodgrapher plots the mood levels reported by LiveJournal users in their posts during the last days, updated every 10 minutes. Two numbers are reported by Moodgrapher: the percentage of posts reporting a certain mood (the dashed, black line below), and the "rate of change" of a mood � the difference between the usual amount of posts with this mood and the amount in a given hour (this is the continuous red line below).
So, an example of a Moodographer graph with historical utility would be:
This graph shows the frequency of "worried" in the LiveJournal.com blogsphere on the days when Katrian roared ashore on the Gulf Coast. It's possible to imagine using a tool like this to analyze the content of student postings to a class blog, where the researcher would use analytical concepts taught in class or some other such category rather than "worried." Such an analysis, conducted over the entire semester, could indicate the degree to which such key ideas were being deployed by students in their writing as the semester progressed.
The second example comes from theyrule.net. This is a Flash application that allows you to visualize the relationship between members of the boards of director of the largest companies in the United States. So, for instance, if you wanted to see how the boards of Northrup Grumman and General Dynamics (two defense contracting behemoths) were connected, the visual example would look like this:
You can see that three men link these two boards together. But what about their links to other corporate boards? That graphic looks like this:
Now the interlocking nature of corporate leadership becomes more apparent. It's possible to imagine using a tool like this to represent individual students in a course (rather than fat cat board members) and to indicate their connection to certain key concepts in the course over time.
The final example has to do with the use of words in the English language. Wordcount.org tracks the usage of words in the English language--written and spoken. Their About page says:
WordCount� is an artistic experiment in the way we use language. It presents the 86,800 most frequently used English words, ranked in order of commonness. Each word is scaled to reflect its frequency relative to the words that precede and follow it, giving a visual barometer of relevance. The larger the word, the more we use it. The smaller the word, the more uncommon it is.
WordCount data currently comes from the British National Corpus�, a 100 million word collection of samples of written and spoken language from a wide range of sources, designed to represent an accurate cross-section of current English usage.
So, my own test was to enter the word Stalin, a word that came up often in my East European history class this afternoon. The result looks like this:
Uncle Joe's name turns out to be in 9,516th place in terms of usage. I found it somewhat interesting to note that just ahead of "Stalin" in terms of usage was "sexy". Go figure. And, because inquiring minds want to know, I decided to see if this site has any sort of feature that tracks queries. And, of course, they do. Here is the result of their query tracking. No surprises here! If you are offended by four letter words, stop looking now and hit the back button on your browser.
All humor aside, I think you can see how an historian might use a system like this. Imagine feeding in the 30,000+ personal narratives collected in the September 11 Digital Archive and subjecting them to this kind of analysis. Or, for the purposes of SoTL research, to track student use of words in their papers over the course of a semester.
Representing the results of SoTL research in such a visual way could lead to some very interesting new insights into what was happening in one's classes.
Posted by mills at 02:43 PM | Comments (1)
October 20, 2005
Of International Societies
I just came back from the annual meeting of the International Society for the Scholarship of Teaching and Learning in Vancouver, BC. In addition to deciding that I want to move to Vancouver as quickly as possible, I came away from the meeting with some new things to think about.
Sometime toward the end of the meeting it dawned on me that the presenters--the vast majority of whom were from the US and Canada, with a nice sized contingent from the UK and Australia--fell into three distinct groups:
1. The Americans tended to be focused on practical applications of SoTL research and were very worried about how they might convince their colleagues that the SoTL "counts" toward tenure and promotion in much the same way that more conventional scholarship might. They also generally expressed strong concerns about the motivation of their faculty colleagues to even consider the SoTL.
2. The Brits and the Canadians sounded much more like American educational researchers--lots of tables, correlations, that sort of thing. They seemed to focus on process and research outcomes much more than on practical applications and were hardly concerned at all that their work might count or that their colleagues would be willing to buy in.
3. The Australians were well ahead of everyone else in both practical applications of their SoTL work and in their integration of the SoTL into the lives of their institutions in all ways. On the downside, they were facing substantial interference from central authorities in their day to day work and some serious cuts in funding.
When I go to Sydney in two years for the next non-US iteration of this conference, I plan to try to figure out how our Australian colleagues got so far ahead of us. One good example of how we have so much to learn from them is the World History curriculum in the Department of Modern History at MacQuarie University in Sydney. In the US World History is a field with only one course--the freshman survey. For years, those in the field have dithered about developing a series of upper division World History courses, but in this department they have already brought several upper division World History courses into their curriculum and these courses are organized largely around the results of the most recent research in the SoTL in history.
The rest of us have some catching up to do...
Posted by mills at 02:54 PM
October 19, 2005
SoTL in Action
What would it look like if the results of research in the scholarship of teaching and learning found their way into websites created for students and teachers?
For the past several years we have been doing just that at CHNM. Relying on the research of Sam Wineburg, Peter Seixas and others, we've created a series of online learning opportunities for students and teachers to see what happens when expert learners make visible their methods of inquiry and analysis. So, for instance, Dana Liebsohn, and Art Historian at Smith College, talks visitors to our World History Sources website through her analysis of the Codex Mendoza. Or, in our History Matters site, Larry Levine discusses his analysis of blues music.
As useful as these are, they are still not where we ought to be. Because we have to take into account the fact that many possible users of our sites still have limited bandwidth, we've set these up in Flash. Because we don't have sophisticated videotaping capabilities, we provide an audio track only. I think it won't be until we get the real live historian on video talking to students more directly that we'll be able to realize the full potential of such an exercise.
The other thing we desperately need are specific examples of scholars and students engaged in "think alouds", talking their way through their analysis of sources. It is through these kinds of exercises that novice learners can really see just how different their analysis of a source is compared to that of an expert learner. Again, though, I'm of the opinion that this isn't really going to be effective unless it is done on video so that the viewer can see the person who is talking struggling to make sense of the evidence. Maybe now with the new iPod video capability (and the sure to follow imitators), we'll have a technology that makes it possible for us to deliver such video easily to our students, rather than making them watch a stream cast on their computers.
Posted by mills at 06:43 AM
October 18, 2005
Students Using Websites
One of the most common laments I hear from high school teachers and college and university faculty as I travel around the country is how uncritically our students use historical resources they find on the Internet. We all know examples of students going to their favorite search engine, typing in some likely search terms, and then clicking on the first few likely websites that appear in the list on their screen. From those few sites they pick one that looks reasonable and use it to write their paper, only to find that the website they were using had significant problems, whether it be that the site provides bogus or misleading information, it has an obscured agenda, or is simply a site put up by Ms. Johnson's third grade class.
Several years ago I set about trying to put a stop to this problem. I know it's not something that can ever be truly "fixed", but at least one can try. The answer I came up with is the Webography Project, a database teaching tool that provides students with some basic advice about how to assess the content of the websites they visit and gives them the chance to enter reviews of websites into the project's database. To date, 54 teachers (high school and college) have signed up to use the project and their students have already entered more than 735 reviews of websites into the database.
I've been gratified to see that the project has started to get some notice in other venues, including Academic Commons and Next/Text.
As nice as it is to have one's work noticed, more interesting is the potential of this database for some interesting research that falls under the heading of the scholarship of teaching and learning. Anyone interested in how students make sense of what they find on historical websites need only search through the 500 plus public entries in this database and read what the students have written and the scores they've assigned to the sites. So, for instance, a key word search using the term "sourcebook" returns 35 different reviews of the various versions of Paul Halsall's sourcebook project. Reading these reviews, one finds how differently students understand what they find on the same website.
I'd like to see more such resources developed--projects that make it possible for students to engage in serious intellectual work while also making the results of that work available for researchers to study.
Posted by mills at 10:33 AM