Classically Inclined

May 10, 2013

Unexpected dips: this year’s module evaluation forms

This week I’ve finished writing up my responses to this year’s load of module evaluation forms – I wrote a little bit about them last year, although under different circumstances. This year, we have a new shiny system – although the forms are still completed manually, they are processed by computer, which means that all the clever number-crunching stuff is now delivered to one’s inbox in a shiny PDF. Along with a duplicate e-mail containing the same shiny PDF along with data in three other PDFs which do not appear to be particularly distinct from one other, but never mind, it’s the main one that’s interesting. Particularly clever is the fact that the scanning machine can capture written responses, so as well as the prettified data the PDFs also contain scans of what students actually wrote – meaning the time I put aside to carefully type them all up was wasted, but that’s a small price to pay for progress.

When I last wrote about these module evaluations, I expressed quite a bit of frustration about the conflicting feedback, and the problems with actually identifying anything concrete to do about the sort of comments that completely contradict each other. For that reason, I’m usually a big advocate of using things like the CIQs and one minute papers to engage with students on a micro-level rather than wait for the final assessment when it’s too late to solve problems that have affected students throughout the course. But this time around, a couple of things stood out, and I do have a few things that I want to do differently next time.

The first thing comes back to my big bugbear about getting students to read secondary literature critically, and giving them an education in doing that rather than assuming that they’ll know it’s what we expect them to do. This year, in my Roman Life Course module I tried using a blog-based discussion format for secondary literature, which worked well in last year’s epic seminar but rather less well this time around. It still did what I wanted it to – got students to discuss secondary literature and realise that they could question it – but the lecture format and the amount I was trying to cover meant that the discussion around the article was less effective than it had been in the seminar format. So I think that next time I run a lecture course, I’m going to try the Purposeful Reading Assignment as described by Faculty Focus – it’s a much more structured individual exercise, which all students can do with the reading, and which should get all of them using their critical thinking skills. I think I’ll stick with the blog-based approach for smaller groups (so seminars of up to a dozen), but for larger groups, you really do need a different tool.

The other big thing I noticed came from those new-style PDFs we’ve suddenly been issued with. The results split the questions asked into three groups – institution-wide questions, student engagement questions and lecturer-focused questions. Most of the scores on the first and third section were quite high, with some slightly lower results over effectiveness of teaching techniques but nothing that signalled a need for anything beyond the annual overhaul of the engine, as it were. However, what shocked me was the comparison between answers in those sections and the student-focused questions – made rather more obvious by the snazzy visual effects in the new reports – where the answers were universally lower.

The three statements students are asked to rate in that section are ‘I did all the required work (e.g. reading, other preparation) for all teaching activities’; ‘I contributed constructively to class discussion or other activities’, and ‘I undertook wider independent study relating to this module’. Obviously these answers are self-reported so there’s going to be some inaccuracy here, both in terms of people underestimating how much work they’ve done and in people overreporting in order not to feel they’re letting the side down, but this area still jumps out as the category most in need of attention and improvement. And this is where it gets tricky – if students are self-reporting low levels of preparation and completing the required work for lectures, how on earth are lecturers supposed to deal with that, let alone encouraging students to do work beyond the set reading? What possible techniques are there for dealing with students opting out of preparation, in-class activity or independent study?

I’m don’t have an answer – I need to think about this one a bit. But this new way of presenting the evaluation data has flagged up a problem that I didn’t know existed, and that I think needs some serious reflection.

4 Comments »

  1. Oh, those are interesting questions. I don’t remember the OU student feedback forms having anything like those.

    Hum. Dealing with lack of preparation for lectures is particularly tricky, because unlike a workshop or seminar, if you stick strictly to the format, you can’t just start with a task that requires them to use what they’ve prepared. Does it absolutely have to be a lecture format? Could you start with a task before moving on to the lecture? It works best if it requires one or a few of them, chosen by you at random, to speak to the rest of the group. They need to know they can’t just rely on others who have done the work, basically.

    As for the in-class activity, your own sense of whether they’re participating might give you a better clue as to whether they really aren’t engaging, or are just shying away from recognising their participation as a “constructive contribution”. If it’s the latter, then using the specific words “constructive” and “contribution” in your on-the-spot feedback to them may help their confidence.

    For independent study, all you can really do is plant some seeds – if you do too much, it becomes a requirement and isn’t independent any more🙂 You probably already suggest supplementary reading; other possible seeding strategies might include highlighting topics or bits of the supplementary reading list that are relevant to later modules, or getting them to write one-minute papers or similar on further questions they’d like to explore, or regularly inviting them to share what they’ve read elsewhere that might be relevant to the themes of the course.

    I hope you’ll update us on what works and what doesn’t. Learner engagement is a big issue in my current job, even though it isn’t in academia, so I’ll be interested to hear how your strategies develop.

    Comment by Liz W — May 10, 2013 @ 9:13 am | Reply

    • Yes, the learner engagement ones are interesting – in one way they don’t have much to do with us as lecturers, but in another they say a lot about the expectations students feel are there for them in terms of how they behave in their degree.

      The small option lecture I managed to do some prep and engagement and small group work with, but it’s still apparently not proving as much a nudge for preparation as it should do. Plus if you have a lecture group of 100+, which is always a possibility in the big first year courses, that sort of thing is a bit more difficult to run without making it look like you’re picking on people (which I desperately don’t want to!).

      I’m going to think about this more and try and do some reading around the subject for strategies – it’s not something I’ve actively thought about before, but it’s clearly the bit of my feedback at the moment that needs the most attention.

      Comment by lizgloyn — May 11, 2013 @ 9:44 am | Reply

  2. A problem I’ve identified in my feedback forms is student expectation. My students seem to expect all the reading on the module bibliography to be available online, so they can access it when and where they like. So either we only include books/articles that are available online, or we scan everything we want them to consider reading. They seem to be very resistant to the idea of going to the library and looking for things themselves, and the idea that you might want to include things in your essay that aren’t on the essay suggested bibliography. Consequently, they’re dissatisfied about the provision we offer. What to do with that?

    Comment by Jane — May 10, 2013 @ 9:56 am | Reply

    • I found that problem last year, but less so this year – at least partially because a lot of the second years I was teaching this year had had me last year. I think part of it is very much about getting in early in the first year and saying ‘university education involves not being spoonfed’ – once the expectations have been set, it becomes a matter of resistance in the later years. In the first year, you can play the ‘this isn’t A-levels and you need to up your game’ card a lot more effectively, not least because that year is all about adjusting. But I agree – if you don’t adjust that expectation early enough, and explain that this is part of university life, then students will miss part of the fundamental point of how university operates.

      Comment by lizgloyn — May 11, 2013 @ 9:47 am | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.