FROM ONE+
The Argument Against Accountability
For you in the meeting business, it’s precisely the same. The number of people who show up at your conference may be entirely less important than who they are, what they do and how much they share about it later. The number of full-paying participants might matter less than the unmeasured and perhaps immeasurable impact that a conference has on the sponsor’s public image. The amount that participants actually learn at a conference may not even be captured on the best surveys, or realized by participants until weeks or months later. That’s when they sign up for next year’s event and begin to tell others.
I found this interesting because I am currently working on a mid-year evaluation of my office and I’m being asked to gather all my assessment data, the current budget numbers and my “vision” to tackle the future of my area.
We have been scanning all attendees of our Late Nite programs and after a year and a half, we know how many students are coming and who they are. Here’s what we don’t know. We don’t know about satisfaction, we don’t know if we are filling needs, we don’t know if the attendance we are getting is high or low in comparison with other institutions with similar programs. I would love to have all the time in the world to do focus groups and have conversations with students regarding their experiences…but I serve as the Late Nite Program Coordinator due to cuts in the division. I have to spend all my time in the trenches and it’s very hard to stick my head out to assess what’s out there – but I know this – taking attendance never tells you much. Even with full demographics on these students – it doesn’t tell me why they come to Late Nite programs, what would make them come back and what they think about the program overall. This type of assessment (attendance #’s) has been the bane of Student Activities existence since the beginning of the field. Numbers don’t always equal success or failure which makes it so much harder to assess.
I am all for transparency and accountability in business dealings. But when that honesty is limited by the language of metrics, we end up using our tools to kill our jobs, our industries and our abilities to fake it until the real results of our activities become apparent.
How about when the “real results” never become apparent to the people who make those financial decisions? How about when no one understands your program and trying to “Tell Our Story” doesn’t work?
If I show that our program went from 200 events in F2009 (smaller events) to 82 larger events in F2010, I have to also be able to justify the overall drop in attendance. (Attendance at several little events adds up). I was able to find a way to show how this looks like a drop, but in other ways can be seen as a positive if you look at it in another way. If you separated out our strongest area – our coffeehouse programs – you can see that attendance is consistently higher with fewer nights programmed. Thank Goodness.
What types of assessment issues are you having on your campus? Do you feel like your assessment could get you stuck in a place you don’t want to be? Where will you expand your assessment and how have you taken your numbers and twisted them to show what you truly want to say?
Coming up with assessment numbers to try to justify programs to others ends up making me feel less justified in the work I’m doing…which feels very counter productive. On a small campus (~1,000 students) assessments take place via Survey Monkey most often and in our department come in the form of enrollment numbers…both are skewed snapshots of what’s actually going on and what we feel like we should be focusing on. Sometimes it can be a good starting point, but I often feel like we’re spinning in circles and not having the right conversations.
Also, from my experiences working with LN, while I had one perspective on getting somewhat accurate numbers for the events, it sometimes seemed difficult to get the student workers on board with keeping up with headcounts, tallies, etc. That, combined with previous GA’s who had just made up numbers made it hard to feel like we were really counting anything relevant.
Mallory – You are completely right. The attendance numbers of LN are never accurate and there’s always inconsistencies. Even now w/scanning. It’s a never ending problem. So basing our success or failure on those numbers – it’s not a great place to start.