Toward the end of the last semester, I found a much-needed brief distraction from the seasonal delirium in an instructive media moment. Arizona Senator John McCain, or his staff, tried to crowd-source help in closing a small gap to reach a goal of 3 million followers. The strategy immediately backfired:
What lesson does the Arizona Senator’s “tale of fail” offer those of us thinking about how to measure and understand our success in effectively engaging students and employers through our efforts?
Here are three reasons this example inspired me to engage my team in thinking more deeply about how we measure engagement of our key stakeholders.
It’s a good reminder that growing an audience creates greater responsibility to deliver to their expectations.
The days of justifying our existence simply based upon the number of students or employers who walk through our doors have long passed on most of our campuses. At the same time, we’ve renewed focus on scaling our efforts to develop students’ career readiness skills, improve their career development strategies and increase employer connections. In this context, it’s tempting to focus on a “personal best” participation goal when working toward scaling learning experiences and events, and to pull out all the stops to ensure that milestone is met. However, if the call to action within the experience we offer misses the mark for the audience we’ve attracted, we’ll also scale an undesirable loss of support and momentum. We need to know our audience and target our messages and offerings appropriately to meet their expectations as the path to achieving our own. Empathizing intentionally with our audience(s) and measuring our reputation in advance of making a call to action gives us an important signal as we consider ways to increase our reach.
We need to balance macro-intelligence with a more granular understanding of how we achieve our impact.
Career services’ champions have rightly resisted assumptions that our students’ reported immediate career destinations survey outcomes are the responsibility solely of career services offices, or that those data can stand alone to adequately measure our impact. We’ve been encouraged by leaders in our field to think strategically about creating knowledge that helps us tell the story of our value proposition effectively in an environment where scholarly analysis carries weight. In turn, we have invested in assessment and research personnel within our own teams, or have contributed as key thought partners to the vision and design of institutional research, to better understand the connections between desired outcomes, student characteristics and behavior and our activities. These more sophisticated studies can identify patterns that predict the outcomes they (and we) want to achieve, and they can inform our overarching strategy. All good things.
However, when it comes to applying these measures to the design of learning experiences and communications that incentivize desired actions among our current audience, the cycles are often too long and signals too noisy and are sometimes too obvious or too macro (along the lines of “We increase our followers when someone who sees a shared tweet follows us”) to inform an effective approach to a specific engagement opportunity.
To compliment these analyses, it will help to be very clear about which next desired action on the road to a positive outcome a specific experience is designed to achieve, and to track its immediate effectiveness in promoting that particular behavior or performance improvement. When and if we are unsatisfied with the way a particular engagement opportunity achieves that outcome, we can then be very intentional of measuring the effect that a specific tweak a strategy has on that particular impact. Our team has recognized that Handshake offers us an important opportunity to identify whether a stakeholder takes a desired next step (whether that’s a student updating their skills on the profile or adding a new resume, applying for an internship, an employer posting a job after a conversation with a team member etc) without us having to interrupt their experience with yet another survey.
Achieving our goals requires the ability to respond with immediacy to signals that we may be missing the mark.
In less than 24 hours, Senator McCain lost nearly 20,000 followers—an average of over 800 per hour. I have not explored the analytics, but it’s reasonable to imagine that there were some early signs that the message was not achieving its goal and that the staff missed a short-term opportunity to correct course. Likewise, reducing the number of career destinations survey respondents who missed the career development memo while they were still part of our captive audience requires us to keep an eye on some more immediate indicators of our expected cumulative impact.
Building review of metrics and signals into a regular process of debriefing and goal setting—and building a culture of safe vulnerability within our teams in order to increase comfort with acknowledging less desirable signals—is important to our ability to cope with inevitable failures will teach us how to improve.
Abra McAndrew serves as Assistant Vice Provost, Student Engagement, and leads a number of SAEM/AISS's key engagement areas, including Career Services, Leadership Programs, the Office of Student Engagement, Innovation Programs, and Undergraduate Research. She provides vision and leadership in developing a nationally-recognized, sustainable strategy for growth of engaged learning on the campus and works to expand the division's capacity to deliver high quality, scaled engaged learning opportunities to students. Her educational background includes a Bachelor’s degree from Smith College, a Master’s in English and Linguistics from the University of Arizona’s College of Humanities, and an MBA from the Eller College of Management. You can connect with her on LinkedIn.