Measuring value of social media and learning: Burying the myth of the magic formula

Harvard Chalkboard
“Harvard Chalkboard” photo by Patrick Haney via Flickr

Jane Bozarth wrote an excellent piece recently for Learning Solutions magazine called Assessing the Value of Online Interactions. In it, she explains how she applies a conceptual value-measurement framework developed by Etienne Wenger, Beverly Traynor and Maarten de Laat (see Assessing Value Creation for Communities of Practice and Networks: A Conceptual Framework).

[A side note: I obviously respect the work of Wenger and Traynor (see references in previous posts here, here and here) having used Wenger’s work in teaching my class on knowledge sharing and collaboration. It has certainly contributed to the development of my point of view on learning and sharing knowledge: that is, it’s largely about social interactions and practice (practice in the sense of a professional or academic domain).]

What I like about Bozarth’s recent piece is that she starts by arguing against those who engage in a knee-jerk search for “the magic metric.” She explains that, as she conducts various workshops on using social media to enable social learning within organizations, someone invariably comes to the session looking to challenge her to prove that engaging in social collaboration technology at work isn’t just simply a waste of time. She writes:

They usually want some magic metric, some formula like, ‘two hours on LinkedIn + four comments in groups = tangible outcomes for the organization.’ It doesn’t work that way. A great deal depends on how the worker chooses to spend that time in social channels, how well he filters and curates information, how she chooses the people with whom she’s interacting. The quality of those interactions depends in turn on many other issues, including trust, a willingness to ask for and offer help, and time invested in developing ties deeper than those purely at the surface. Likewise, a worker expected to improve performance and support organizational goals must know what the expectations are around that. (Bozarth, 2012. Learning Solutions Magazine)

This certainly aligns with my experience – especially the “it depends” elements of social channel choices, filtering and curating, network connections, trust, willingness to be open and to invest time in developing deeper ties. See my post on half-baked thinking for more on that topic.

But the critical element of Bozarth’s piece, for me, is really a call-to-action to think more deeply and critically about how we see and understand how people gain value by adding digital interactions into their social and professional lives. We need to stop trying to see and understand in purely formulaic fashion (e.g., magic metrics and ROI). And we need to stop treating digital interactions as if they are entirely separate from our non-digital interactions.

The Wenger, Traynor, de Laat framework makes explicit one way of attempting to think more deeply about seeing and understanding how people gain value through interactions both non-digital and digital. It’s not a simple approach because it requires you to combine quantitative and qualitative approaches to connect-the-dots between short-term activity and some value that results over the longer term. And connecting-the-dots, in large part, is based on story telling and reflection. Bozarth tells a story in her piece to illustrate this; but the story is only clear in hindsight. In the short term we simply do not know how (or if) specific interactions may benefit us. We need to let the full story unfold.

A framework that relies on story-telling to help us understand value creation presents a number of implementation challenges. Collecting and analyzing stories is a non-trivial task, as Jenny Mackness articulates nicely in her post on the value-creation framework.

But it’s an effort that takes us on the right path. Right-er than ROI or mythical magic metrics.

7 thoughts on “Measuring value of social media and learning: Burying the myth of the magic formula

  1. Thanks for your nice comments, Jeff. I am hoping to help move some conversations from an emphasis on counting activities to one on looking for real meaning. The reality of “it depends” often throws a painful wrench into the comfort of convenient measures. -jb

  2. I adopted a similar approach in the old days (2011) to discussions in online classes. Only when no discussions were required did the discussion board activity explode with authentic, student driven exchanges. The value of the social interaction exchanges did not have to be sold by the prof. The voluntary nature of the interaction speaks to the students’ perspective of the value fo the online social exchanges. http://bit.ly/WM0PtR

    1. Thanks, Richard. Especially for the link to your work – it’s really important stuff. Would love to continue sharing on these topics.

      Our experience here is very similar – the authentic exchanges coming when the students feel a need for dialogue (to clarify, test thinking, ask questions etc.). And when you have that going on, then it seems to have a greater chance of more interesting upstream outcomes as well (e.g., the participants grasp concepts in ways that would not have been possible were it not for the online dialogue).

      Which is why I am continually interested in this Wenger framework. It seems to give us a story-line, connecting-the-dots framework for making sense of how we got to something of value – in hindsight. Fictional example: “We implemented a new protocol in the assessment part of our design process to better understand our client’s environment, and that led to a new insight that resulted in a successful solution. In hindsight, the seeds for this new protocol were first planted in a comment Richard made online. We explored that, developed a prototype approach, and that led us to where we are today.”

      I know there are dangers in this kind of sense-making. I think we also need to be good at reflection and using data to make sure our conclusions stand up to scrutiny. But it’s just a much more appealing approach that looking for a magic formula.

Comments are closed.