Khoros Atlas Logo

Science of Social Blog

Lithium Alumni (Retired) Lithium Alumni (Retired)
Lithium Alumni (Retired)

Dr Michael WuMichael Wu, Ph.D. is 927iC9C1FD6224627807Lithium's Principal Scientist of Analytics, digging into the complex dynamics of social interaction and group behavior in online communities and social networks.

 

Michael was voted a 2010 Influential Leader by CRM Magazine for his work on predictive social analytics and its application to Social CRM.He's a regular blogger on the Lithosphere's Building Community blog and previously wrote in the Analytic Science blog. You can follow him on Twitter or Google+.

 


 

Hi there! This is the third article in this mini-series on Facebook Engagement, where I develop the deeper levels of engagement metrics that quantifies the full spectrum of engagement. I apologize that these posts tend to be a bit more data intensive, but these eight levels of metrics are necessary for the development of the Facebook Engagement Index, which I will reveal soon. If you missed the earlier posts in this series, I recommend reviewing them before proceeding.

 

  1. Quantifying Facebook Engagement: More than Just Counting Fans and Likes
  2. Deeper Facebook Engagement: Dissecting Interactivity

 

Last time I covered the intermediate level (i.e. Level 3 to Level 5) engagement metrics, which are all metrics that quantifies how fans interact on the fan page.

  • Level 3: Thread Depth – amount of interaction
  • Level 4: Unique fans per conversation – with how many other fans?
  • Level 5: Average/median response time – dynamics of interaction

This time we will investigate the next two levels of the engagement spectrum (i.e. Level 6 and Level 7). We will also re-examine the meaning of fans and what it means to be a Facebook fan vs. a real fan in the traditional sense.

 

Are Your Fans Truly Loyal Fans?

If you are a true fan of say, a certain celebrity or sports star, would you continue to participate in conversations and/or activities about that person? My guess is “yes,” because that is what it takes to be a fan. You need to continue to participate in all the activities that other fans do in order to be a part of that fandom. And true fans often take pride in their participation. If you slow down or stop participating, then technically you are no longer a fan anymore.

 

fig06_Engagement Data_web.gifSo far, we’ve been looking at the statistics of the active fans who are participating in conversations. As a scientist, I am rather curious to see how many of them are truly loyal fans and actually return to the fan page and remain active over time. To answer this question, I computed the active fans’ re-engagement probability with the fan page. That is, the probability that a fan becomes active (i.e. posts something) more than once on the same fan page (see Figure 6).

 

According to the data, most Facebook fans are not very loyal to the fan pages. Only about 30% of the active fans re-engage with the fan page more than once (i.e. through posting). 70% of the active fans will post only once and never re-engage the fan page again!

 

So, most Facebook fans are not real fans in the traditional sense, because they are not particularly loyal. Facebook skewed the meaning of fans by calling any user, regardless of how passionate he is, a fan. If that is the case, we will need another term for those users who are true fans of a brand. They do exist, albeit in much smaller quantity. And we will call them the superfans.

 

Do Fans Develop Relationships on Fan Pages?

In the first article of this mini-series, I demonstrated the structural similarity between a fan page and a community. In my mini-series on cyber anthropology, we also established that community is where relationships are established and developed. Since a fan page is structurally a community, I wondered if it also functions like a community (i.e. enables the development of relationships).

 

In order to build relationships, people must engage each other actively and participate reciprocally in conversations. To see if the fans are doing this, I computed the active fans’ return probability to the same conversation. That is, the probability that a fan posts something more than once within the same thread of conversation (see Figure 7). Note that this is a very lenient criterion for having a conversation. In reality, it would probably take many more posts within a thread to have a meaningful conversation. And it would take many such threads of conversation to contribute significantly to the development of real relationship. But I am giving the “so-called” fans here the benefit of the doubt and see what the numbers tell us.

 

fig07_Engagement Data_web.gifTo my surprise, most fans do not participate in reciprocal conversations. The probability of a fan returning to the same conversation is very low, only about 9.6%. That means 90.4% of the active fans post only once in a thread and never return to that conversation again. This fan behavior would definitely hinder the development of any meaningful relationship on fan pages.

 

Therefore, even though a Facebook fan page is structurally like a community, it doesn’t function like one, because the fan page environment is not conducive for the development of relationships. I will speculate here about why this is the case, since I don’t have any hard data to prove it. I suspect one of the major reasons is due to the stream interaction style of the wall post. Streams are great for news feeds, status updates, or any content that is broadcasted for mass consumption. Users typically interact with the stream by watching its content, and they can interact with any piece of content at the moment when it flows by.

 

However, this type of stream interaction is terrible for carrying on a long conversation. Because as you are having a conversation, that conversation is also being pushed down the stream by new content. Eventually, the conversation will be pushed off the wall as the natural flow of the stream. Ironically, the more people participate, the faster the stream flows. So the busier the fan page is, the faster any conversation flows off the wall, and the harder it is for you to hold a meaningful conversation. This is precisely why it is so hard to develop any relationships on fan pages, because the streaming wall posts made it very difficult to have any meaningful conversation on the fan page.

 

Conclusion

Engagement_Depth_04_resize.gifToday I presented data on the Level 6, and Level 7 engagement metrics. The data suggests that Facebook fans are not really fans in the traditional sense because they do not actively re-engage the fan page. Despite that, Facebook still calls them a fan. Moreover, the Level 7 data also confirmed our suspicion in the previous post that fans typically only post once within any conversation. This makes it very hard to carry on a meaningful conversation with other fans and therefore hinders the development of any real relationships on fan pages.

 

With levels 6 and 7, we are almost done characterizing the full spectrum of engagement.

  • Level 0: Total fan counts
  • Level 1: Active fans
  • Level 2: Interactivity through comments
  • Level 3: Thread Depth – amount of interaction
  • Level 4: Unique fans per conversation – who you interact with
  • Level 5: Average response time – dynamics of interaction
  • Level 6: Re-engagement probability with the fan page
  • Level 7: Return probability to the same conversation
  • Level 8: Dare to speculate what this level might be?

 

Next time, we will cover the last and the deepest level of engagement metric. If you like to speculate, I’ll give you a hint. The eighth level of engagement deserves is own post because it is a network-based metric. The data there is interesting, but given what we’ve already learn about how fan pages work, it is not a big surprise. So stay tuned for the deepest level of engagement!

 

In the mean time, I welcome kudos, comments, critiques, suggestions, tweets, retweets, debates, or discussion of any kind.  🙂  So, let’s get some discussion going before I see you again next time.

 

 

 

 

Read more
Labels (1)
  • Engagement

4 19 19.9K
Lithium Alumni (Retired) Lithium Alumni (Retired)
Lithium Alumni (Retired)

Dr Michael WuMichael Wu, Ph.D. is 927iC9C1FD6224627807Lithium's Principal Scientist of Analytics, digging into the complex dynamics of social interaction and group behavior in online communities and social networks.

 

Michael was voted a 2010 Influential Leader by CRM Magazine for his work on predictive social analytics and its application to Social CRM.He's a regular blogger on the Lithosphere's Building Community blog and previously wrote in the Analytic Science blog. You can follow him on Twitter at mich8elwu.

 


 

It’s been about 3 weeks since I last blogged on Lithosphere as I’ve been busy traveling around UK and Italy for both work and play. The play part involved my wife and I traveling around UK and Italy, sightseeing and enjoying delicious food and fine wine. In UK, we visited Oxford, Stratford-upon-Avon, and Bath. And before we left for Italy we even stopped by Salisbury to see Stonehenge. Then, in Italy, we toured Milan, Florence, and Rome. And of course we couldn’t miss visiting the Vatican.

 

The work component of the trip involved me speaking at a number of conferences, business meetings, social media events, workshops, and interviews. I talked about a wide range of topics from social CRM, psychology of gamification, cyber anthropology, science of influence, the Facebook Engagement Index, as well as some of the more technical topics, such as machine learning, social network analysis and predictive social analytics.

 

Among the topics I presented, gamification was by far the most popular topic. It was heavily tweeted and several excellent blog articles resulted from my presentation at Digital Surrey (a very engaging not-for-profit community of digital professionals).

  1. The Science of Gamification (@mich8elwu at #digitalsurrey) by Mark Wilson
  2. The Science of Gamification on the gamification blog by Gabe Zichermann
  3. The Science of Gamification at DigitalSurrey on GameTuned by James Monjack
  4. Want to change behaviour? Pull the trigger on Strange Fascination by Jane Franklin
  5. Benjamin Ellis also took some very nice photos at the event

 

Now that you know where I’ve been, let me return to the topic of Facebook engagement. Last time I showed you the structural similarity between a Facebook fan page and a community. By treating fan pages as communities, we can develop a whole spectrum of engagement metric from the very shallow (level 0) fan count to something that is eight levels deep. And I talked about the first two levels in my last post.

 

  • Level 0: Total fan counts
  • Level 1: Active fans
  • Level 2: Interactivity (through comments) – Commented post fraction.

 

Today we will examine several deeper level engagement metrics.

 

Disentangling Interactivities

Since the Level 2 engagement metric looks at what fractions of the posts were interactive (i.e. commented), Level 3 hones in on the interactive posts and tries to quantify how much interaction took place in those posts. This is traditionally characterized by a metric called thread depth: the number of comments a post receives. I computed the average thread depth across all posts within a fan page and plotted the distribution on a log scale in Figure 3. The median level average thread depth is about 12.5, meaning that posts on fan pages receive about 12 comments on average.

 

fig03-04_Engagement Data_web.gif

 

Although the average thread depth is simple way to estimate the amount of interaction, it doesn’t go deep enough to distinguish who you are interacting with. So it cannot tell you whether it is the same fan posting 100 comments or 100 different fans posting a comment each. The latter is clearly more desirable because it means more of your fans are interacting with each other. So for the next level of engagement (Level 4), I computed the average number of unique fans per conversation (see Figure 4).

 

Here, the median level for the average number of unique fans per conversation is about 11.7. This means that posts on fan pages typically receive comments from 11 other fans (not counting the initiator of the conversation). Notice that this value is very close to the median level of the average thread depth. This observation suggests that most of the fans only post once within any conversation. This, as we shall see in the next post, will have significant implications in terms of the conduciveness of fan pages as a suitable environment for building relationships.

 

The Dynamics of Interactions

If your fans are engaged enough to post, comment, and interact with other fans, that is great. It is already quite an achievement already, but we are not done yet. The full spectrum of engagement can be very deep. The next level of engagement (Level 5) goes a step further and looks at the dynamics of the interaction between fans. That is, the timing and velocity of how fans interact on your fan page.

 

fig05_Engagement Data_web.gif

 

Having a fan who interacts with 10 other fans through 100 comments may sound excellent, but you might change your mind when you found out that it took them over a month to respond to each other. I computed the average response time between every posting and show the result in figure 5a. The median level average-response-time is about 9.8 hours. Since the response time data has large variance, I also computed the distribution of median response time between every message (figure 5b). Although the shape of this distribution is similar to that of average response time, the distribution is shifted. Now the median level of the median-response-time is only about 2.2 hours.

 

Do you know where your fan page stands among these distributions? Are you beating the median level? And if so, by how much?

 

Engagement_Depth_02_resize.gifConclusion

Besides giving you a little coverage of my recent European speaking tour, we returned to the subject of Facebook engagement. Today, we dug deeper to understand how fans engage within a fan page by looking at thread depth, the unique number of other fans they interact with, and the dynamics of these interactions. By doing so, we covered three more levels of engagement metrics. So our spectrum of engagement is five levels deep now.

 

  • Level 0: Total fan counts
  • Level 1: Active fans
  • Level 2: Interactivity through comments
  • Level 3: Thread Depth – amount of interaction
  • Level 4: Unique fans per conversation – with how many other fans?
  • Level 5: Average/median response time – dynamics of interaction

 

The Level 3 (thread depth) and Level 4 (unique fans) data suggest that most of the fans only post once within any conversation. We will see if this is indeed the case next time. This is a very important point, and we will discuss some of its consequence in subsequent posts.

 

e20boston2011-speaking-125x125.gifAlthough we are already at Level 5, we are not yet at the bottom. There are three more levels to go! In subsequent posts, I will reveal more data on the deeper levels of engagement. But for now, let me know what you think about this spectrum of engagement metrics. As usual, kudos, comments, suggestions, critiques, and discussions are always welcome. Stay tuned for even deeper level of engagement...

And by the way, I will be participating in the Big Data Analytics for Social Media panel at the Enterprise 2.0 Conference in Boston next week. So if you happen to be in Boston from June 21--23, then we are probably destined to meet.  🙂  Please stop by and say hello. See you next week in person, or till next time on Lithosphere.

 

 

Read more
Labels (1)
  • Engagement

6 14 19.9K
Lithium Alumni (Retired) Lithium Alumni (Retired)
Lithium Alumni (Retired)

Dr Michael WuMichael Wu, Ph.D. is 927iC9C1FD6224627807Lithium's Principal Scientist of Analytics, digging into the complex dynamics of social interaction and group behavior in online communities and social networks.

 

Michael was voted a 2010 Influential Leader by CRM Magazine for his work on predictive social analytics and its application to Social CRM.He's a regular blogger on the Lithosphere's Building Community blog and previously wrote in the Analytic Science blog. You can follow him on Twitter at mich8elwu.

 


 

Since our annual Lithium Network Conference (LiNC 2011) is this week, I’m going to take a little detour from our gamification journey. Rather than trying wrap up my mini-series on gamification, I am going to show you some exciting data from a little secret project (code name Project Atlas) that I’ve been working on. The details of this project will be revealed at LiNC, so stay tuned. And if you get the chance, come join us at LiNC.

 

A few months ago, I was posed an interesting question: “Can we quantify the level engagement on a Facebook fan page that is a little deeper than just the number of fans or likes?” My default answer was, “Probably... give me some data and some time to play with it.” Subsequently, I got a small data set from our Social Media Monitoring (SMM) platform.

 

This dataset consisted of ~39 million participation records from 11+ million unique fans on 3,050 fan pages spanning from Feb-2009 to May-2011. This may sound like a lot of data, but really it is a tiny data set compare to the data volume processed by our SMM platform. Today, I will show you some of the results from playing with this small data set.

 

A Facebook Fan Page is Structurally a Community

Before I go into the data, I want to take you back to a mini-series I wrote on Cyber-Anthropology. It examines social media from a relational perspective and describes the complementary roles of social networks and communities in the development of interpersonal relationships throughout human history. In that mini-series, I characterized the structural and functional differences between social networks and communities. If you missed those posts, I will summarize the key point here (feel free to skip ahead to the next section if you are familiar with the structural and functional differences): 

 

Social Networks:

  1. Held together by pre-established interpersonal relationships between individuals
  2. People know everyone that is in their social network (i.e. direct connections)
  3. Each person has one social network. But a person can have many different social graphs depending on which relationship we want to focus on (see Social Network Analysis 101)
  4. The primary anthropological function of social networks is to maintain people’s pre-existing relationships
  5. Social networks have a network structure
  6. Within each person’s social networks there are sub-communities with different interests

 

Communities:

  1. Held together by the common interests of a group of people
  2. Pre-existing relationships may exist, but are not required, so new members generally do not know any or most of the people in the community
  3. Any one person may be part of many communities at a given time
  4. The primary anthropological function of communities is to develop people’s weak ties into strong relationships
  5. Communities have overlapping and nested structures
  6. Within each communities, social networks develop naturally as people build their tie strength

 

From this perspective, we can see that although Facebook is definitely a social network, a fan page is structurally more like a community. It is held together by the common interest (e.g. around a brand), and most of the fans don’t know each other when they join. Moreover, people can be part of many fan pages at any given time. So a fan page is really a community within the Facebook social network.

 

The Depth of Engagement on Your Fan Page

fig00_Engagement Data_web.gifIf we are treating a fan page as a community, how can we measure the engagement of fans on that fan page? Well, to start with, clearly you need to have fans! People realized this and many have used fan count as a way to measure engagement, but fan count is in reality, like the total registration or membership of your fan page. When someone liked your fan page, they merely joined your fan page as a community member. However, as I described in an earlier post (i.e. No Game, No Gain: Realizing the ROI of Your Facebook Fans), the true value of your fans cannot be realized until they take actions to interact with you and with others. Therefore fan count is only the most superficial characterization of engagement, because it says nothing about the fans’ subsequent action and their interactions.

 

I consider fan count the level 0 engagement metric. Figure 0 shows the distribution of fan counts in my data set. We see a wide variety of pages with fans counts spanning over 7 orders of magnitude (from tens to 39 millions) with a median level around 3,400. Note: Most of the distributions we deal with are power-law distributed and must be displayed on a logarithmic scale.

 

I am going to describe several deeper engagement metrics for your fan page and show you some data. I will focus on the most visible action (i.e. posting a message or comment) for now, and describe other actions, such as likes, in subsequent articles. For the rest of this article, I’ve used fan pages that have 1,000 posts or more.

 

fig01_Engagement Data1_web.gif

 

fig01_Engagement Data2_web.gifIf fan count is level 0, then level 1 is should focus on the active fans of your fan page (i.e. those who posted something). With all things being equal, it is clear that a fan who posts something is probably more engaged than a fan who doesn’t post. Figure 1a shows the distribution of active fans across all the fan pages I examined. The median number of active fans is around 2,900. However, the number of active fans may be biased by the age of the fan page. Younger fan pages that haven’t been around long enough may not have the time needed to develop a large active-fan base. Figure 1b shows the distribution of age normalized active fans. After normalization, the median level for the number of active fans per day is only about 19, but the most active pages can still have up to 2,200 active fans per day.

 

As we have learned from observing the 90-9-1 rule, the majority of your fans are inactive at any given time and only a small fraction of your fans are actively participating. Figure 1c shows that a conservative estimate of the fractional active fans (i.e. active fans divided by the total fans). This distribution has a median value of 3.45%. So on average, only 3.45% of your fans are actively engaging (i.e. posting). This is slightly less than what the 90-9-1 rule would have predicted, but the distribution definitely covers the expected 10% active fans with a wide margin.

 

fig02_Engagement Data_web.gifIf your fans are sufficiently engaged to post messages on your fan page, then the next level of engagement (level 2) digs deeper and looks at what fraction of the posts are interactive. That is, what fraction of posts have a comment? Figure 2 shows the distribution of interactive posts. The data shows that a significant portion of the posts on fan pages are not interactive. The median level for the fraction of interactive posts is about 66.8% (with a pretty large standard deviation). That means on average, over 1/3 of the posts on any fan page will never get a response before disappearing down the stream and off the wall.

 

Conclusion

Alright, that is probably enough data for you today. The full spectrum of engagement can go very deep and we need to quantify each level of engagement to get a full picture of how well your fans engage. Unfortunately, fan count is merely the shallowest (level 0) of all engagement metrics and doesn’t tell you very much. We can go much, much deeper.

 

In this post I covered the first two levels, but there are actually eight levels of engagement, and each level goes deeper than the previous. I will cover the deeper level engagement metrics in the subsequent posts. After I introduce all the components, I will show you how to combine these different engagement levels into a single score that quantifies the overall engagement of your fans. But for now:

  • Level 0: Total fan counts
  • Level 1: Active fans
  • Level 2: Interactivity through comments

 

I must say that one of the best things that Lithium did for me, as a scientist, is acquiring Scout Labs (now Lithium SMM). Through our SMM platform, I basically get an unlimited supply of social and behavior data. To me, that’s data heaven! Project Atlas is just the beginning of my intellectual playground. I certainly look forward to sharing more data and deeper insights in the future.

ECEW2011_logo.gif 

By the way, after LiNC, I will be traveling in Europe starting next week for about 3 weeks. I will be participating in several speaking engagements, interviews, launch events, and a little bit of vacation between them. So I apologize in advance if I am unusually slow in responding. Coming up next week are:

  1. May 24th: European Customer Experience World 2011
  2. digital_surrey_logo_small.gifMay 26th: Digital Surrey on the Science of Gamification

If you are around London, I’d love to meet you. See you later and stay tuned for deeper engagement!

 

 

Read more
Labels (1)
  • Engagement

10 18 25.3K
About Science of Social Blog
Dr. Michael Wu, Khoros' former Chief Scientist, drills into the gamification, superusers, the value of big data and more.