Khoros Community

Ranks Designed to "Flow" Part2

Lithium Alumni (Retired) Lithium Alumni (Retired)
Lithium Alumni (Retired)

An important mechanism for getting into the state of flow is to have a balance between ability and challenge. In a community, this means having a set of ranking criteria that matches your superusers' ability. If the criteria are too easy, superusers will quickly reach the top rank and become bored. But if the criteria are too difficult, superusers will become frustrated over their lack of advancement. In either case, the risk is that superusers will give up trying and abandon the community eventually.


Know your superusers!

Knowing your superusers is the key to designing a ranking structure that matches their capability. This is done by computing the post rate for all users that have registered for more than 2 weeks on the community. I will illustrate this calculation with Lithosphere as an example (see screen shot). As of June 6, 2009, Lithosphere had 524 members who have registration age of more than 14 days (2 weeks). Column B shows the total post count by these users (including blogs, ideas, comments and replies, but excluding deleted messages). Then I divided each user's post count by their respective registration age in days (column C) and sorted the resulting post rate (column D).




Now, if you believe the 90-9-1 rule, the top 1% (the 99 percentile) should be your superusers. In our experience, the fraction of superusers consists of only about 0.1% of the community population. Sean O'Driscoll also claimed that only 0.5% of the members in his data are superusers (see Interview with superuser guru Sean O'Driscoll from our 2009 Customer Conference). If we use our conservative estimate of 0.1%, the capability of Lithosphere's superusers is simply the 99.9 percentile of the post rates. Capability, in this example, means how fast your superusers can post. Using Excel's percentile function, you can calculate the 99, 99.5, or 99.9 percentile values from the post rates (column D). From the screen shot, we can see that the 99.9 percentile post rate is about 0.851 posts/day. This is the approximate capability of the superusers on Lithosphere. The superusers on your community will have a different number, and communities with very active superusers will have a larger number. I've certainly seen communities with superusers posting up to 35 posts/days. Armed with this knowledge of your superusers, you can now scale your community's ranking ladder so that its ranking criteria match your superusers' capability.


Our empirical data suggests that, on average, community members expect a promotion to higher rank once every month. This would require an average of 12 ranks in 1 year, or 24 ranks in 2 years. Using the ranking criteria formula I presented last time, we can calculate the post requirement for the 24th rank. If we use the incremental difference of 10 as in my previous blog, then c(24)=(10/2)×(24+242)=5×(24+576)=3000 posts. For Lithosphere superusers (posting 0.851 posts/day), this would take them roughly 3525 days or 9.7 years to reach a post count of 3000, because (3000 posts)÷(0.851 posts/day)=3525 days. Clearly this ranking criterion is too challenging for the Lithosphere superusers. At 0.851 posts/day, we can only expect the Lithosphere superusers to post about 621 posts in 2 years, since (2 years)×(365 days/year)×(0.851 posts/day)=621 posts. Even if we make a wild speculation that the superusers capability will double in 2 year, this would only brings the expected post count to 621×200%=1242 posts, nowhere near 3000 posts.


Clearly, we need to rescale this ranking structure. With simple algebra, I solved for the incremental difference as a function of the post criterion from the formula I presented in my previous blog article:

 incremental difference.png .

But what do we plug in for the post criterion c(n)? Since Lithosphere is a young community, the superusers definitely have room for improvement. I will assume a 20% yearly increase in the superusers' capability, so that their expected post count over 2 years could potentially reach (621 post)×(120%)×(120%)=894 posts. Therefore roughly 894 posts should be the proper post requirement for the 24th rank, hence d=2×(894 post)/(24+242)=1789/600=2.98. Rounding this to 3, we may now generate the optimal ranking criteria that are match to the capability of the superusers on Lithosphere. The post requirement for the first rank is 3 posts, then 9, 18, 30, 45, 63, 84, 108, 135, 165, 198, 234, 273, 315, 360, 408, 459, 513, 570, 630, 693, 759, 828, and finally 900 posts at the 24th rank. Even if Lithosphere's superusers did not improve their capability at all, they would have gotten approximately 621 posts in 2 years and ascend through 19 ranks requiring 570 posts.


So, is your ranking ladder too easy or too difficult for your superusers? A sanity check by computing the post rate of just the top users on your community and comparing that to the post requirement of the top rank can quickly tell you if your ranking structure makes sense. A ranking structure that is design to "flow" could engage and captivate your superusers with superior efficacy. Next time we will answer a question that every community manager once had, "How many ranks do I need, and how many is enough?" Stay tuned at mich8elwu.


About the Author
Dr. Michael Wu was the Chief Scientist at Lithium Technologies from 2008 until 2018, where he applied data-driven methodologies to investigate and understand the social web. Michael developed many predictive social analytics with actionable insights. His R&D work won him the recognition as a 2010 Influential Leader by CRM Magazine. His insights are made accessible through “The Science of Social,” and “The Science of Social 2”—two easy-reading e-books for business audience. Prior to industry, Michael received his Ph.D. from UC Berkeley’s Biophysics program, where he also received his triple major undergraduate degree in Applied Math, Physics, and Molecular & Cell Biology.