Khoros Community

Highlighted
Advisor
Advisor

Have you ever edited your ranking formula?

Hey guys, what would you recommend if I need to fix my rank formula. When we started off as a community, our rank was too short and our community keeps growing, so we need to fix this. So we think these are our options:

 

  • Start over. Our concern here is if our users would get a little lost and the ones that came a long way would have to star from 0. 
  • Edit our rank.  We want to go for this option because it looks safer. 

 

Thanks in advance,

Angie

Tags (1)
0 Kudos
9 Replies 9
Honored Contributor Honored Contributor
Honored Contributor

Re: Have you ever edited your ranking formula?

Hi Angie (@aagurto)

 

I have had a similiar problem in the past. The community rank structure I had inherited had a lot of problems, such as:

 

- The vast majority of community members weren't ranking up at all beyond the lowest rank because the ranking structure demanded that all users above the lowest rank had to add tags, but few relatively users added tags (and it wasn't behaviour that we particularly valued or promoted).

 

- There weren't enough mid-level ranks so after ranking up once or twice it took a really long time to get to the next rank (most people never got there).

 

- Because the community was mature, most superusers had been at the top rank for a long time and there was no rank/incentive/reward for them to keep going or seperate the absolute "elite" superusers.

 

Throwing out and starting again was tempting because the ranking criteria was clearly broken and I didn't particularly like the rank names either, but I thought it was far too risky a strategy. If users felt at all like they were receiving a "lesser rank" under the new regime, it would be like telling them their past contributions were not valued and you would risk losing them. And of course there would be no way to predict who would feel that way as rank names tend to be arbitary. Eg is "trusted" higher or lower than "valued"? 

 

Editing the structure seemed a much safer path, and my focus was to add a lot of new ranks, remove the bottlenecks, reward/focus only on the behavour we wanted to encourage (posts, kudos and solutions) and ensure that no user went backwards in the new structure. This wasn't easy to do and required a lot of planning and tweaking of the algorithms, but the work was definitely worth it. 

 

I also highly recommend that you give users plenty of notice before the change. Make sure they understand why the change is happening and what impact it might have on them. Give your most important users extra time to absorb the change - tell them first. You might even like to involve your superusers to some extent in the planning process (although I would recommend you always keep your ranking algorithms secret - even from your superusers). 

 

Good luck,

 

Jason

Generic1.jpg

Reply
Loading...
Honored Contributor Honored Contributor
Honored Contributor

Re: Have you ever edited your ranking formula?

There is a product idea for a ranking predictor that would be very beneficial in situations like this.

----------------------------------

Lili McDonald
Community Manager @ National Instruments


Reply
Loading...
Lithium Alumni (Retired) Lithium Alumni (Retired)
Lithium Alumni (Retired)

Re: Have you ever edited your ranking formula?

Hi @aagurto. I definitely support all of the recommendations from  @JasonHill. For an existing community, building on what you have and addressing the shortcomings as you go, while time consuming, should help to ensure that no one moves down due to the change. One additional step I would recommend is creating a spreadsheet with the lowest rank at the bottom of the page, as we have it on the rank page in the admin. I have a column for the name, a column for the formula and using User Reports, a column with the current number of users in each rank. That will help you understand where some of the issues lie, such as places where people have topped out and are not advancing. Once you make your changes and push them live, I would update the rows with any new ranks you have added and the numbers of users now at the various ranks. Then monitor over time, starting weekly, then monthly then every six months as you feel more comfortable with the changes you have made. But remember that communities continue to evolve and grow over time, so it is very important to make sure that your rank structure evolves with the community. One part of that is making sure that at least some number of users continue to move up through the ranks and that no user tops out.



If my post answers your question, please mark it as an "Accepted Solution."
Reply
Loading...
Honored Contributor Honored Contributor
Honored Contributor

Re: Have you ever edited your ranking formula?

Good one @RobbL,

 

That was the same process I went through. I lot of user reports, playing with spreadsheets, and tweaking algorithms to see what the impact would be. Before I made any changes I also picked out about 100 users who (hopefully) represented all types of users on our community at all stages of their community journey. I plugged them into our new ranking structure to ensure that none went backwards. In addition, I also did this for our most important users - our superusers and any potential/emerging superusers I had identified. Again, it was crucial to make sure that none went backwards in the structure. After implementing, I then went back and checked again to ensure my calculations were correct and that everyone had moved to the correct rank as anticipated. The idea that @lilim has referenced would certainly make this process easier!

 

All the best with this process @aagurto, I'm sure all the hard work will pay off. We certainly noticed a significant lift in community activity following the change. 

 

Jason

  

Generic1.jpg

Reply
Loading...
Honored Contributor Honored Contributor
Honored Contributor

Re: Have you ever edited your ranking formula?

We have done much of the same as what is already stated above, for 7 languages. We extracted our ranks, we created new formula's (as we use a point based system) and ensure nobody, at least on the highest levels) would rank down. we de-coupled our expert program from the ranking structure (previously someone would "automatically" rank up in an expert rank when reaching that level, now we have admittance based programs) There may have been a user on the lower ranks that dropped but we had no idea how to avoid that, since our forums are pretty large and have been around for a while. I guess we had to take some collateral damage. For our APJ forum we actually created a dual rank structure as they needed to adopt the new formula based rank approach (post+kudo+AS = xx points = rank xyz) and they would indeed have too many users dropping to a lower level. I dont have the details of that but if you need them I can definately reach out to my peer CM.

@RobbL to see the users in each rank, I assume you pulled user reports for each rank? Is there any way to see the number of users (estimated) in the rank structure. Sounds like it could be great, being similar to the number of users that received a certain badge.

 

Would love to hear more :-)

 

Learning from others and helping where I can!
Community Passionista!
0 Kudos
Reply
Loading...

Re: Have you ever edited your ranking formula?

One quick tip: Turn off Real-Time Notifications before you start making the switch to avoid users getting spammed as you make changes.

Reply
Loading...
Lithium Alumni (Retired) Lithium Alumni (Retired)
Lithium Alumni (Retired)

Re: Have you ever edited your ranking formula?

Hi @Wendy_S. If you are asking whether you can determine the current number of users in a rank, you can do so from the CSV export. But I expect you are asking if there is a method, similar to what we have with badges, to be able to see the number of users that would qualify for a potential new rank. Unfortunately, the current answer is No. I agree, that would be very useful.



If my post answers your question, please mark it as an "Accepted Solution."
0 Kudos
Reply
Loading...
Honored Contributor Honored Contributor
Honored Contributor

Re: Have you ever edited your ranking formula?

I agree with @BrianOblinger. That's a great tip.

 

 

@RobbL You got it :-) that is what I was looking for. Maybe I will post it as an idea. It might simplify the process for any starting OR aging community to see which rank buckets people fall in to (and possibly get stuck in)

 

Great discussion!
Wendy

Learning from others and helping where I can!
Community Passionista!
0 Kudos
Reply
Loading...
Advisor
Advisor

Re: Have you ever edited your ranking formula?

Hey guys, 

 

@JasonHill thank you so much for that input. I totally understand what you guys are going for. Me and my team had the made the decision to edit the rank we have now. We've had some superusers a little bit stuck in the top level, so we're making a long way ahead now. Also, as you and @RobbL, mentioned, we are promoting specific behaviors too. And we are thinking more of this rank as a way to develop our users, too. 

 

We'll also be really careful when we communicating this change to our community.  We'll keep close our super users, too.

 

@Wendy_S glad you enjoyed this thread.  

 

@BrianOblinger I'll do so. Thanks for the tip.

 

Thanks again, have a great week.

Reply
Loading...