Public
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Honored Contributor

Your Assistance Please: Customizing the ‘Community Experience’ survey

Good day everyone,

I would like to utilize Value Analytics to collect community satisfaction and would like to hear your best practices.

  • Is it possible to customize the Khoros ‘Community Experience’ survey questions?
  • May I change the template?
  • What are your best practices for frequency?

Thank you,

Toby

Community Manager - Seismic Software
8 Replies 8
Highlighted
Honored Contributor

For #1 and #2, not really.  Everything you can edit can be seen in Admin -> Features -> Value analytics.  Only some of the answers can really be changed, not the questions.  This is because they do bench marking across all Communities using this.  You'd have to use a third party survey platform to do any real changes.... Though if you were wanting to go the real brute force way, you can I believe modify the text key for the questions, thus making the questions say whatever you want them to say, but you would lose out on any benefits Khoros can offer during QBRs.

Frequency - I say test it and see what works over time - It really can depend on your user base, who they are/type of Community, etc.  We have ours set to I believe every 90 days right now, doesn't prompt for 2+ minutes.  

Highlighted
Honored Contributor

Thanks @StanGromer - I figured as much; thanks for the confirmation.

Cheers,

Toby

Community Manager - Seismic Software
Highlighted
Honored Contributor

PS @StanGromer 
I totally get "benchmarking across all Communities using this" but since this is my community, the ability to tailor the questions would be super helpful. Thanks again for your help.
Cheers,

Toby

@NickH @JakeR - is there a way to change the questions?
Would this need a product idea?
Thank you,
Toby

Community Manager - Seismic Software
Highlighted

I cant speak for Nick, but I do have some answers to your questions, Toby.  I also have a strong opinion on one of them as well, but I promise to be as delicate as I can with delivering it.  You know I have nothing but love for you, so forgive me if any of this comes across like a sledgehammer...

First, the quick and easy response to the question about 'frequency'.  To wit, who do you serve the survey to (authenticated users only or all Visitors), when do you serve it, and where do you serve it? 

I am a big believer in serving the survey to all Visitors (both anonymous and those who are authenticated / logged-in).  I have come across some customers that are extremely stringent and conservative when calculating the value of a deflection, and so they demand that anybody who gets a satisfied answer on the community and does not have to open a ticket has to be authenticated on the community before they count the deflection.  I think this sort of demand is a bit short-sighted (and it is certainly a thumb in the eye to what Richard Nixon called the 'silent majority' of people out there), but it may be just a cultural artifact within the business on how to measure things.  

Regarding where the survey is served, I think a Community Manager should be both prudent and surgical with the survey's deployment.  Meaning, it should only pop on boards where there are people potentially solving problems / getting answers.  I will admit to getting flustered when I see the survey popping in the 'watercooler' area of a Community, or even in the 'getting started' or 'welcome' area of the Community.  It just seems like a lack of attention to detail on the Community Manager's part when thinking about the digital customer experience.

Now, onto the crux of your specific question around 'when' the survey is served.  The frequency should be no earlier than 2 minutes and no more than 5 minutes.  Considering that a person is most likely coming from a search engine and landing right on the thread with their solution (or, at the very least, a discussion around their problem), 2-3 minutes should be all that they need to discover the solution to their problem.  However, if it is a B2B high-tech community where the answers tend to be a bit more complicated (and the conversations a bit more detailed), I think 4-5 minutes is the proper time.  There are always exceptions to these suggestions (i.e. - people who read more slowly than the national average, people that are distracted by other things on the page, people that are being distracted at home while trying to research this stuff, etc), and so no frequency rate is perfect.  But those are the best practices we have around frequency.

In regards to changing the template and/or the Customer Experience questions.  There is very little room for change.  I think there are some unorthodox hacks and also maybe some slight adjustments around industry / product wording, but by and large, there is little flexibility to alter the questions.  And I am extremely happy about this!

....Ok, let me clarify that last provocative sentence above with the strong opinion I mentioned at the beginning.

I'll start by quoting Mr. Spock from Star Trek II: The Wrath of Khan.  "The needs of the many outweigh the needs of the few....or the one".  As a red-blooded American, that sounds like commie malarkey to me, but as somebody who has worked with our customers for 13+ years and lived neck-deep in the data, Spock's philosophy really does need to be applied here.  The larger Khoros team needs to be able to tell customers, with confidence, what they should aspire to when it comes to performance.  The only way you get there is by having an unflinching and inflexible survey methodology.  Sad but true.  Doctors must struggle with a similar thing when patients want to be cleared too early from an injury or want to be discharged from a hospital too early.  There is an instinct to make people happy as well as an urge to satisfy them.  However, there is also a standard to keep in order for the science, data, and message to cleanly flourish. 

I have written before about the inflexibility around our Value Analytics survey elsewhere in this community and have spoken about it with many Khoros customers, on-site, at their offices.  Nobody likes being limited with what the questions and choices for answers can be, but everybody LOVES the confidence we have when it comes to telling them what the Resolution Rate of their Community should approximately be in Year 2 if they are a B2C High Tech company (or Year 1 if they are a Retail Community, or Year 3 if they are a B2B High Tech Community, etc).  If we allow everybody to change the questions and/or answers, now we no longer have that confidence....much less the data to speak with any real level of insight.

It may be a gigantic letdown to have to accept this, but I believe it would be worse to be stumbling around in the dark not knowing how your Community 'should be' performing against similar Communities in the same industry (B2C high tech, B2B high tech, CSP, Retail, Financial Services, etc) with the same audience (B2B, B2C, Internal, etc), and a similar age (Year 1, Year 2, Year 3, etc).  So the ability to modify both questions and answers in the survey is hampered only by the needs of the many outweighing the needs of the few.

Highlighted
Honored Contributor

Hi @JakeR 
Thank you for taking the time to respond.
To clarify a few things about my community:

  • It is private so it will only be seen by registered users and not passers-by.
  • It will fire after 4 min - I did not want it to launch as soon as a member visits; that's a terrible experience. Google Analytics tells me the ave visit is 7 min, so I made my decision with data.
  • My community is not a maze of boards so placement "should not" be an issue.

Of course, I want to benchmark against and learn from industry leaders, that is why I participate within this community and connect with other CMs on LinkedIn and Twitter. The goal of the survey is to hear what my members think about the community I have built for them: what is working and where can I improve. I also understand Khoros is not MailChimp, I just wanted to know if tailoring questions is possible. Why I asked: "should this be a product idea."

Best,
Toby

 

Community Manager - Seismic Software
Highlighted
Honored Contributor

I understand all the points you made @JakeR but my business still broke your model anyway. It was more important to us that the survey replicate the brand website experience so we can compare between the two sites.

As a result our metrics skew very heavily and our community is not really comparable to others using the Khoros survey. Hope you guys don't use us in bench marking 😛

Highlighted
Trusted Contributor

@Toby If you decide to play around with the text keys keep in mind that they’re not change in Community Analytics (was confirmed as a bug)


Highlighted
Honored Contributor

@DanK I can almost guarantee you they do leave them in.  You are the crazy wild outliers that nobody can explain flying off the page that throws the entire data set off 🤣