Community Analytics numbers retroactively changing

[MichaelGrayson]

I have noticed that several metrics for a specified date range last month (23rd-29th) are no longer reading the same values as they did 2 weeks ago. 

The total visits, total unique visitors, and visit referrals have all changes by around 5%. 

Does anyone know what might be causing this? Or how it might be avoided / predicted? 

[StanGromer]

These types of questions risk melting your brain. As a best practice, we lock our data and never look back at month end - In the sense it's all living outside of Khoros.  

As for why - If you moved a node/section, those views move with it.  If you delete a node/section, you just deleted those sections so the views no longer exist (I believe...).  Further, depends on what numbers you are looking at, and when.  Khoros Analytics can take up to 48 hours to reflect properly, so if you tried to pull month-end data on the 1st of the month, it could be possible you missed an entire days worth of data without knowing it due to the delays.

Then there is the random mix of potential bugs, bot traffic, random hiccups in the server, etc.

TLDR; Nobody can probably give you a 100% correct answer, because reasons 🍺

[MichaelGrayson]

The data was collected first 5 days after the end of the period and was for the entire community - so that rules out the first few ideas.


Sounds like a hodgepodge of various factors seems most likely.

Thanks for the input!

[PrernaR]

@MichaelGrayson 
The data should not retroactively change unless we(The community analytics team) have re-processed and backfilled data for a historical time period for a specific reason. 

Community Analytics data gets refreshed everyday between 4 AM - 5 AM UTC unless the data processing sees some issues which could induce delays.

So if we're looking for data for 29th May, it would have been available by 30th May 5 AM UTC. 

Another potential cause of numbers slightly varying is the use of an approximation algorithm (hyperloglog) in the backend which can lead 1-2% variance at times. 

Please let us know if you're continuing to see any more issues. 

CC @SudheeshN 

[StanGromer]

@PrernaR You have to realize how crazy of an answer that is?  You start by saying the data should not retroactively change, yet finish with the fact it could be off randomly by 1%-2% at any time.  This is why I joked about this topic melting my mind, and now it just melted again. We are trying to manage businesses off this data, yet not a single person can explain the data with any type of truthful answer, as there is no truth to be had.

[PrernaR]

@StanGromer  
Apologies if the response resulted in more of a confusion. Let me try to provide some more details in an attempt to clarify. 

The first two causes I mention, 

1. The data should not retroactively change unless we(The community analytics team) have re-processed and backfilled data for a historical time period for a specific reason. 

2. the data processing sees some issues which could induce delays.

These are rare occurrences and none of it occurred in the time period that's referred to here 23rd May - 29th May.  If delays/changes in data occur, we post notifications on the Community Analytics Status Page 

 3. You start by saying the data should not retroactively change, yet finish with the fact it could be off randomly by 1%-2% at any time.

 This might be a misleading statement. 

The approximation algorithm won't result in random variations,  it will take effect only if : 

• The data is being queried cumulatively across days and months 
  and 
• There are overlapping visits (explained below) across those time periods.

Also this is only for Unique Visits and Unique Visitors. 

Reference to the algorithm on the backend. 
https://www.elastic.co/blog/count-elasticsearch 

However for a fixed time period being queried,  data should remain same each time.  

Some more details on the scenarios for overlapping visits. 

1. Overlapping visits across 2 months

'Visits' is defined as the number of unique user sessions. A session is defined as a group of interactions one user takes within a given time frame. Session times out in the case of 30-minute inactivity.

Assume that a user was browsing through a community during the last hour of a month (Month-1) and he/she continued browsing through the first hour of next month (Month-2)

When we measure visits for Month 1, this user's session will get counted as 1 visit

When we measure visits for Month 2, this user's session will get counted as 1 visit

So, if you sum up visits for Month1 and Month 2, you will get 2.

Whereas, if you measure visits for the entire 2 months time period, this user session will be counted as a single visit.

Hope this explains. 

[PrernaR]

@MichaelGrayson 
If you're still seeing changes with numbers please let us know the community you're querying for , the date ranges and the metrics. And we can file a ticket for investigating it.