Forum Discussion

elbranscomb's avatar
elbranscomb
Executive
3 years ago

Bulk Data API - new user questions

Hi, we have just started using the Bulk Data API to get data from production and our team have a couple of questions that hopefully are simple to answer. We can't find the answers in the docs or here so sorry if these are covered somewhere else. 

1. fromDate and toDate is what we’re using to fetch the data. Just not entirely sure if it maps to the timestamp of event.time.ms or if it’s another timestamp (i.e. when event was processed, when event was made available for export, etc).
2. Does this endpoint paginate results in any way or is it always one giant dump?

Thanks!

Bulk Data API
  • JavidH's avatar
    JavidH
    3 years ago
    1. You are right. The fromDate and toDate are mapped to the event.time.ms timestamp.
      event.time.ms - represent the time of user activity occurred in the community (in UTC).
    2. Bulk API won’t support pagination, it is always whole data served as a stream for the requested timestamp.

    Hope this helps.

  • JavidH's avatar
    JavidH
    Khoros Staff
    3 years ago
    1. You are right. The fromDate and toDate are mapped to the event.time.ms timestamp.
      event.time.ms - represent the time of user activity occurred in the community (in UTC).
    2. Bulk API won’t support pagination, it is always whole data served as a stream for the requested timestamp.

    Hope this helps.

  • MarkSchwanke's avatar
    MarkSchwanke
    Boss
    3 years ago

    elbranscomb It's been some years since I used sql. Wondering if you know of any document or trick for setting up connection to do some simple queries locally. I looked for a little walk me through it document here but didn't find one unless I'm missing something.