Forum Discussion

tmarshall's avatar
tmarshall
Advisor
9 years ago

Bulk Data API - Initial Load, verify daily pull

So we are still working on perfecting our bulk data API and have two more questions

 

1. We are limited to 24 hours pulls. How did others get historical data? Did you pull 24 hours at a time X times with X calls, or was Lithium able to provide a full load for 1 time load?


2. How do people sync up what is pulled via bulk data API versus the daily record count. Was another API used to get the count to be able to verify that the bulk data API was working correctly? How do others verify if all data is imported correctly into target system.

 

Thanks,

Tim

 

  • NaokiT's avatar
    NaokiT
    Lithium Alumni (Retired)
    9 years ago

    1.  You will have to write a script that calls 24 hours worth of data at a time for historic data.  Lithium won't be able to give you a historic one time dump.  

     

    2.  For verificaiton, unfortunately you wont' be able to verify using an API.  At this time there is no aggregated API on LSI data to get daily, weekly or monthly counts on a programattic basis.  It's something on our roadmap but for later in the year.  In the meantime, you'll have to manually periodically compare in UI or via CSV export.  

    • tmarshall's avatar
      tmarshall
      Advisor
      9 years ago

      NaokiT Thank you very much for the reply and look forward to the continued updates with bulk api.