Forum Discussion

tmarshall's avatar
tmarshall
Advisor
9 years ago

Bulk Data - Looping Delay Needed?

Hi NaokiT   We are running the automated script now and executing the script every 3 or 4 minutes to be able to catch up from 1/1/2015 to present. The first 6 months the file size is very small and...
NaokiT's avatar
NaokiT
Lithium Alumni (Retired)

Hi tmarshall

 

Unfortunately it's hard to guess how long it will take for a 24 hour file transmission to complete as there are multiple factors involved - file size, network latency, cluster response time, etc.  The only way to prevent this is to run a staggered query that would wait till the last transmission stops before firing next 24 hour window.  If you prefer to use the time based approach unfortunately I can't give you a time that will definitely work so you'll have to expand and experiment until the errors stop.

 

regards

Naoki

tmarshall's avatar
tmarshall
Advisor
9 years ago

NaokiT Thanks for the feedback! The Informatica BI expert was able to babysit the script, and we managed to get all the data into the database so we are now all caught up to the present. Thanks for your help and look forward to continued improvements to the bulk data api!

 

Thanks,

Tim