we would like to get !all! data (buzzword: big data) generated on the platform to join this with our other data pools and to relate the community activities with our classic DWH.
Is this possible? If yes, how can we get the data (highlevel technical explanation would be great)?
I guess the REST API is not appropriate for this...
I can only partially answer in that the REST API is definitely not appropriate especially if we are talking about a lively open community with a good level of content. It might work if you have a closed community with limited amount of content though.
Claudius wrote:[..] the REST API is definitely not appropriate, especially if we are talking about a lively open community with a good level of content. [..]
Thanks. This confirms what I assumed already regarding the REST API. And we plan (or at least hope ;-)) to get a high traffic community
Good question. Perhaps Lithium can chime in? Perhaps there is some custom, fee based additional access to the back end database possible?
besides the "big data" stuff, also for a regular DWH it will most likely not be appropriate to gather the data via REST API. In "classic" BI we want to stage our data first, ensure its integrity and make sure it still fits to the DWH data model, add aggregations and relations, et. cetera. To pull all the data via REST to begin this process will most probably not be good either.
Hence the question remains: any possibility to get bulk/mass/big data out of the SaaS platform besides REST API?
