Forum Discussion

PerBonomi's avatar
5 years ago

Where do I store a large amount of data, for reference?

Hi folks, planning a custom component which will basically check the user's id against a pre-existing list.

I imagine this list could grow into the thousands. What are my options for storing this data/list?

One option I use for smaller datasets is a hidden post and simply store it as a comma separated string.

Another option might be to store a string in a custom metadata field, I guess? Any ideas how large that could be?

(I don't have access to a separate server.)

5 Replies

  • Are you not able to store the list as a role? You can update the role list from CSV as needed. That puts your users in the database with "native" tools for searching/lookup.

    I recently added a block of code to our header to change one of the top links for users with certain roles. I imagine this would work similarly, and can share a code example if you'd like.

    If that isn't an option, I might would consider a JSON or JS array stored in a custom content block (not a component) so that you can update it from Admin without needing to publish via Studio.

  • PerBonomi you can create custom role eg. "VIP", and assign this role to any users you like. Then you can write some logic to show something to the users with this role. This way, you can do 100 of thousands of users without damaging the system😀

  • Thanks for your thoughts everyone. We have a lot of roles already so I'm a bit loathe to add more, for a workaround, essentially.

    I do like the json file in assets, but even if the risk is small, those files are public, so could technically be found? Considering I'm planning to store user data I'd rather avoid that.

    Guess I'm down to using a custom metadata field linked to community or a user account.

  • AdamN's avatar
    AdamN
    Khoros Oracle
    5 years ago

    If you're wanting to store the data in the plugin, but don't want it publicly accessible, you could just store it directly in a custom component and then include/import that custom component as needed in your code. If you're working with the Plugin SDK, you could perhaps even build some automation around this to simplify your process.

    If you need to update the data on the fly in production (without going through Studio Self Publish), I agree that a custom setting is the best way to store this data. Though the idea of posting to a private board is a clever work-around, and one I've seen leveraged before for similar purposes!

    I saw you mentioned not having access to a separate server, but for anyone else with a similar use case in the future, leveraging the http.client context object via Freemarker might be an option to pull in the data as needed. Just be aware that you can only use this context object in Page Initialization (Common.init.ftl) and Endpoints, and you need to be careful about Page Initialization since the call is blocking and could thus slow down your page loading when executed.