How to manage hundreds of configuration files with user settings - AppSettings or not?

3
votes

I am currently working on an asp.net core (c#) Web API Project.

Each call in to the Web API will have a unique key to help identify the user making the call.

Each caller will have a specific set of configuration values associated to it. I expect there will be about 100 to 200 different users who will access this Web API. I want to try and hold all the configurations for each caller in a configuration file using the unique key to separate each. I expect there will be around 30 to 50 settings per caller in the configuration file (could be up to 100 setting for some).

My question: Is using the AppSetting.json file a good approach to hold these values, or should I create a separate configuration json file for each called and load when required? Efficiency is something that will need to be considered.

2
c#asp.net.netasp.net-coreappsettings
Is there some reason you are not using a database to hold user data? Configuration files are best for application configuration, e.g., per environment, and shouldn't really be used for user profile data. – Polyfun
Where does the application store/get other data? Also, are the combinations of settings really unique per user, or could you create a number of profiles to share between users? – Bernard Vander Beken
"Best" should generally be avoided in questions, as there are many viable solutions, with specific pros and cons.. that said, neither of the approaches presented are what I'd recommend (or would want to maintain). It might be worth considering taking a step back from the artificially imposed either-or approach. Anyway, consider websites that have user accounts and many dozens of settings or pieces of information about a user: "most" are using SQL or some other database model. – user2864740
Ihow big are the settings? 1kb-100kb per user? Just use a Dictionary you load once in the StartUp, otherwise if they're larger in size you will want a Cache, NoSQL or RDBMS solution. Good luck – Jeremy Thompson
Thank you for the responses. At this stage I have not considered using a Database as I didn't want the overhead. But reading the comments I think maybe I should take a step back and reconsider the solution. My take away message is that the config approach is not the most pragmatic. – Junaid Vindhani

2 Answers

0
votes

In a given context it looks like the configuration per client is a part of your solution. I would strongly consider moving the configuration to the code and handle it as a normal part of the business domain.

Saving such things in an appsetting.json looks dangerous (easy to make a change in a configuration for a wrong client) and hard to maintain. This may also impact debugging as you might have a different set of configuration files on a dev and prod environments.

There is also a performance consideration. Reading numerous files from the disk is not the best thing to do. You could always cache it in the memory but then your memory footprint increases.

0
votes

I'd be thinking SQLite DB over JSON files.

For 100 users with 50 settings it will be much easier to manage in a dB and that will prevent file locking during periods of high concurrency in multi-threaded situations. If the settings aren't relational a better choice would be a NoSQL db.

Managing 5,000 settings spread across 100 to 200 files will be a nightmare. Consider NoSQL or a lightweight RDBMS if doing bulk SELECTs, UPDATEs, (INSERTs for app upgrades) etc is necessary.

It's so small the performance differences will be barely noticeable, hence the easiest to read, maintain and build on would be my professional advice.