How to schedule a task for new data created?

Question

In my system, user can create a schedule with time and conditions. Before 30 mins of schedule time, if the conditions are not satisfied the system will raise an alarm to notice users about that.

My system are spring boot applications and using spring scheduled task to trigger alarms. The problems is when user creates a lot of schedule in the future, if I create a scheduled task for each schedule data, there will be memory problem.

My current solution is a create a schedule run at a time of everyday to scan all data in next 24 hours and create scheduled task for them to trigger alarm. This will reduce scheduled tasks created but if user creates new schedule data in next 24 hours after scanning, that data will be not trigger any alarm.

So what should I do?

Answer 1

Is there a reason that you are scheduling all of this in JVM memory? If the JVM crashes (or is simply rebooted), the timers would then be lost as if the user never scheduled any alarm. As you mentioned, creating a timer per request would likely not be a scalable solution.

Without knowing the specific details of your system, the most common approach would be to persist (i.e. in a DB, flat file, etc.) the data each time a user requests to schedule event. This way, in the event of a crash or reboot, you won't lose events. Similarly, this approach can scale to multiple servers if necessary. Then, at whatever granularity you support (i.e. minute, hour, day, etc.) there would be a process or thread (only a single monitor thread) find all of the events which have expired since you last ran. Finally, once this thread has identified events that need an "alarm," this one thread can control sending these events for active processing. This thread can either individually handle each event or otherwise submit them to an active work queue for parallelization.

More specifically, if you have alarms which could go off at any minute, you should schedule a monitor thread to run every minute. This thread should find all the events which require an alarm and then actually send that alarm.

Remember that how often you should schedule your monitor thread is a function of the resolution you want for your alarms and your tolerance for late alarms. If late alarms are totally unacceptable, then your monitor must run at least as often as the finest granularity for scheduling an alarm event. This is, of course, assuming alarms are always scheduled in the future-- otherwise, you will probably want to double the frequency of your monitoring checks. To see why, consider the following example:

minute 0: Run monitor
minute 0: User schedules alarm for minute 0
minute 1: Run monitor

If we run the monitor once per minute but allow the user to schedule an alarm in the current minute, it's quite possible that we'll miss the event (as shown in the example above). I can go into this more deeply if necessary, but this is here mostly for completeness as I have no indications from your description that this will actually pose any problems.

Good luck.