Azure Worker Role Scheduled Tasks

Question

I have created a scheduled task system. I am using a Windows Azure worker role with half a core. I am pulling the tasks from a SQL Azure Database, which I queue and dequeue. Tasks can run independent of each other. I estimate that each worker role can execute thirty tasks in an hour. As my solution grows, I am concerned I must add a new worker role for each new thirty customers. If I get over a thousand customers, I would have about 35 worker role projects.

Is the worker role the wrong solution for my business requirements?
The code for each worker role project is the same. Is there a way I can execute the code below for more worker roles without adding another project?
I have tried the parallel task library, however, my code fails to work properly. On rare occasion, a customer's orders would end up in the wrong account; user names and password were getting crossed. Even with parallel tasks, there would still be a limit on the number of tasks?

Here is my code:

public class WorkerRole : RoleEntryPoint
{
    string conString = ConfigurationManager.ConnectionStrings["XXX"].ConnectionString;
    IJMATaskProvider m_TaskProvider;

    public override void Run()
    {
        // This is a sample worker implementation. Replace with your logic.
        //Trace.TraceInformation("CloudCartConnector.TaskRole entry point called", "Information");
        while (true)
        {
            Thread.Sleep(30010);
            ExecuteTasks();
            Thread.Sleep(500010);
            Trace.TraceInformation("Working", "Information");
        }
    }

    public override bool OnStart()
    {
        ServicePointManager.DefaultConnectionLimit = 12;
        m_TaskProvider = JMATaskFactory.Get(conString, CloudCartConnectorLogProvider.Instance);
        return base.OnStart();
    }


    void PerformTask(JMATask task)
    {

        //Update database
    }


    public string ExecuteTasks()
    {
        try
        {
            List<JMATask> tasks = m_TaskProvider.GetAllTasks();

            foreach (JMATask task in tasks)
            {
                PerformTask(task);
            }

            return "OK";
        }
        catch (Exception ex)
        {


        }
    }
}

Gaurav Mantri Gaurav Mantri · Accepted Answer · 2013-12-30T17:19:28

Is the worker role the wrong solution for my business requirements?

Absolutely NOT. Worker roles are meant for that purpose only - to process background tasks.

The code for each worker role project is the same. Is there a way I can execute the code below for more worker roles without adding another project?

As @Jakub mentioned, you can have many instances of same worker role running processing the tasks. They key here is that one task is processed by only one worker role instance.

One thing you could do is implement some kind of master/slave implementation where one worker role instance will fetch the tasks from the database, push them in the queue (master) and then all other worker role instances (slaves) will fetch the data from the queue and process tasks. I wrote a blog post some time ago which describes this approach that you may find useful: http://gauravmantri.com/2013/01/23/building-a-simple-task-scheduler-in-windows-azure/.

Azure Worker Role Scheduled Tasks

2 Answers