Azure Service Fabric Scaling

Question

I have an app that runs scheduled tasks on Azure Service Fabric. My app must run thirty to forty tasks at the same time, so I am using asynchronous programming. I have some questions:

Do you recommend running the tasks asynchronously? If not, should I run the task synchronously and scale up? How do I scale up? I need no return information from running the task.

Should I separate the queuing and dequeuing into separate stateful services? If so, how would these services communicate to each other?

Here is my code:

internal sealed class JMATaskRunner : StatefulService
   {
    public JMATaskRunner(StatefulServiceContext context)
        : base(context)
    { }

    /// <summary>
    /// Optional override to create listeners (e.g., HTTP, Service Remoting, WCF, etc.) for this service replica to handle client or user requests.
    /// </summary>
    /// <remarks>
    /// For more information on service communication, see http://aka.ms/servicefabricservicecommunication
    /// </remarks>
    /// <returns>A collection of listeners.</returns>
    protected override IEnumerable<ServiceReplicaListener> CreateServiceReplicaListeners()
    {
        return new ServiceReplicaListener[0];
    }


    public async Task<List<JMATask>> GetMessagesAsync()
    {
        await AddTasks();

        List<JMATask> ts = new List<JMATask>();

        IReliableQueue<JMATask> tasks =
            await this.StateManager.GetOrAddAsync<IReliableQueue<JMATask>>("JMATasks");

        using (ITransaction tx = this.StateManager.CreateTransaction())
        {
            var messagesEnumerable = await tasks.CreateEnumerableAsync(tx);

            using (var enumerator = messagesEnumerable.GetAsyncEnumerator())
            {
                while (await enumerator.MoveNextAsync(CancellationToken.None))
                {
                    ts.Add(enumerator.Current);
                }
            }
        }

        return ts;
        //return messagesEnumerable.ToList();
    }

    public async Task AddMessageAsync(JMATask task)
    {
        IReliableQueue<JMATask> tasks =
            await this.StateManager.GetOrAddAsync<IReliableQueue<JMATask>>("JMATasks");

        using (ITransaction tx = this.StateManager.CreateTransaction())
        {
            await tasks.EnqueueAsync(tx, task);
            await tx.CommitAsync();
        }
    }

    /// <summary>
    /// This is the main entry point for your service replica.
    /// This method executes when this replica of your service becomes primary and has write status.
    /// </summary>
    /// <param name="cancellationToken">Canceled when Service Fabric needs to shut down this service replica.</param>
    protected override async Task RunAsync(CancellationToken cancellationToken)
    {
        // TODO: Replace the following sample code with your own logic 
        //       or remove this RunAsync override if it's not needed in your service.

        IReliableQueue<JMATask> tasks =
            await this.StateManager.GetOrAddAsync<IReliableQueue<JMATask>>("JMATasks");

        while (true)
        {
            cancellationToken.ThrowIfCancellationRequested();

            var messagesEnumerable = await GetMessagesAsync();

            using (ITransaction tx = this.StateManager.CreateTransaction())
            {
                foreach (var message in messagesEnumerable)
                {
                    var result = await tasks.TryDequeueAsync(tx);
                    await PerformTask(result.Value);
                }

                await tx.CommitAsync();
                await Task.Delay(TimeSpan.FromSeconds(2), cancellationToken);
            }
        }


    }

    async Task<JMATask> PerformTask(JMATask task)
    {
        await Task.Run(() => Perform(task));
        return task;
    }

    void Perform(JMATask task)
    {
        Thread.Sleep(50000);
    }

    async Task<JMATask> AddTasks()
    {
        m_TaskProvider = JMATaskFactory.Get(conString);

        //List<JMATask> tasks = m_TaskProvider.GetAllTasks();

        //foreach(JMATask task in tasks)
        //{
        //    await AddMessageAsync(task);
        //}

        JMATask task = m_TaskProvider.GetJMATask(80);
        JMATask task2 = m_TaskProvider.GetJMATask(97);

        await AddMessageAsync(task);
        await AddMessageAsync(task2);
        return new JMATask();
    }
}

Clint Clint · Accepted Answer · 2016-07-11T13:08:22

For service fabric (and other actor-based systems), you typically want to scale out as opposed to scaling up [Scaling Up vs Scaling Out] see (http://www.vtagion.com/scalability-scale-up-scale-out-care/)

Azure Article on scaling up and down (slight misnomer).

Essentially, service fabric takes care of most of the concerns around failover, load balancing etc.

The documentation on how to scale the clusters is well worth a read.

The documentation on reliable actors also goes into the "threading" model of the system, which as it's actor-based is fundamentally asynchronous.

Azure Service Fabric Scaling

3 Answers