1
votes

I have deployed Chat Bot into window server but from few days I am getting following error in Bot “There was an error sending this message to your bot: HTTP status code InternalServerError” and application insight shows following message "Operation returned an invalid status code 'Unauthorized" and it has happened 8 times in last 4 days.

What does it mean:

1) Messages are not able to reach to the window server.

2) Messages are reaching to the server and Chat bot is having some issue.

3) It’s an issue at the Microsoft end

I am using 3.15.3 version (Bot.builder).

Has anyone run into the same problem?

1

1 Answers

1
votes

Unfortunately it seems that this is an issue in the Bot Framework itself, the current workaround for this issue is to register an DocumentDbBotDataStore instead of the TableBotDataStore, in you Global.asax.cs_ApplicationStart put these:

var uri = new Uri(ConfigurationManager.AppSettings["DocumentDBUri"]);
var key = ConfigurationManager.AppSettings["DocumentDBKey"];

var store = new DocumentDbBotDataStore(uri, key);

Conversation.UpdateContainer(
    builder =>
    {
        builder.Register(c => store)
            .Keyed<IBotDataStore<BotData>>(AzureModule.Key_DataStore)
            .AsSelf()
            .SingleInstance();
    });

Edit:

There's a huge duct tape fix I didn't want to post that I implemented a while back and managed to get it to work. The idea is to basically locate the trouble making service and hack the bot framework by overriding it and force it to make a retry. The bot framework's token cache manager gets faulty at some point and the access token doesn't get refreshed in time before making a request we causes the Unauthorized Error, so we also hack it and force it to refresh the token.

Here's how:

We override the bot framework's IBotToUser decorator that makes an HttpRequest that causes the error:

public class RetryHandlerDecorator : IBotToUser
{
    private readonly IMessageActivity _toBot;
    private readonly IConnectorClient _client;

    public RetryHandlerDecorator(IMessageActivity toBot, IConnectorClient client)
    {
        SetField.NotNull(out _toBot, nameof(toBot), toBot);
        SetField.NotNull(out _client, nameof(client), client);
    }

    IMessageActivity IBotToUser.MakeMessage()
    {
        var toBotActivity = (Activity)_toBot;
        return toBotActivity.CreateReply();
    }

    async Task IBotToUser.PostAsync(IMessageActivity message, CancellationToken cancellationToken)
    {
        try
        {
            await _client.Conversations.ReplyToActivityAsync((Activity)message, cancellationToken);
        }
        catch (Exception e)
        {
            if (IsTransientError(e))
            {
                await HandleRetry(message, cancellationToken);
            }
            else
            {
                throw;
            }
        }
    }

    private async Task HandleRetry(IMessageActivity activity, CancellationToken token)
    {
        await ForceRefreshTokenAsync();

        await _client.Conversations.ReplyToActivityAsync((Activity)activity, token);
    }

    private async Task ForceRefreshTokenAsync()
    {
        var credentialsManager = new MicrosoftAppCredentials(
            ConfigurationManager.AppSettings[MicrosoftAppCredentials.MicrosoftAppIdKey],
            ConfigurationManager.AppSettings[MicrosoftAppCredentials.MicrosoftAppPasswordKey]);

        // force the generation of a new token
        // this will store the token in a static cache list, so no harm in creating a new instance of MicrosoftAppCredentials
        await credentialsManager.GetTokenAsync(true);
    }

    private static bool IsTransientError(Exception e)
    {
        switch (e)
        {
            case ErrorResponseException ex:
                return ex.Response.StatusCode == HttpStatusCode.Unauthorized;
            default:
                return false;
        }
    }
}

We override the bot framework's module that registers the IBotToUser service:

public class BotToUserModuleOverride : Module
{
    protected override void Load(ContainerBuilder builder)
    {
        builder.RegisterType<RetryHandlerDecorator>().Keyed<IBotToUser>(typeof(RetryHandlerDecorator))
            .InstancePerLifetimeScope();

        RegisterAdapterChain<IBotToUser>(builder,
                typeof(RetryHandlerDecorator), // this was previously AlwaysSendDirect_BotToUser 
                typeof(AutoInputHint_BotToUser),
                typeof(MapToChannelData_BotToUser),
                typeof(LogBotToUser)
            )
            .InstancePerLifetimeScope();
    }

    public static IRegistrationBuilder<TLimit, SimpleActivatorData, SingleRegistrationStyle> RegisterAdapterChain<TLimit>(ContainerBuilder builder, params Type[] types)
    {
        return
            builder
                .Register(c =>
                {
                    var service = default(TLimit);

                    return types.Aggregate(service,
                        (current, t) => c.ResolveKeyed<TLimit>(t, TypedParameter.From(current)));
                })
                .As<TLimit>();
    }
}

And finally we hack the bot framework's IoC container by registering our module that overrides the service, this needs to be done in the Global.asax.cs ApplicationStart:

Conversation.UpdateContainer(
    builder =>
    {
        builder.RegisterModule(new BotToUserModuleOverride());
    });

So what will happen is that whenever an UnauthorizedError gets raised by the ReplyToActivityAsync method, there will be a process forcing the refresh of the access token and retrying.