Botbuilder, serve image as base64 string

Question

For a chatbot built in c# I need to show a profile card (ThumbnailCard)with an image. Images are stored in a database as base64.

CardBuilder:

internal static ThumbnailCard CreateProfileCard(Belegplan profile)
{
    ThumbnailCard card = new ThumbnailCard()
    {
        Title = $"{profile.Nachname}, {profile.Vorname}",
        Text = $"({profile.Bereich}-{profile.Jobtitel}). {profile.SitznummerGesamt}:: Stockwerk: {profile.Stockwerk}, Tischgruppe: {profile.Tischgruppe}, Sitznummer:{profile.Sitzplatz?.ToString()}.",
    };
    String imgUrl = DataService.GetPhotoDataByName(testchatbotservice.Properties.Settings.Default.DB_Azure, profile);
    if (!String.IsNullOrEmpty(imgUrl))
    {
        card.Images = new List<CardImage>()
        {
            new CardImage()
            {
                Url = "data:image/png;base64,"+ imgUrl
            }
        };
    }

    return card;
}

It works in the botframework emulator, but not when I use Web as channel. I get BadRequest when posting the card back.

Attachment attachment = new Attachment()
{
    ContentType = HeroCard.ContentType,
    Content = CardsBuilder.CreateProfileCard(person)
};
var reply = context.MakeMessage();
reply.Attachments.Add(attachment);
await context.PostAsync(reply, CancellationToken.None);

How should I create the card to display the image? In case base64 is not possible, storing the images in a blog storage table in Azure is an option, but it requires some adjustments in our workflow so it is not desired.

UPDATE

Images stored as varbinary(max). I get the fetch the images and then they are converted into base64 string Using an image from a static folder it works, getting the data from database still not working. The exception:

POST to test_chatbot_service failed: POST to the bot's endpoint failed with HTTP status 500 POST to the bot's endpoint failed with HTTP status 500

Failed method Microsoft.Bot.ChannelConnector.BotAPI+d__29.MoveNext

{ "message": "An error has occurred.", "exceptionMessage": "Object reference not set to an instance of an object.", "exceptionType": "System.NullReferenceException", "stackTrace": " at testchatbotservice.BasicLuisDialog.d__19.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.LuisDialog1.<DispatchToIntentHandler>d__9.MoveNext() in D:\\a\\1\\s\\CSharp\\Library\\Microsoft.Bot.Builder\\Dialogs\\LuisDialog.cs:line 270\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.LuisDialog1.d__8.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Dialogs\LuisDialog.cs:line 241\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.Internals.DialogTask.ThunkResume1.<Rest>d__5.MoveNext() in D:\\a\\1\\s\\CSharp\\Library\\Microsoft.Bot.Builder\\Dialogs\\DialogTask.cs:line 164\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Internals.Fibers.Wait2.-PollAsync>d__19.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Fibers\Wait.cs:line 0\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Internals.Fibers.Frame1.<Microsoft-Bot-Builder-Internals-Fibers-IFrameLoop<C>-PollAsync>d__9.MoveNext() in D:\\a\\1\\s\\CSharp\\Library\\Microsoft.Bot.Builder\\Fibers\\Fiber.cs:line 0\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Internals.Fibers.Fiber1.-PollAsync>d__16.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Fibers\Fiber.cs:line 0\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at Microsoft.Bot.Builder.Internals.Fibers.Wait2.Microsoft.Bot.Builder.Internals.Fibers.IAwaiter<T>.GetResult() in D:\\a\\1\\s\\CSharp\\Library\\Microsoft.Bot.Builder\\Fibers\\Wait.cs:line 378\r\n at Microsoft.Bot.Builder.Dialogs.Chain.LoopDialog1.d__3.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Dialogs\Chain.cs:line 752\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.Internals.DialogTask.ThunkResume1.<Rest>d__5.MoveNext() in D:\\a\\1\\s\\CSharp\\Library\\Microsoft.Bot.Builder\\Dialogs\\DialogTask.cs:line 164\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Internals.Fibers.Wait2.-PollAsync>d__19.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Fibers\Wait.cs:line 0\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Internals.Fibers.Frame1.<Microsoft-Bot-Builder-Internals-Fibers-IFrameLoop<C>-PollAsync>d__9.MoveNext() in D:\\a\\1\\s\\CSharp\\Library\\Microsoft.Bot.Builder\\Fibers\\Fiber.cs:line 0\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Internals.Fibers.Fiber1.-PollAsync>d__16.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Fibers\Fiber.cs:line 240\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.Internals.DialogTask.d__23.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Dialogs\DialogTask.cs:line 321\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.Internals.ReactiveDialogTask.d__3.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Dialogs\DialogTask.cs:line 376\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.Internals.ScoringEventLoop`1.d__5.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Dialogs\ScoringEventLoop.cs:line 87\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.Internals.EventLoopDialogTask.d__3.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\Dialogs\DialogTask.cs:line 435\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.Bot.Builder.Dialogs.Internals.SetAmbientThreadCulture.d__3.MoveNext() in D:\a\1\s\CSharp\Library\Microsoft.Bot.Builder\ConnectorEx\PostToBot.cs:line 103\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n at

Fei Han Fei Han · Accepted Answer · 2018-03-21T05:57:55

works in the botframework emulator, but not when I use Web as channel. I get BadRequest when posting the card back.

I do a test with the following sample code to send Thumbnail card with base64 binary of the image, which works fine with Web Chat channel. If possible, you can replace the imgUrl with the base64string of image that you stored in the database and test if the code work for you.

private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
{
    var activity = await result as Activity;

    int length = (activity.Text ?? string.Empty).Length;

    if (activity.Text=="card")
    {
        var replymess = context.MakeMessage();

        String imgUrl = ImageToBase64();

        List <CardImage> cardImages = new List<CardImage>();
        cardImages.Add(new CardImage(url: imgUrl));

        ThumbnailCard plCard = new ThumbnailCard()
        {
            Title = "I'm a thumbnail card",
            Text = "test pic",
            Images = cardImages
        };

        Attachment plAttachment = plCard.ToAttachment();

        replymess.Attachments.Add(plAttachment);

        await context.PostAsync(replymess);

    }
    else
    {

        await context.PostAsync($"You sent {activity.Text} which was {length} characters");
    }

    context.Wait(MessageReceivedAsync);
}

Image to Base64:

public static string ImageToBase64()
{
    var path = System.Web.HttpContext.Current.Server.MapPath(@"~\imgs\testpic.PNG");
    Byte[] bytes = File.ReadAllBytes(path);
    string base64String = Convert.ToBase64String(bytes);

    return "data:image/png;base64," + base64String;
}

Test result:

Besides, please check if your bot can work as expected when not send Thumbnail card. And to troubleshoot the issue, you can try to use fiddler to capture and check the request, which would help you get more useful and detailed error message about "BadRequest".

Botbuilder, serve image as base64 string

1 Answers