How to improve BlendEffect quality? Lumia Imaging 3.0 & UWP

Question

I am developing an app that put text into a background image. The textblocks are placed on a canvas control as children and I render the canvas to PNG. Then I use the BlendEffect to blend it with a background image.

UPDATE This post is specifically about chaining effects with Lumia Imaging SDK. Unfortunately one of the person who commented was having a tunnel vision and insist that I must learn about the difference between lossy and lossless image. An opinion as useful as telling a kid how to save an image in Microsoft Paint in my context. In some other comment, he even arrogantly wish that I must be having tons of bugs in my app and am having some mental problem.

I am here to learn about Lumia Imaging and obviously I encountered a person who has no experience with that SDK and insisted on showing off. Then down voted this post to make himself feel better.

With that said, @Martin Liversage was helpful by pointing out that it was not merely JPEG artifacts. I have tried playing with several chaining steps and options while saving as JPEG and indeed the images came out differently. While saving as PNG did improve the quality, it is still not what I expect. So I am here asking anyone with personal experience using the SDK about what I can I do in my code to improve my result.

Here's my code

private async void saveChainedEffects(StorageFile file)
{
    var displayInformation = DisplayInformation.GetForCurrentView();
    var renderTargetBitmap = new RenderTargetBitmap();
    await renderTargetBitmap.RenderAsync(Textify.CanvasControl);
    var width = renderTargetBitmap.PixelWidth;
    var height = renderTargetBitmap.PixelHeight;
    IBuffer textBuffer = await renderTargetBitmap.GetPixelsAsync();
    byte[] pixels = textBuffer.ToArray();

    using (InMemoryRandomAccessStream memoryRas = new InMemoryRandomAccessStream())
    {
        //Encode foregroundtext to PNG
        var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, memoryRas);

        encoder.SetPixelData(BitmapPixelFormat.Bgra8,
                             BitmapAlphaMode.Straight,
                             (uint)width,
                             (uint)height,
                             displayInformation.LogicalDpi,
                             displayInformation.LogicalDpi,
                             pixels);

        await encoder.FlushAsync();

        IImageProvider effectBackground;

        if (SelectedEffect.Name == "No Effect")
        {
            effectBackground = imageProcessorRenderer.M_Source;
        }
        else
        {
            effectBackground = (SelectedEffect.GetEffectAsync(imageProcessorRenderer.M_Source, new Size(), new Size())).Result;
        }

        StreamImageSource streamForeground = new StreamImageSource(memoryRas.AsStream());

        //Sharpening the text has unintended consequences to I set to 0d
        using (SharpnessEffect sharpnessEffect = new SharpnessEffect(streamForeground, 0d) )
        using (BlendEffect blendEffect = new BlendEffect(effectBackground, sharpnessEffect, BlendFunction.Normal, 1.0f))
        {
            string errorMessage = null;
            Debug.WriteLine("M_SourceSize (Normalized) {0}", imageProcessorRenderer.M_SourceSize);
            Debug.WriteLine("PreviewSize {0}", imageProcessorRenderer.PreviewSize);

            try
            {
                using (var filestream = await file.OpenAsync(FileAccessMode.ReadWrite))
                using (var jpegRenderer = new JpegRenderer(blendEffect) { Size = imageProcessorRenderer.M_SourceSize, Quality = 1.0, RenderOptions = RenderOptions.Mixed })
                {
                    IBuffer jpegBuffer = await jpegRenderer.RenderAsync().AsTask().ConfigureAwait(false);
                    await filestream.WriteAsync(jpegBuffer);
                    await filestream.FlushAsync();
                }
            }
            catch (Exception exception)
            {
                errorMessage = exception.Message;
            }

            if (!string.IsNullOrEmpty(errorMessage))
            {
                var dialog = new MessageDialog(errorMessage);
                await dialog.ShowAsync();
            }
        }
    }
}

Here's the image as seen on my PC screen before it's saved to JPEG

Then when the image is saved to JPG, there's a noticeable reduction in the quality as seen below. Enlarge the image and pay attention to the edges of the font.

So what are my options if I want to get as close as to the original image quality?

Answer 1

From @Martin Liversage and @Hans Passant feedback, I've decided to avoid JPEG and save to PNG instead. The artifacts on the edges of fonts mostly gone. I guest now I just have to play with the image sharpening to get the result I want.

UPDATE I still do not get the quality level that I want so it's not simply don't use JPEG and everything will be perfect. Hence the question, how to improve BlendEffect quality.

Here's my code

private async void saveChainedEffects(StorageFile file)
{
    var displayInformation = DisplayInformation.GetForCurrentView();
    var renderTargetBitmap = new RenderTargetBitmap();
    await renderTargetBitmap.RenderAsync(Textify.CanvasControl);
    var width = renderTargetBitmap.PixelWidth;
    var height = renderTargetBitmap.PixelHeight;
    IBuffer textBuffer = await renderTargetBitmap.GetPixelsAsync();
    byte[] pixels = textBuffer.ToArray();

    using (InMemoryRandomAccessStream memoryRas = new InMemoryRandomAccessStream())
    {
        //Encode foregroundtext to PNG
        var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, memoryRas);

        encoder.SetPixelData(BitmapPixelFormat.Bgra8,
                             BitmapAlphaMode.Straight,
                             (uint)width,
                             (uint)height,
                             displayInformation.LogicalDpi,
                             displayInformation.LogicalDpi,
                             pixels);

        await encoder.FlushAsync(); 

        IImageProvider effectBackground;

        if (SelectedEffect.Name == "No Effect")
        {
            effectBackground = imageProcessorRenderer.M_Source;
        }
        else
        {
            effectBackground = (SelectedEffect.GetEffectAsync(imageProcessorRenderer.M_Source, new Size(), new Size())).Result;
        }

        StreamImageSource streamForeground = new StreamImageSource(memoryRas.AsStream());

        //Sharpen the text
        //using (SharpnessEffect sharpnessEffect = new SharpnessEffect(streamForeground, 0.3d) )
        using (BlendEffect blendEffect = new BlendEffect(effectBackground, streamForeground, BlendFunction.Normal, 1.0f))
        using (BitmapRenderer bitmapRenderer = new BitmapRenderer(blendEffect, ColorMode.Bgra8888))
        {


            Bitmap bitmap = await bitmapRenderer.RenderAsync();
            byte[] pixelBuffer = bitmap.Buffers[0].Buffer.ToArray();

            using (var stream = new InMemoryRandomAccessStream())
            {
                var pngEncoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, stream).AsTask().ConfigureAwait(false);

                pngEncoder.SetPixelData(BitmapPixelFormat.Bgra8, 
                    BitmapAlphaMode.Straight, 
                    (uint)bitmap.Dimensions.Width, 
                    (uint)bitmap.Dimensions.Height, 
                    displayInformation.LogicalDpi, 
                    displayInformation.LogicalDpi, 
                    pixelBuffer);

                await pngEncoder.FlushAsync().AsTask().ConfigureAwait(false);

                //Need an IBuffer, here is how to get one:
                using (var memoryStream = new MemoryStream())
                {
                    memoryStream.Capacity = (int)stream.Size;
                    var ibuffer = memoryStream.GetWindowsRuntimeBuffer();
                    await stream.ReadAsync(ibuffer, (uint)stream.Size, InputStreamOptions.None).AsTask().ConfigureAwait(false);
                    string errorMessage = null;

                    try
                    {
                        using (var filestream = await file.OpenAsync(FileAccessMode.ReadWrite))
                        {                                 
                            await filestream.WriteAsync(ibuffer);
                            await filestream.FlushAsync();
                        }
                    }
                    catch (Exception exception)
                    {
                        errorMessage = exception.Message;
                    }

                    if (!string.IsNullOrEmpty(errorMessage))
                    {
                        var dialog = new MessageDialog(errorMessage);
                        await dialog.ShowAsync();
                    }
                }
            }                
        }
    }
}

UPDATE 2

I've fixed the issue and would like to share my findings in case anyone is in similar situation. It turned out that it's not so much about saving to JPEG or PNG. While PNG does improve the final image, you have to pay attention to the followings

1. Make sure that the foreground and background image has similar dimensions. Lumia Imaging will stretch the foreground by default to cover the background and the aspect ratio won't be respected.

2. If you need to resize your foreground, do it in renderTargetBitmap.RenderAsync use the overload that allows you to specify the size.

3. Avoid resizing your foreground using SoftwareBitmapRenderer especially when you are working with transparent image before blending. For me this affected my image fidelity and reduced the color range for some reason.

This is how rendered the foreground beforehand which blends perfectly with the background.

private async Task<SoftwareBitmap> renderForeground()
{
    RenderTargetBitmap renderTargetBitmap = new RenderTargetBitmap();

    await renderTargetBitmap.RenderAsync(gridToRender, (int)imageProcessorRenderer.M_SourceSize.Width, (int)imageProcessorRenderer.M_SourceSize.Height);
    var width = renderTargetBitmap.PixelWidth;
    var height = renderTargetBitmap.PixelHeight;
    IBuffer textBuffer = await renderTargetBitmap.GetPixelsAsync();

    //Get the output into a software bitmap.
     var outputBitmap = new SoftwareBitmap(
         BitmapPixelFormat.Bgra8,
         width,
         height,
         BitmapAlphaMode.Premultiplied);

    outputBitmap.CopyFromBuffer(textBuffer);

    return outputBitmap;
}