Receiving non Base64 encoded messages via Queue Trigger on Azure Functions

TOC

Summary

Azure Functions users know that queue triggers can receive Base64-encoded messages as a prerequisite, and other messages are treated as errors and moved to {YOUR_QUEUE_NAME}-items-poison.

The prerequisite can be changed by writing the messageEncoding parameter in host.json if you use Microsoft.Azure.WebJobs.Extensions.Storage v5. This setting also seems useful in terms of the changes that Azure.Storage.Queue v12 no longer automatically Base64 encode and decode.

When a message is sent to the queue with a version of the SDK prior to v12, it is automatically Base64-encoded. Starting with v12, that functionality was removed. When you retrieve a message by using the v12 SDK, it is not automatically Base64-decoded. You must explicitly Base64-decode the contents yourself.

https://docs.microsoft.com/en-us/azure/storage/queues/storage-tutorial-queues?toc=%2Fazure%2Fstorage%2Fqueues%2Ftoc.json&tabs=dotnet%2Cenvironment-variable-windows

If you specify a blob trigger or queue trigger as the condition when creating a new Azure Functions project, Microsoft.Azure.WebJobs.Extensions.Storage will be specified as a dependent package. The following are the dependencies when creating a new Azure Functions project in Visual Studio (Windows).

  • Visual Studio 2022
    • Microsoft.NET.Sdk.Functions v4.0.1
    • Microsoft.Azure.WebJobs.Extensions.Storage v4.0.5
    • Azure Functions Runtime v4
    • .NET 6
  • Visual Studio 2019
    • Microsoft.NET.Sdk.Functions v3.0.1.3
    • Microsoft.Azure.WebJobs.Extensions.Storage v3.0.1.0
    • Azure Functions Runtime v3
    • .NET Core 3.1

Prerequisites

  • Azure Functions
    • Host
      • Runtime: v4.1.3.17473
      • OS: Windows
    • Local
      • OS: Windows
      • Editor: Visual Studio 2022 (v17.1.1)
      • Azure Function Core Tools
        • Core Tools: v4.0.3971
        • Function Runtime: v4.0.1.16815
      • Azure Storage: On Azure
  • Nuget Packages
    • Microsoft.NET.Sdk.Functions v4.1.0
    • Microsoft.Azure.WebJobs.Extensions.Storage v5.0.0

Source code

Exec Run & Debug Azure Functions project with the code below in Visual Studio, and try adding it by unchecking Base64 encoding when adding messages to the queue in the Azure portal. You can confirm that the message can be received by Function.

csproj

Simple.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <AzureFunctionsVersion>v4</AzureFunctionsVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Storage" Version="5.0.0" />
    <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="4.1.0" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" Version="4.0.1" />
  </ItemGroup>
  <ItemGroup>
    <None Remove="Microsoft.Azure.WebJobs.Extensions.Storage" />
    <None Remove="Microsoft.NET.Sdk.Functions" />
    <None Remove="Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator" />
  </ItemGroup>
  <ItemGroup>
    <None Update="host.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="local.settings.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <CopyToPublishDirectory>Never</CopyToPublishDirectory>
    </None>
  </ItemGroup>
</Project>

host.json

Add messageEncoding parameter whose valid value is base64 or none.

{
    "version": "2.0",
    "extensions": {
        "queues": {
            "messageEncoding": "none"
        }
    }
}

The messageEncoding parameter detail is here:

This setting is only available in extension version 5.0.0 and higher. It represents the encoding format for messages. Valid values are base64 and none.

https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue?tabs=in-process%2Cextensionv5%2Cextensionv3&pivots=programming-language-csharp#host-json

As an aside, I couldn’t change it by overwriting the value of host.json in local.settings.json.

Function App

Using pretty simple Function App.

using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;

namespace MyFunctionApp
{
    public class Function1
    {
        [FunctionName("Function1")]
        public void Run([QueueTrigger("myqueue-items", Connection = "")]string myQueueItem, ILogger log)
        {
            log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");
        }
    }
}

Warning

Microsoft.Azure.WebJobs.Extensions.Storage v5 doesn’t include the Table Storage bindings.

https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue?tabs=in-process%2Cextensionv5%2Cextensionv3&pivots=programming-language-csharp

Version 5.x of the Storage extension NuGet package and version 3.x of the extension bundle currently don’t include the Table Storage bindings. If your app requires Table Storage, you must continue using version 4.x of the extension NuGet package or version 2.x of the extension bundle for now.

But, great news is here:
What’s new in the Azure Functions Tables extension for .NET Beta – Azure SDK Blog

Previously, the Azure Tables bindings for .NET were available as part of the Microsoft.Azure.WebJobs.Extensions.Storage package (version 4 and earlier). Version 5 of the storage extensions package no longer contains the Tables bindings. Instead, the Tables bindings are available in a new package called Microsoft.Azure.WebJobs.Extensions.Tables, which is now in beta.

References

  1. Azure Queue storage trigger and bindings for Azure Functions overview | Microsoft Docs
  2. NuGet Gallery | Microsoft.Azure.WebJobs.Extensions.Storage 5.0.0
  3. What’s new in the Azure Functions Tables extension for .NET Beta – Azure SDK Blog
Let's share this post!
TOC
閉じる