Better Together: Azure Event Grid with Logic Apps

Introduction

The Azure Event Grid provides a seamless integration both between Azure to Azure and Azure to external services, enabling us to achieve an event-driven architecture.

In this blog post, we will be looking at an integration solution that I recently implemented to transact user registrations with Event Grid in Azure Blob Storage and the Logic Apps:

• We maintain an online web portal dedicated to the registration of new supplier users.
• Upon user registration, an automated process is initiated then store the supplier details in Azure Blob Storage.
• A Logic App is set up to monitor the system topic of the storage account’s Event Grid.
• The Logic App retrieves the event message, extracts file details, performs validation, and subsequently transmits it to an SFTP server. The file is then available for import by a third-party application.

The “When a resource event occurs” trigger is used in the Logic App, specifically listening to “BlobCreated” type events only.

Event Grid trigger

At this point, you might wonder “Why not use the built-in Blob storage trigger instead?”. The key benefit of the Event Grid trigger is it eliminates the need for continuous polling, allowing Logic App to operate in a reactive mode, i.e. it springs into action only when a new Blob is created in Blob Storage, resulting in a reduction of resource consumption and costs💸.

Deploying Event Grid Resources

From the Azure Portal, configuring a Logic App with Event Grid is a breeze, but the real challenge surfaces when you need to seamlessly integrate it into a CI/CD pipeline. The tricky part here was getting the deployment of the ARM template for the Event Grid subscription just right.

While Azure UI facilitates the creation of a new subscription tied to a Logic App with an Event Grid trigger, doing the same from an ARM template requires the use of functions, especially if your Logic App resides in a separate resource group from the storage account.

Here’s my template for the Event Grid Topic and the Event Subscriptions

 {
      "type": "Microsoft.EventGrid/systemTopics",
      "apiVersion": "2023-06-01-preview",
      "name": "[variables('eventGridTopicName')]",
      "location": "[resourceGroup().location]",
      "properties": {
          "source": "[concat(subscription().id,'/resourceGroups/edi-', parameters('env'), '-rg/providers/Microsoft.Storage/storageAccounts/',parameters('storageAccountName'))]",
          "topicType": "microsoft.storage.storageaccounts"
      }
    },
    {
      "type": "Microsoft.EventGrid/systemTopics/eventSubscriptions",
      "apiVersion": "2023-06-01-preview",
      "name": "[concat(variables('eventGridTopicName'),'/',variables('logicAppName'))]",
      "dependsOn": [
          "[resourceId('Microsoft.EventGrid/systemTopics', variables('eventGridTopicName'))]"
      ],
      "properties": {
          "destination": {
              "endpointType": "WebHook",
              "properties": {
                "endpointUrl": "[listCallbackUrl(concat(resourceId(variables('resourceGroupName'),'Microsoft.Logic/workflows', variables('logicAppName')),'/triggers/When_a_resource_event_occurs'), '2017-07-01').value]"  
              }
          },
          "filter": {
              "subjectBeginsWith": "/blobServices/default/containers/edievents/blobs/supplierlogin/...",
              "subjectEndsWith": ".json",    
              "includedEventTypes": [
                  "Microsoft.Storage.BlobCreated"
              ]
          },
          "eventDeliverySchema": "EventGridSchema",
          "retryPolicy": {
              "maxDeliveryAttempts": 30,
              "eventTimeToLiveInMinutes": 1440
          }    
      }
    } 

Some notes:

• A System Topic was used if you are listening to Azure services, the list of Azure services that support system topics can be found here. For events originating from your custom applications, use the custom topics.
• Utilize the listCallbackUrl function to retrieve the trigger URL of the Logic App subscribed to this event.
• Update the filter section to apply some filtering criteria. This way, Event Grid will only broadcast select events based on the originating Blob container and specific file types.

Conclusion

If you haven’t yet incorporated Azure Event Grid in your toolkit, I highly recommend giving it a go, as it offers several notable benefits such as:

• Real-time Responsiveness: Events trigger actions immediately when they occur. This real-time responsiveness is invaluable in scenarios where quick reactions are needed, such as processing user registration as we saw in the solution above, and updates to business-critical information.
• Scalability: Event-driven systems can easily scale to handle varying workloads. Azure services like Azure Functions and Logic Apps can automatically scale up or down based on the event load, ensuring optimal resource utilization.
• Cost Efficiency: Event-driven systems often result in cost savings. Resources are only utilized when events are generated, reducing the need for continuous polling and idle resource consumption.
• Loose Coupling: Events decouple different components of a system. This loose coupling means that changes in one component don’t necessitate changes in others, leading to more maintainable and flexible systems.