Showing posts with label Azure Blob. Show all posts
Showing posts with label Azure Blob. Show all posts

Thursday, February 23, 2023

Azure storage event trigger with user managed identity arm template


 



Recently I was working on one of the event-based solution where I had to trigger a logic app when a file is put on the storage account. It all looked fine when I was doing it manually on the portal but when I had to promote it to higher environment through the pipeline, I faced one issue. The issue was basically to find the dynamic URL of the Logic App. To get dynamic URL of the HTTP logic app is fine and you can get that with the below piece of code.


Get HTTP Logic APP URL

"url": "[substring(listCallbackUrl(resourceId(parameters('ResourceGroup'), 'Microsoft.Logic/workflows/triggers', parameters('LogicApps'), 'manual'), '2016-10-01').basePath,0,add(10,indexOf(listCallbackUrl(resourceId(parameters('ResourceGroup'), 'Microsoft.Logic/workflows/triggers', parameters('LogicApps'), 'manual'), '2016-10-01').basePath,'/triggers/')))]"



But I was struggling to get below URL through ARM template.







I have created a template which creates user-based managed identity Logic APP and will be triggered as soon as I put a file into the storage account.

So, before we run this ARM template, lets create a user managed identity and the storage account.

I have created a resource group and 2 resources inside it.

·       Resource Group            : event-trigger-logicapp

·       User Managed Identity   : event-test-identity

·       Storage Account           : eventtriggerstorage01





Important: Give contributor role to user managed identity to subscription level.

 

Now create a new "Custom Template Deployment" resource in event-trigger-logicapp resource group.






ARM Template: Make sure you update Logic App Name, Storage Account, Resource Group, Subscription ID and User Managed Identity values as highlighted.

{

    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

    "contentVersion": "1.0.0.0",

    "parameters": {

        "workflows_EventGridTest_name": {

            "defaultValue": "storage-event-logicapp",

            "type": "String"

        },

        "connections_azureeventgrid_name": {

            "defaultValue": "azureeventgrid",

            "type": "String"

        },

        "storageAccounts_storageaccountlogica_externalid": {

            "defaultValue": "/subscriptions/7bbce658-682f-417d-88f6-810b6fb20f4d/resourceGroups/event-trigger-logicapp/providers/Microsoft.Storage/storageAccounts/eventtriggerstorage01",

            "type": "String"

        },

        "userAssignedIdentities_ManagedContributor_externalid": {

            "defaultValue": "/subscriptions/7bbce658-682f-417d-88f6-810b6fb20f4d/resourceGroups/event-trigger-logicapp/providers/Microsoft.ManagedIdentity/userAssignedIdentities/event-test-identity",

            "type": "String"

        }

    },

    "variables": {},

    "resources": [

        {

            "type": "Microsoft.Web/connections",

            "apiVersion": "2016-06-01",

            "name": "[parameters('connections_azureeventgrid_name')]",

            "location": "australiaeast",

            "kind": "V1",

            "properties": {

                "displayName": "test2",

                "statuses": [

                    {

                        "status": "Ready"

                    }

                ],

                "customParameterValues": {},

                "createdTime": "2023-02-01T03:49:55.8553087Z",

                "changedTime": "2023-02-01T03:49:55.8553087Z",

                "api": {

                    "name": "azureeventgrid",

                    "displayName": "Azure Event Grid",

                    "description": "Azure Event Grid is an eventing backplane that enables event based programing with pub/sub semantics and reliable distribution & delivery for all services in Azure as well as third parties.",

                    "iconUri": "https://connectoricons-prod.azureedge.net/releases/v1.0.1538/1.0.1538.2619/azureeventgrid/icon.png",

                    "brandColor": "#0072c6",

                    "id": "/subscriptions/7bbce658-682f-417d-88f6-810b6fb20f4d/providers/Microsoft.Web/locations/australiaeast/managedApis/azureeventgrid",

                    "type": "Microsoft.Web/locations/managedApis"

                },

                "parameterValueType": "Alternative",

                "testLinks": []

            }

        },

        {

            "type": "Microsoft.Logic/workflows",

            "apiVersion": "2017-07-01",

            "name": "[parameters('workflows_EventGridTest_name')]",

            "location": "australiaeast",

            "dependsOn": [

                "[resourceId('Microsoft.Web/connections', parameters('connections_azureeventgrid_name'))]"

            ],

            "identity": {

                "type": "UserAssigned",

                "userAssignedIdentities": {

                    "/subscriptions/7bbce658-682f-417d-88f6-810b6fb20f4d/resourceGroups/event-trigger-logicapp/providers/Microsoft.ManagedIdentity/userAssignedIdentities/event-test-identity": {}

                }

            },

            "properties": {

                "state": "Enabled",

                "definition": {

                    "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",

                    "contentVersion": "1.0.0.0",

                    "parameters": {

                        "$connections": {

                            "defaultValue": {},

                            "type": "Object"

                        }

                    },

                    "triggers": {

                        "When_a_resource_event_occurs": {

                            "splitOn": "@triggerBody()",

                            "type": "ApiConnectionWebhook",

                            "inputs": {

                                "body": {

                                    "properties": {

                                        "destination": {

                                            "endpointType": "webhook",

                                            "properties": {

                                                "endpointUrl": "@{listCallbackUrl()}"

                                            }

                                        },

                                        "filter": {

                                            "includedEventTypes": [

                                                "Microsoft.Storage.BlobCreated"

                                            ]

                                        },

                                        "topic": "[parameters('storageAccounts_storageaccountlogica_externalid')]"

                                    }

                                },

                                "host": {

                                    "connection": {

                                        "name": "@parameters('$connections')['azureeventgrid_1']['connectionId']"

                                    }

                                },

                                "path": "/subscriptions/@{encodeURIComponent('7bbce658-682f-417d-88f6-810b6fb20f4d')}/providers/@{encodeURIComponent('Microsoft.Storage.StorageAccounts')}/resource/eventSubscriptions",

                                "queries": {

                                    "x-ms-api-version": "2017-06-15-preview"

                                }

                            }

                        }

                    },

                    "actions": {

                        "Compose": {

                            "runAfter": {},

                            "type": "Compose",

                            "inputs": 111

                        }

                    },

                    "outputs": {}

                },

                "parameters": {

                    "$connections": {

                        "value": {

                            "azureeventgrid_1": {

                                "connectionId": "[resourceId('Microsoft.Web/connections', parameters('connections_azureeventgrid_name'))]",

                                "connectionName": "azureeventgrid",

                                "connectionProperties": {

                                    "authentication": {

                                        "identity": "[parameters('userAssignedIdentities_ManagedContributor_externalid')]",

                                        "type": "ManagedServiceIdentity"

                                    }

                                },

                                "id": "/subscriptions/7bbce658-682f-417d-88f6-810b6fb20f4d/providers/Microsoft.Web/locations/australiaeast/managedApis/azureeventgrid"

                            }

                        }

                    }

                }

            }

        }

    ]

}


After a successful run, you should be below resources are created out of template deployment.






Let’s create a “toprocess” name container and put a file there.






You will notice that logic app has triggered.







If you look into the “Event Grid System Topic” a URL gets created automatically. This URL gets created internally so as per my understanding you can't get this URL through ARM similar to HTTP one. (I could be wrong though 😊)










Thursday, August 5, 2021

Move, Copy and Delete blob with Azure Logic App

 

Recently I have been working with the blob storage where in many cases I had to move, copy, and delete the blob from the storage after performing some actions on those blobs. In this blog I am going to talk about how to perform copy, move, delete operation on Azure blob storage from the Logic APP.

 

This is quite simple to perform these operations as Logic App provides these inbuild function so all we need to do is to call relevant function for a particular requirement.

 

Requirement: In my requirement I had to read the file content from the boob storage and after reading it I had to move it to different blob storage and in some scenario, I had to delete it permanently from the blob storage. So, let’s see how we can achieve it.

 

Create A Logic APP:

This first step is obviously creating a Logic App. If you want to see how to create from scratch, then please follow my previous blog.

 


You can use any trigger as this in totally independent functionality.

 

Create a Connection:

After creating a logic app and trigger, you have to make a connection to the Azure blob (if you dint have already). You need to provide your credentials to create connection.

 



Azure Blob inbuilt function:

When you choose “Azure Blob” as “+ New Step” and after creating a connection with blob, you can see multiple option Azure Logic App provides for use to use. You can see in below screenshot where you can choose Create Blob, Copy blob, delete blob etc.

 

 


 

Copy Blob:

Copying blob functionality can be achieved by specifying Source URL and Destination URL. In below example I have given dynamic values as this is the common pattern, we use otherwise you can use a static one.

 


In below screenshot you can see how the dynamic URL is generated after running the Logic APP. This is something we can also provide as static value.

 


Delete Blob:

This is also one of the inbuilt functions provided by the logic app to play with Azure blob. Simply use the Delete blob from the option and provide the blob path which you want to delete. Below is the dynamic one.

 


Dynamic one will be changed to below URL if we run this app.

 


Move Blob from one to another folder:

There is no functionality like that but what we can do to achieve it to simply copy first from source to destination folder and then perform delete operation in the source folder to delete the blob