In this post, I demonstrate how to automatically purge content from an Azure Content Delivery Network (CDN) using Event Grid and Azure Automation. I am writing this post with two audiences in mind. First, if you have a CDN and need to purge old content every time content is updated or removed from a source, this post will show you how. However, if you don’t have a CDN but are interested in understanding how to configured Event Grid to trigger automation jobs in an Azure, this post is also for you.
Background
A CDN distributes and caches content across the globe, providing lower latency delivery and reducing global bandwidth usage. It may be necessary to rapidly refresh the cache when new content is added to the source. This post will demonstrate how to do that using Event Grid and Azure Automation.
If you would like to follow along at home, you will need an Azure CDN Configured with an Azure Storage Account V2 as the source. I have a post demonstrating how to set that up here. You will also need an Azure Automation account. If you don’t have an Azure Automation account, I have a whole playlist on setting one up here. The Azure Automation runbook used in this example can be found here.
Flow Overview
The basic flow of operations is as outlined below:
- Content added (or removed) from the Storage Account blob source.
- Add or Delete action on the blob storage triggers Event Grid.
- Event Grid runs a web hook for the Azure Automation runbook, passing JSON data including file path.
- The Automation runbook parses JSON and runs a purge job against the path.
Azure Automation
The web hook is required to set up the Event Grid action, so let’s start there. Start by going to your Azure Automation account and create a runbook.
Create Purge Runbook
For this example, I’m only going to publish a single write-output command as a placeholder. Remember, for now, I only need a web hook for the runbook. We will get to the working runbook shortly. Save and publish the runbook.
Now that we have a published runbook we can create a web hook to trigger it. From the runbook, go to the Add web hook at the top of the page.
This will take you to the first of two steps to create a web hook. Give the new web hook a name, make sure it’s set to enabled and set an expiration date. Note the date, the web hook will stop working after that date. Lastly, copy and paste the URL someplace secure. You will not be able to retrieve this URL once the web hook has been created. If you lose it, a new one will need to be created.
Click OK to move to the next step. Specify the run environment and parameters in the Modify run settings section. If the Runbook required parameters or needs to run on a hybrid worker, that could be set here. Neither applies to this example, so click OK and Create to add the web hook.
The CDN module is required to run the purge command. Add that by going into Modules Gallery in the automation account. Search for CDN. Add the Az.CDN module.
If you are using the default AzureRM commands in the Automation Account, add the AzureRM.CDN module instead. Also change the Unpublish-AzCDNEndpointContent command in the script to the AzureRM command: Unpublish-AzureRMCDNEndpointContent.
Event Grid
Now that we have a web hook, let’s move onto setting up Event Grid. Start by going to the Storage Account used as the CDN source. Look for Events on the sidebar of the Storage Account.
This will open the Events, Get Started page. Select the More Options option.
You will see multiple options to choose from. Select the Web Hook option.
Give the Event Subscription a name. Leave the Event Schema and Topic Details as default. Notice that the Event Types have the option of Blob Created or Blob Deleted. Leave both selected. Under Endpoint Details, select Web Hook.
Next, under Web Hook, click on Select and endpoint. This will bring up the Subscriber Endpoint. Enter the Web Hook URL created in the previous step. Click Confirm Selection and Create to add the event.
After the event is created, go to Event Subscriptions to verify. Notice you can watch the activity from the Event Subscriptions page. It is also possible to modify the event by clicking on the event name.
Test the Event
Now that the web hook is in place, let’s test. The event will trigger when new blobs are uploaded or deleted from the container. To test, simply upload or remove content from the container. The example below uses Storage Explorer to download and then upload index.html. This adds content and triggers the event.
Events now show activity as shown below.
Go to the Azure Automation Runbook. Under Jobs, you will see a job completed. This is the job Event Grid triggered.
Click on the Job and go to Output. You will see that the job output, in this example a simple message.
The output is not very useful. More important is the input. Notice the input WEBHOOKDATA is JSON Code with details about the event. Copy the data into an editor, such as Visual Studio Code and save it as a .json file. I used the Prettier VS Code extension to format the content as JSON.
Under RequestBody, there are a couple of values that give path information. This information is used below to run a purge against a specific file each time it’s uploaded or deleted. This example will use the value assigned to “url” to build a path for the purge operation.
Azure Automation Runbook
Almost there! The Event Trigger is configured to fire a runbook. The runbook is configured and we have the parameter needed to run the purge command. The last step is to build a runbook.
Below I step through each section of the runbook. If you are unfamiliar with creating runbooks, I have the YouTube Playlist for you! The complete runbook is available on my GitHub site here .
The first section in the script is the comments. After that is the parameter section. This adds the JSON input to the $Web hookData variable for use later in identifying the file path.
After the parameter section is the authentication section. This authenticates the runbook with the Azure Automation Run As account, giving it permissions to run the purge command.
The next section assigns the CDN Profile name, Resource Group and Endpoint Name to variables. Update these fields with values from your environment. I also set the default error action to a variable, this is used for error handling.
After that, I manipulate the JSON data to get the correct file path. The data I need is in the RequestBody section of the JSON, the first step is to assign that data to a new variable, $requestbody
Once that’s assigned, I assign the value of “url” located in data to the $filePath variable. The value of url includes the full FQDN and file path. I don’t need the FQDN for the purge command, only the file path. The replace command is used to replace the FQDN with nothing, removing the part of the url I don’t need. For example, if this is the full url:
https://cdntestcir.blob.core.windows.net/source/index.html
replacing “https://cdntestcir.blob.core.windows.net” with “nothing” will leave “/source/index.html”, the path needed for the purge command. Update the url value from your environment.
The full command looks like below.
The try, catch blocks are used for error handling. Errors encountered as the runbook runs are written to the Error stream. For more information on the Try, Catch, Finally, blocks, see “The Big Book of PowerShell Error Handling” available from PowerShell.org https://leanpub.com/u/devopscollective
Last, the unpublish-AzCDNEndpointContent command is used to purge the file. This command takes a minute or two to run.
Final Testing
Test by updating or deleting a file from the CDN Source Storage Account. This will trigger Event Grid to send the web hook and cause the Runbook to run. The Azure Automation runbook will report the completed job once finished.
The Activity Log on the CDN Endpoint will show a successful purge. Clicking on the drop-down arrow will show details of the job.
Summary
Here we are at the end. This post demonstrated how to use Event Grid and Azure Automation to automate the process of purging content from a CDN when content is deleted from the source or new content is added. The CDN use case is fairly specific, but Event Grid is integrated into many Azure services and the same workflow could be applied to a multitude of services to automate within Azure.
