Month: July 2023

Getting Started with KQL Part 1: Azure Log Workspace and diagnostic settings

Kusto Query Language (KQL), is a query language developed by Microsoft for querying and analyzing data. KQL is specifically designed for working with structured, semi-structured, and unstructured data, and it offers a simple and intuitive syntax for expressing complex queries.

KQL is used in log analytics, security monitoring, and business intelligence. It allows users to extract insights from vast amounts of data by using a combination of filtering, aggregating, and transforming operations. With its rich set of operators, functions, and control flow statements, KQL enables users to perform advanced analytics and create sophisticated queries.

In the next few blog post’s we will be going through how to send data to a log workspace and creating KQL queries to show how we can visualize and gather data.

I have wanted to do more with KQL and I am using this series to improve my own KQL and hopefully it will be of use to other if you are just starting out.

First step is we need to create a Azure Log workspace.

Go to Log Analytics workspace blade in Azure and click create.

  • Select a subscription
  • Create or select and existing resource group
  • Give the workspace a name
  • Set the region

Select tags if needed, review and create.

It will take a few minutes to finish deploying.

Once we have the log workspace created we can start to send data that we can then query.

First we will be sending storage account diagnostic data.

To enabled to go the storage account and select diagnostic settings.

Enabled the diagnostic for the required storage type, I am going to enable for blob and file.

Click on the storage type and click add diagnostic settings.

Select the logs to send and the log analytics workspace.

After enabling for both file and blob the diagnostic status should now show as enabled.

We will can generate some data by creating and delete some blob and azure file share.

Once we have some data can start to write our KQL query. First we can run against the full table by using StorageBlobLogs. I used the Azure monitor reference to find the correct table to use.

https://learn.microsoft.com/en-us/azure/azure-monitor/reference/

If we want to select specify rows and filter the data that is returned we can use the where operator. In below example I am select only the storage write results.

StorageBlobLogs
| where Category == 'StorageWrite'

In the next post we will go through using different operators like project, summarize, rendering…

Azure Key Vault: Access With PowerShell

Azure Key Vault is a cloud-based service provided by Microsoft Azure that allows users to securely store and manage keys, secrets, and certificates used in their applications and services.

It acts as a repository for sensitive information and provides a secure way to access and manage sensitive data. Key Vault offers features such as key and secret rotation, access policies, and auditing capabilities, ensuring that sensitive data remains protected at all times.

It integrates with other Azure services and provides encryption and access control, making it a reliable solution for safeguarding critical data.

In this post we will be going through configuring Azure Key Vault, adding some secrets and calling these secrets using PowerShell.

Before using Azure Key Vault, there are a few prerequisites that need to be in place:

  1. Azure Subscription: You will need an active Azure subscription to create and manage Azure Key Vaults.
  2. Resource Group: Create a resource group in Azure
  3. Access Control: Ensure that you have the necessary permissions and role-based access control (RBAC) rights to create and manage Azure Key Vaults. The required roles typically include Owner, Contributor, or Key Vault Contributor.
  4. Network Configuration: Configure your virtual network and firewall rules to allow access to the Azure Key Vault. You can limit access to specific IP addresses or Azure services based on your security requirements.

I will be using a pre-existing resource group and VNET and we wont be covering those in this post.

Azure Key Vault and secrets can be created either using AZ CLI, AZ PowerShell and the Azure portal. In this post we will be using the Azure portal and will create a new secret using AZ PowerShell.

First step is to create a new key vault in the Azure admin portal search for key vault and open the blade.

Click create

Select the resource group

  • give the vault a name
  • set the region
  • set the pricing tier
  • set soft delete

Set the permission model and resource access.

I will be leaving public access open but in production I would limit this and create a private endpoint.

Review the setting and create the key vault.

The deployment will take a minute to complete.

Before we can being using the key vault we need to give permission.

Go to the key vault, select IAM and click add.

Next select the role to assign. In this case I am using Key Vault Administator.

Select the member, I am using a group, this could also be a managed Identity incase we need to allow Azure function or automation account to connect.

Review and assign the permissions.

Now that we have the key vault and permission set we can add some secrets.

Go to objects and secrets. Click on generate/import

Give the secret a name and a value, we can also set activation / expiration dates and tags.

Click create to add the secret.

Now we should see the secret in the secrets blade.

We can view the value directly in the Azure console be clicking on the secret and view the secret value.

The last step is to test that we can call the value using PowerShell. To run these command we first have to install the AZ PowerShell module.

Connect using AZ PowerShell 

Connect-AzAccount

If there are multiple subscriptions we need to set using Set-AzContext

Set-AzContext -Subscription "Subscription name"

Use the Get-AzKeyVaultSecret command to to view the secret

Get-AzKeyVaultSecret -VaultName "vault name"

To retrieve the value as plain text use 

Get-AzKeyVaultSecret -VaultName "vault name" -Name "secretvalue" -AsPlainText

To create a new secret using AZ PowerShell

$setsecretvalue = ConvertTo-SecureString "This is another secret value" -AsPlainText

Set-AzKeyVaultSecret -VaultName "vault name" -Name "secret name" -SecretValue $setsecretvalue

Now we can call the secret to view that the value has been set.

This was quick run through of calling secrets using Azure Key Vault and PowerShell, this can be use full when a scripts that need to authenticate and can be used to remove any hard coded passwords or strings from scripts.