Azure NSG flow logs are a feature of Azure Network Security Group (NSG) that allows administrators to track and monitor network traffic flowing through their Azure virtual network.
The flow logs provide detailed information on the source and destination of the traffic, as well as the protocol and port being used. This information can be used to identify trends and patterns in network usage, as well as to detect and troubleshoot potential security threats.
Azure NSG flow logs provide a valuable tool for administrators to maintain visibility and control over their Azure network environment.
To set the NSG flow logs to be sent to Log workspace we can use Traffic Analytics.
In this post we will be going through enabling NSG Flow Logs, enabling Traffic Analytics and reviewing the logs for allowed and denied traffic using Azure Log Analytics Workspace.
There will be a cost for using the Azure Storage, workspace and Traffic Analytics so confirm these before proceeding as the more data sent the more it will cost.
When creating a new deployment in Azure it is good security practice to restrict access between subnets to the required ports only. This can sometimes be a bit difficulty if the application communication is not fully documented.
This is where NSG Flow can be used as we can use this to review the traffic between the subnets going over the NSG’s. There are some prerequisite for this
- Storage account in the same region as the NSG
- Log Analytic workspace
- Network Watcher (this should be created automatically once a vNet is created)
- Network Security group
- VM running in two network and have an NSG assigned
Once all the above are ready we can start to enabled the logging. NSG flow can be either enabled from NSG directly or through Network Watcher. We will be doing this in Network Watcher.
First step is to go to Network Watcher > NSG Flow
Select the required Subscription, the required NSG’s, storage account and retention (0 on retention means the logs are kept forever) since this is a test environment I will only keep the logs for 5 days.
On configuration select version 2 and enabled traffic analytics. On traffic analytics set the process interval to either 1 hour or 10 mins, select the subscription and workspace.
Apply tags if in use, then review and create.
The deployment should only take a minute or so and then the NSG flow should show as succeeded.
Once enabled it can take little bit for data to start showing. We can check that the container has been create on the storage account.
Open the storage account and go to Containers there should be a new insight-logs container.
If we navigate down the folder structure there will be a JSON file that has all the NSG access requests, we could use the JSON file it self but it pretty difficult to read.
To check that data is going to the workspace, go to Log Analytics Workspace. Select the workspace we used earlier.
We will use the AzureNetworkAnalytics_CL table and flowlog to view the data. I used the below Learn article to help understand the table / fields and create the queries.
https://learn.microsoft.com/en-us/azure/network-watcher/traffic-analytics-schema
We can run the below KQL query this will return
AzureNetworkAnalytics_CL
| where SubType_s == "FlowLog"
| project FlowType_s, FlowEndTime_t, SrcIP_s, DestIP_s, DestPort_d, L4Protocol_s, L7Protocol_s
Now that we have confirmed data is being shipped to the log workspace, we can start to create the KQL query that will show all traffic between the specific IP of the servers.
The below query will return inbound and outbound request for specified source and destination IP
AzureNetworkAnalytics_CL
| where SubType_s == "FlowLog"
| extend Direction = case(FlowDirection_s == 'O', 'Outbound', FlowDirection_s == 'I','Inbound', FlowDirection_s)
| extend Access = case(FlowStatus_s == 'A', 'Allowed', FlowStatus_s == 'D', 'Denied', FlowStatus_s )
| extend Protocol = case(L4Protocol_s == 'T', "TCP", L4Protocol_s == 'U', "UDP", L4Protocol_s)
| project TimeGenerated, Direction, Access, SrcIP_s, DestIP_s, DestPort_d, Protocol, L7Protocol_s, NSGRule_s
| where SrcIP_s contains "Source IP" and DestIP_s contains "Destination IP"
We can now see all the traffic that is going from one server two the other, we can use this to review and allow the required traffic.
To export we can then export the results and use that to review the required ports.
The last step will be to update the rules on the NSG to only allow the required traffic.
TheSleepyAdmins 