Userid in azure datalake (gen2) log

Question

I am getting my head around the logs in Azure, and particularly the logs I get from ADSL Gen2 data lake. It is really true that I can't get the login / userid for the changes to the data lake?

I have these fields

TenantId TimeGenerated [UTC] AccountName Location Protocol OperationName AuthenticationType StatusCode StatusText DurationMs ServerLatencyMs Uri CallerIpAddress CorrelationId SchemaVersion OperationVersion AuthenticationHash UserAgentHeader ClientRequestId Etag ServiceType RequestHeaderSize ResponseHeaderSize LastModifiedTime [UTC] Category TlsVersion SourceSystem Type _ResourceId

AuthenticationType is just called AccountKey

CallerIpAddress is the IP address of the user

But the userID like [email protected] or similar is what I am looking for. So how do I include fields that describe how the operation was authenticated

To be clear (after some input from KarthikBhyresh-MT)

I have my own ADSL that I am playing around in In azure portal under ADSL > Diagnostic setting (classic) I have enabled Blob logging version 2.0, Read/Write/Delete/Delete data (just as suggested)

I then use Microsoft Azure Storage Explore to upload some file, deleting some of the files again, and generally make something to log

In azure portal under ADSL > Logs (preview) I read the StorageBlobLogs

If I run the simplest query where RequesterUpn is not empty I get my username for an even where AuthenticationType is OAuth. That is the login to the service.

But when I find the OperationName: DeleteFile I have no information of who did it I have the autenticationHash(1) and CallerIpAdress(2) and I could look up the IpAdress from the OAuth log event to put Delete action to a name

UserPrincipalName [string] - User principal name used in bearer authorization. Example: [email protected] — KarthikBhyresh-MT
I found the list of Fields that describe how the operation was authenticated that includes requester / upn The User Principal Name (UPN) of requestor. For example: [email protected]. but they are all empty in the log — Erik Warming
it would be empty if the request was unsucceful or anonymous or a SAS request — KarthikBhyresh-MT

KarthikBhyresh-MT KarthikBhyresh-MT · Accepted Answer · 2021-10-30T05:42:08

If you have turned on the below specifics in ADLS account

Optionally route if needed

I can see from the logs in the storage about ADLS now.

A sample record containing upn. You can find it at identity.upn where OAuth is the authorization used, which is supported Version 2.0 of Storage Analytics logging.

{
    "time": "2021-10-30T05:12:17.3923930Z",
    "resourceId": "/subscriptions/<Subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Storage/storageAccounts/<storage-account-name>/blobServices/default",
    "category": "StorageRead",
    "operationName": "GetBlobServiceProperties",
    "operationVersion": "2020-08-04",
    "schemaVersion": "1.0",
    "statusCode": 200,
    "statusText": "Success",
    "durationMs": 712,
    "callerIpAddress": "<ip-address><port>",
    "correlationId": "fced83b0-xxxx-xxxx-xxxx-cd769c000000",
    "identity": {
        "type": "OAuth",
        "tokenHash": "E098F823BC1BE1D9AC73F22F82xxxxxxxxxxxxxxxxxxxx5537E013A5E6BDF71E",
        "requester": {
            "appId": "691458b9-xxxx-xxxx-xxxx-ed83a7f1b41c",
            "audience": "https://storage.azure.com/",
            "objectId": "b1c5060f-xxxx-xxxx-xxxx-31cce61160f4",
            "tenantId": "72f988bf-xxxx-xxxx-xxxx-2d7cd011db47",
            "tokenIssuer": "https://sts.windows.net/72f988bf-xxxx-xxxx-xxxx-2d7cd011db47/",
            "upn": "[email protected]"
        }
    },
    "location": "East US",
    "properties": {
        "accountName": "<storage-account-name>",
        "userAgentHeader": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36 Edg/95.0.1020.30",
        "serviceType": "blob",
        "objectKey": "/<storage-account-name>",
        "serverLatencyMs": 711,
        "requestHeaderSize": 2713,
        "responseHeaderSize": 282,
        "responseBodySize": 802,
        "tlsVersion": "TLS 1.2"
    },
    "uri": "https://<storage-account-name>.blob.core.windows.net:443/?restype=service&comp=properties&_=1635xxxx35961",
    "protocol": "HTTPS",
    "resourceType": "Microsoft.Storage/storageAccounts/blobServices"
}

Refer official MS Storage Analytics log format doc for more details.

Example record for "category": "StorageWrite"

Userid in azure datalake (gen2) log

2 Answers