How to limit number of entities copied from azure table storage by AzCopy?

Question

I'm trying to copy sample of azure table storage entities to my local file. The source is quite large with millions of records. How can i limit data transfer and take only first 1K entities?

The straightforward command would copy all of them (AzCopy 7.3.1):

AzCopy.exe /Source: ...table.core.windows.net/profile /SourceKey:..somekey /Dest:C:\Temp /Manifest:qa-profile /SplitSize:128

Peter Pan Peter Pan · Accepted Answer · 2019-04-18T06:29:46

Just per my experience, I think you can not copy the limit number of entities from Azure Table Storage simply by AzCopy.

However, you can try to use PowerShell with Azure Module to do it. Here is my steps and sample code which you can refer to.

To run PowerShell as administrator to install Azure Module via the command Install-Module -Name Az -AllowClobber -Scope CurrentUser, that you can refer to the offical document Install the Azure PowerShell module to know more details.
Follow the section Sign in to Azure of the offical tutorial Perform Azure Table storage operations with Azure PowerShell to sign in to Azure with your account on PowerShell.

Here is my sample code you can try to run on your local machine after done the step 2.

$storageAccountName = "<your storage account name>"
$resourceGroup = "<the resource group name of your storage>"
$location = "<the location of your storage>"
$storageAccount = Get-AzStorageAccount -Name $storageAccountName -ResourceGroupName $resourceGroup
$ctx = $storageAccount.Context

$tableName = "<your table name>"
$cloudTable = (Get-AzStorageTable –Name $tableName –Context $ctx).CloudTable
$i = 0 ; $n = 1000 ; Get-AzTableRow -table $cloudTable | ForEach-Object {ConvertTo-Json $_; $i++; If($i -eq $n) {break}} > results-1000.json

The result content is like as below.

{
    "ProductName":  "AAAAAAA",
    "PartitionKey":  "A",
    "RowKey":  "1",
    "TableTimestamp":  "\/Date(1542619135228)\/"
}
{
    "ProductName":  "BBBBBBB",
    "PartitionKey":  "B",
    "RowKey":  "2",
    "TableTimestamp":  "\/Date(1542619145997)\/"
}

Hope it helps.

How to limit number of entities copied from azure table storage by AzCopy?

1 Answers