E_RUNTIME_SYSTEM_ERROR on Azure Data Lake Analytics

1
votes

I am doing some benchmark on Azure Data Lake Analytics. After running a query during an execution phase Combine_partition, I got error:

ERROR E_RUNTIME_SYSTEM_ERROR MESSAGE A system error has been reported [VirtualAlloc returned 0]

It looks like out of memory exceptions. Any idea?

Error

2
azureazure-data-lakeu-sql
There will be a larger error report with more detail on the Errors tab of the main job result / report. Can you get that? Also what code are you running? Please post it. – wBob

2 Answers

1
votes

This looks like your join may have received too much data and ran out of the 6GB memory that a vertex get.

The suggestion is to distribute the data or operation better by either having a better distribution strategy for that join (e.g., avoiding data skew where one bucket is too large), use a different operation that provides you with better scale, or add some data hints that "lures" the optimizer to make better partitioning decisions.

0
votes

Issue solved.

Vertex ran out of the memory, so I changed my query, eliminating GROUP BY using WINDOWING functions on same data and partitions.

Tnx for help.