Azure Monitoring - no data in storage account

Question

I thought that (after following the steps in http://azure.microsoft.com/en-us/documentation/articles/cloud-services-how-to-monitor/) that this would persist the data from "Verbose" monitoring in a corresponding Azure storage account (which I created as per the article).

However, I don't see any data in storage. There are now 3 containers in the storage account ("mam", "mdsconfig", "wad-control-container") and I expected the last one to contain monitoring data - but there's only a single XML file (about 3.6K).

Could someone please advise? I must have missed a step or misunderstood something! This will contain all the cpu/network/sql stats, right?

EDIT: This is my diagnostics.wadcfg file:

<?xml version="1.0" encoding="utf-8"?>
<DiagnosticMonitorConfiguration configurationChangePollInterval="PT1M" overallQuotaInMB="4096" xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">
  <DiagnosticInfrastructureLogs />
  <Directories>
    <IISLogs container="wad-iis-logfiles" />
    <CrashDumps container="wad-crash-dumps" />
  </Directories>
  <Logs bufferQuotaInMB="1024" scheduledTransferPeriod="PT5M" scheduledTransferLogLevelFilter="Verbose" />
  <PerformanceCounters scheduledTransferPeriod="PT5M">
    <PerformanceCounterConfiguration counterSpecifier="\Processor(_Total)\% Processor Time" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\Memory\Available MBytes" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\Memory\Committed Bytes" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\LogicalDisk(_Total)\Disk Read Bytes/sec" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\Web Service(_Total)\ISAPI Extension Requests/sec" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\Web Service(_Total)\Bytes Total/Sec" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\ASP.NET Applications(__Total__)\Requests/Sec" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\ASP.NET Applications(__Total__)\Errors Total/Sec" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\ASP.NET\Requests Queued" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\ASP.NET\Requests Rejected" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\ASP.NET\Applications Running" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\Process(WaWorkerHost)\% Processor Time" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\Process(WaWebHost)\% Processor Time" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\Process(IISExpress)\% Processor Time" sampleRate="PT1S" />
    <PerformanceCounterConfiguration counterSpecifier="\Process(w3wp)\% Processor Time" sampleRate="PT1S" />
  </PerformanceCounters>
  <WindowsEventLog bufferQuotaInMB="1024" scheduledTransferPeriod="PT5M" scheduledTransferLogLevelFilter="Verbose">
    <DataSource name="Application!*" />
    <DataSource name="System!*" />
  </WindowsEventLog>
</DiagnosticMonitorConfiguration>

ServiceDefinition.csdef:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="CloudTest" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-03.2.0">
  <WebRole name="MyRole" vmsize="Small">
    <Sites>
      <Site name="Web">
        <Bindings>
          <Binding name="Endpoint1" endpointName="Endpoint1" />
        </Bindings>
      </Site>
    </Sites>
    <Endpoints>
      <InputEndpoint name="Endpoint1" protocol="http" port="80" />
    </Endpoints>
    <LocalResources>
      <LocalStorage name="MainDrive" cleanOnRoleRecycle="false" sizeInMB="1000" />
    </LocalResources>
    <Imports>
      <Import moduleName="Diagnostics" />      
    </Imports>
    <ConfigurationSettings>
      <Setting name="SomeSetting" />
    </ConfigurationSettings>
    <Certificates>
      <Certificate name="HubWebService client" storeLocation="LocalMachine" storeName="My" />
      <Certificate name="RootCATest_Uat2" storeLocation="LocalMachine" storeName="My" />
    </Certificates>
  </WebRole>
  <WorkerRole name="SendEmails" vmsize="Small">
    <Imports>
      <Import moduleName="Diagnostics" />     
    </Imports>
    <ConfigurationSettings>
      <Setting name="SomeSetting" />
    </ConfigurationSettings>
    <Certificates>
      <Certificate name="HubWebService client" storeLocation="LocalMachine" storeName="My" />
      <Certificate name="RootCATest_Uat2" storeLocation="LocalMachine" storeName="My" />
    </Certificates>
  </WorkerRole>
</ServiceDefinition>

Answer 1

Stupid question, but are you looking at Azure tables at all? Most of diagnostics is there. It does take a few good minutes to show up, after the first time a properly configured diagnostics schema is deployed

Answer 2

Before it starts outputting data to storage, you have to enable them in your application as well -> http://azure.microsoft.com/en-us/documentation/articles/cloud-services-dotnet-diagnostics/

The important piece in that configuration is the scheduledTransferPeriod attribute which sets how often data are pushed from your roles to Storage. If the transfer is not set, you won't get any information out of the role.

You can also programmatically trigger it if you like (on demand diagnostics transfer).