If I send a TensorFlow training job to a SageMaker instance, what is the typical way to view training progress? Can I access TensorBoard for this launched EC2 instance? Is there some other alternative? What I'm looking for specifically are things like graphs of current training epoch and mAP.
1 Answers
3
votes
you can now specify metrics(metricName, Regex) that you want to track by using AWS management console or Amazon SageMaker Python SDK APIs. After the model training starts, Amazon SageMaker will automatically monitor and stream the specified metrics in real time to the Amazon CloudWatch console for visualizing time-series curves.
Ref: https://docs.aws.amazon.com/sagemaker/latest/dg/API_MetricDefinition.html
