Learning Curves for Multi-Class Logistic Regression

1
votes

I have written a multi-class classifier using logistic regression that is trained using a one vs all approach. I want to plot the learning curve for the trained classifier.

Should the learning curve be plotted on a class by class basis or should it be a single plot for the classifier as a whole? Does it make a difference?

To clarify, the learning curve is a plot of the training & cross validation/test set error/cost vs training set size. This plot should allow you to see if increases the training set size improves performance. More generally, the learning curve allows you to identify whether your algorithm suffers from a bias (under fitting) or variance (over fitting) problem.

Some details regarding my code:

  • analyzes MNIST handwritten digit images
  • predicts digit (0-9) in image
  • based on Andrew Ng's Coursera class on machine learning
1
optimizationmachine-learninglogistic-regressionsupervised-learning
What exactly do you mean by "learning curve"? Just a plot of the learning progress? Why should this influence the predictions of the classifier? – cfh
Learning Curve. A plot of the training and test set error vs training set size. Which should allow me to get a sense for whether increasing the training set size will improve performance significantly – jkarimi

1 Answers

1
votes

Generally, I'd plot all the things. Or write a script to collect everything and plot it all.

I think the need for the whole classifier plot is obvious. But the class-by-class ones strike me as valuable to ensure that you're not having problems from a single class. If, say, "5" is stubbornly resistant to increased training data, but the overall classifier is still being helped by it, I'd prefer to investigate the situation for that one class before I poured on more data.