Errors in training set for classification algorithm

3
votes

I am looking for an algorithm to build a classifier for object detection. The classifier will be used to detect cars in a traffic scene through the object its features. I have looked into algorithms like boosting and multiple instance learning but they don't seem suitable for my situation. I have two dataset, one with positive samples and one with negative samples but these sets contain a small percentage of errors (<10%). The errors in the datasets cannot be manually removed because my goal is to generate the datasets online without human interference.

So my question is: does anyone know a suitable algorithm (if one exists) for training a classifier which can handle a relatively small percentages of errors in positive and negative training sets?

Thanks, Tom

1
machine-learningcomputer-visionclassification
Pretty much all classification algorithms can handle label errors, you'll just get worse test set performance. You should look into the general area of binary classification algorithms. Boosting isn't an algorithm per se, as it needs an underlying classifier to work. - Andrew Mao
10% should not be a big number. Multiple instance learning is a way to handle label errors but it is used for a totally different purpose which I can get into if you want to... - Autonomous
Thanks, I started looking into random forests. This is working quite well up till now. - Tom Neutens

1 Answers

0
votes

All real world data will have errors. If you know of specific errors you should filter them before you train. The best way to catch these errors is to incrementally train your classifiers. Every time you add to your corpus and retrain a new classifier you should check the precision, recall, and F1. If the new classifier performs worse you know that you added some garbage to your working corpus that must be backed out.

You absolutely should not leave the garbage in your corpus as you will kill your accuracy. If you are ingesting data from an untrusted source it is possible to intentionally subvert your classifier with bad data. Thus, you should have some sort of filter or domain expert looking at new data before it is feed to the classifier to ensure you maintain a gold standard corpus.