AIML Special Presentation: Active Learning with Deep Neural Networks

In Active Learning the system gives select data to an expert to annotate, which is costly and should be minimised.  Prof Butine proposes the first general Bayesian method to work well in this context, and his experiments show it is the only method consistently better than random.  

His method, Bayesian Estimate of Mean Proper Scores (BEMPS), estimates the increase in strictly proper scores such as log probability (the original "Bayesian Active Learning", which doesn't work well) or negative mean square error within this framework. He also proved convergence results for this general class of costs and has recently developed a version which performs well with imbalanced data when some classes are rare but costly to get wrong.