Learning+curves

toc

The //Advanced mode// of the Experimenter can be used to generated learning curves for classifiers. These approaches can be setup in the //Simple mode// as well, but it is more cumbersome than in the advanced mode.

= Number of instances = For varying the number of instances a classifier is trained on, we use the classifier (package ) in conjunction with the  filter (package ) and J48 as base classifier (package ): code format="text" splitEvaluator -> classifier -> filter -> percentage code code format="text" 90, 80, 70, 60, 50, 40, 30, 20, 10 code
 * start the Experimenter (class )
 * select the configuration mode //Advanced// in the //Setup// panel
 * choose as //Destination// either an ARFF file (= ) or a database (= ) and configure the listener to your needs
 * choose as //Result generator// the (or leave the )
 * open the options dialog of the by left-clicking on the edit field
 * in case of regression datasets, choose the instead of the  (the latter is used for classification problems)
 * open the options dialog for the //splitEvaluator// by left-clicking on the edit field
 * //choose// the classifier that you want to analyze and setup it's parameters, in our case this is with  as base classifier and  as filter
 * close all dialogs again (accepting them with OK)
 * set the //Generator properties// to //enabled//
 * choose as property //percentage// and click on //Select//:
 * now you can add all the percentages that you want to test, e.g. (NB: this is the percentage being //removed//!):
 * add the datasets you want to generate the learning curve for
 * save the experiment
 * go to the //Run// panel and start the experiment
 * after the experiment has finished, select the //Analyse// panel and perform your analysis on the results

= Classifier parameter = This example shows how to generate a learning curve that does not vary on the number of instances, but on a specific classifier parameter, e.g., the //confidenceFactor// (= commandline option ) of. code format="text" splitEvaluator -> classifier -> confidenceFactor code code format="text" 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, 0.50 code
 * start the Experimenter (class )
 * select the configuration mode //Advanced// in the //Setup// panel
 * choose as //Destination// either an ARFF file (= ) or a database (= ) and configure the listener to your needs
 * choose as //Result generator// the (or leave the )
 * open the options dialog of the by left-clicking on the edit field
 * in case of regression datasets, choose the instead of the  (the latter is used for classification problems)
 * open the options dialog for the //splitEvaluator// by left-clicking on the edit field
 * //choose// the classifier that you want to analyze and setup it's parameters, in our case this is
 * close all dialogs again (accepting them with OK)
 * set the //Generator properties// to //enabled//
 * choose as property //percentage// and click on //Select//:
 * now you can add all the factors that you want to test, e.g.:
 * add the datasets you want to generate the learning curve for
 * save the experiment
 * go to the //Run// panel and start the experiment
 * after the experiment has finished, select the //Analyse// panel and perform your analysis on the results

= See also =
 * Databases