Bioinformatics Toolbox

classperf

Evaluated the performance of a classifier

Syntax

classperf cp = classperf(groundtruth) classperf(cp, classout) classperf(cp, classout, testidx) cp = classperf(groundtruth, classout,...) cp = classperf(...,'positive', p, 'negative', n)

Description

classperf provides an interface to keep track of the performance during the validation of classifiers. classperf creates and updates a classifier performance (CP) object that accumulates the results of the classifier. Later, classification standard performance parameters can be accessed using the function get or as fields in structures. Some of these performance parameters are ErrorRate, CorrectRate, ErrorDistributionByClass, Sensitivity and Specificity. classperf, without input arguments, displays all the available performance parameters.

cp = classperf(groundtruth) creates and initializes an empty object, CP is the handle to the object. groundtruth is a vector containing the true class labels for every observation. groundtruth can be a numeric vector or a cell array of strings. When used in a cross-validation design experiment, groundtruth should have the same size as the total number of observations.

classperf(cp, classout) updates the CP object with the classifier output classout. classout is the same size and type as groundtruth. When classout is numeric and groundtruth is a cell array of strings, the function grp2idx is used to create the index vector that links classout to the class labels. When classout is a cell array of strings, an empty string, '', represents an inconclusive result of the classifier. For numeric arrays, NaN represents an inconclusive result.

classperf(cp, classout, testidx) updates the CP object with the classifier output classout. classout has smaller size than groundtruth, and testidx is an index vector or a logical index vector of the same size as groundtruth, which indicates the observations that were used in the current validation.

cp = classperf(groundtruth, classout,...) creates and updates the CP object with the first validation. This form is useful when you want to know the performance of a single validation.

cp = classperf(...,'positive', p, 'negative', n) sets the 'positive' and 'negative' labels to identify the target disorder and the control classes. These labels are used to compute clinical diagnostic test performance. p and n must consist of disjoint sets of the labels used in groundtruth. For example, if

groundtruth = [1 2 2 1 3 4 4 1 3 3 3 2]

you could set

p = [1 2];
n = [3 4];

If groundtruth is a cell array of strings, p and n can either be cell arrays of strings or numeric vectors whose entries are subsets of grp2idx(groundtruth). p defaults to the first class returned by grp2idx(groundtruth), while n defaults to all the others. In clinical tests, inconclusive values ('' or NaN) are counted as false negatives for the computation of the specificity and as false positives for the computation of the sensitivity, that is, inconclusive results may decrease the diagnostic value of the test. Tested observations for which true class is not within the union of p and n are not considered. However, tested observations that result in a class not covered by the vector groundtruth are counted as inconclusive.

Examples

% Classify the fisheriris data with a K-Nearest Neighbor classifier
load fisheriris
c = knnclassify(meas,meas,species,4,'euclidean','Consensus');
cp = classperf(species,c)
get(cp)
 
% 10-fold cross-validation on the fisheriris data using linear
% discriminant analysis and the third column as only feature for
% classification
load fisheriris
indices = crossvalind('Kfold',species,10);
cp = classperf(species); % initializes the CP object
for i = 1:10
    test = (indices == i); train = ~test;
    class = classify(meas(test,3),meas(train,3),species(train));
    % updates the CP object with the current classification results
    classperf(cp,class,test)  
end
cp.CorrectRate % queries for the correct classification rate

 
cp =
 
	biolearning.classperformance

                        Label: ''
                  Description: ''
                  ClassLabels: {3x1 cell}
                  GroundTruth: [150x1 double]
         NumberOfObservations: 150
               ControlClasses: [2x1 double]
                TargetClasses: 1
            ValidationCounter: 1
           SampleDistribution: [150x1 double]
            ErrorDistribution: [150x1 double]
    SampleDistributionByClass: [3x1 double]
     ErrorDistributionByClass: [3x1 double]
               CountingMatrix: [4x3 double]
                  CorrectRate: 1
                    ErrorRate: 0
             InconclusiveRate: 0.0733
               ClassifiedRate: 0.9267
                  Sensitivity: 1
                  Specificity: 0.8900
      PositivePredictiveValue: 0.8197
      NegativePredictiveValue: 1
           PositiveLikelihood: 9.0909
           NegativeLikelihood: 0
                   Prevalence: 0.3333
              DiagnosticTable: [2x2 double]


ans =

    0.9467

classperf

Syntax

Description

Examples

See Also