Day 11 (7/22/19): Partial Rehearsal and Out-Of-Distribution Recognition

This morning I reviewed the results of the different incremental learning models I started training over the weekend. Here are the results for five different methods.

*Accuracy is computed as the average of the batch 1 and batch 2 accuracies to represent the overall test set. Omega is the average ratio between the accuracy of the incremental model vs. the offline model (offline accuracy with default hyperparameters: .7330)

No Regularization:

Batch 1--> Testing 45/45 Accuracy: 0.0241

Batch 2--> Testing 46/46 Accuracy: 0.7702

Accuracy: 0.3972

Omega: 0.5418

L2 Regularization:

Testing 45/45 Accuracy: 0.0132

Testing 46/46 Accuracy: 0.7868

Accuracy: 0.4000

Omega: 0.5457

L2SP Regularization:

Batch 1--> Testing 47/47 Accuracy: 0.0000

Batch 2--> Testing 45/45 Accuracy: 0.8185

Accuracy: 0.4093

Omega: 0.5583

Pseudo-Rehearsal (w/ random sampling of ten images per batch-1 class):

Testing 45/45 Accuracy: 0.4166

Testing 46/46 Accuracy: 0.7988

Accuracy: 0.6077

Omega: 0.8291

Full Rehearsal:

Testing 45/45 Accuracy: 0.6499

Testing 46/46 Accuracy: 0.7496

Accuracy: 0.6997

Omega: 0.9546

These metrics obtained from my models are consistent with what was expected (Omega value of full rehearsal>partial rehearsal>simple regularization).

________________________________________________________________________

After speaking with Dr. Kanan about my project and progress I spent the rest of the afternoon reading papers and going through presentations created in the kLab. These topics ranged from evaluating bounded classification methods for CNNs (aka open-set or out-of-distribution recognition) to new techniques for streaming learning. 

Tomorrow I will be trying to experiment with some Open-set Recognition methods (similar to how I experimented with incremental learning techniques), and I'm excited to see how that turns out.