1. Problem Statement

2. Pre-process

3. Algorithm Selection

4. Experiment

5. Conclusions

Odor feature vector is 931-dimensional.

Odor feature vector is 931-dimensional.

• slow regression / learning

• huge storage space

• noisy due to correlation

Odor feature vector is 931-dimensional.

• slow regression / learning

• huge storage space

• noisy due to correlation

Find an optimal subset of feature columns that works best for selected algorithms.

• Calculate Correlation Coefficient

  • 931*931 matrix, [0,1]

• Calculate Correlation Coefficient

  • 931*931 matrix, [0,1]

• Remove Most Correlated Column

  • max column sum -> most correlated

  • snapshots for 10 removals

  • 93 sets for experiment

• Regression

• Machine Learning

• Neural Network

• Probabilistic Graphical Model

• Boosting

• Meta Algorithms

• Regression

  • Logistic Regression

    • Simple Logistic

  • Classification and Regression Tree

    • ADTree

    • SimpleCART

• Machine Learning

  • Naive Bayes

  • Support Vector Machine

    • SVM

    • SMO

  • Kernel Estimation

    • KNN

• Neural Network

  • Multilayer Perceptron

    • too slow & low accuracy ~70%

  • Radial Basis Function Network

    • low accuracy ~65%

• Probabilistic Graphical Model

  • Bayesian Network

  • Hidden Markov Model

    • to be tried

• Boosting

  • AdaBoostM1

    • little gain from 96.07% to 96.42%

    • considerably slower

• Meta Algorithms

  • Bagging with REPTree

10 times 10-fold cross-validation

10 times 10-fold cross-validation

A one-time 10-fold cross-validation works as the following:

• Ten labeled train/test set pairs are generated.

• Train a classifier with an algorithm from a train set containing 90% labeled data from the entire input. Test the classifier performance using the rest 10% data which is the test set.

• Repeat step 2 with the 2nd to 10th train/test set pairs.

• Take the average performance as output.

This is repeated 10 times to make sure our results are statistically significant.

Results

• Accuracy

• Training Time

• Testing Time

• Model Size

• Bayesian algorithms are vulnerable to noise. They lose a bit accuracy after the feature vector length is larger than 400.

• Some machine learning algorithms, such as SVM, tend to ignore the negative impacts of noise. The accuracy do not drop when lots of noises are added. It does not goes any higher either.

• The highest accuracy (97%) is achieved by BayesNet with a very short feature vector length of around 70.

• Most algorithms peak at [250,300]. Several tree-based algorithms step up a lot from 260 to 270.

  • Therefore 270 appears to be an optimal choice.

• BayesNet provides highest accuracy.

• Tree-based algorithms are fast. They are optimal with 270-dimensional feature.

• AI algorithms (abandoned during algorithm selection phase) are a order of magnitude slower. They do not, however, provide lower error.

• Wrong cases will be inspected to find outliers or to improve current models.

• Finally, the optimal choices.

Algorithm	Feature Vector Length	Accuracy (%)	Training Time (s)	Testing Time (s)	Model Size (B)
BayesNet	70	0.97	0.03	0	5.2e5
Bagging	270	96.07	1.02	0	4.7e4
SMO w/Polykernel	440	96.9	0.67	0	2.8e6