In this problem we will apply discriminant analysis to recognize the digits in the MNIST data set (http://yann.lecun.com/exdb/mnist/). As a bonus problem we will construct “Fisher digits”. We will train our model using the training data sets (“train-images-idx3-ubyte.gz” and “train-labels-idx1-ubyte.gz”) and test the performance using the test data set (“t10k-images-idx3-ubyte.gz” and “t10k-labels-idx1-ubyte.gz”).

1. The images are 28 x 28 pixels in gray-scale. The categories are 0, 1, … 9. We concatenate the image rows into a 28 x 28 vector and treat this as our feature, and assume the feature vectors in each category in the training data “train-images-idx3-ubyte.gz”) have Gaussian distribution. Draw the mean and standard deivation of those features for the 10 categories as 28 x 28 images using the training images (“train-images-idx3-ubyte.gz”). There should be 2 images for each of the 10 digits, one for mean and one for standard deviation. We call those “mean digits” and “standard deviation digits” in CSE455/555.

2. Classify the images in the testing data set (“t10k-images-idx3-ubyte.gz”) using 0-1 loss function and Bayesian decision rule and report the performance. Why it doesn’t perform as good as many other methods on LeCun’s web page? Before coding the discriminant functions, review Section 2.6.

3. [Optional] Construct the “Fisher digits” from the MNIST data set according to Sections 3.8.2 and 3.8.3. This web page on Fisher faces (http://www.scholarpedia.org/article/Fisherfaces) and this web page (https://www.bytefish.de/blog/fisherfaces/) might be helpful. Answer two questions about these sections: (a) Why should the vector w minimizing Eq. (103) satisfy Eq. (104)? (b) Why should the between-class scatter matrix in Eq. (115) is n subscript 1 asterisk times n subscript 2 divided by n times the one in Eq. (102) in two-class case (i.e., c=2)? In addition, convince ourselves that Eq. (125) is the quotient between two “volumes” by referring the Wikipedia page on determinant (https://en.wikipedia.org/wiki/Determinant).

Please submit your code, documentation and solutions electronically through UBLearns. Please typeset your mathematics in LaTeX or Word.

Bibliographical citation ALL reviews should begin with a bibliographical citation. You will find
information about how to do this at the end of this handout. Historians use Chicago Manual of Style for all their citations.

b. Introduction You should have a brief introduction that lays out YOUR thesis of your review what you are hoping to state/argue about the book and your introduction MUST include the authors thesis for the book (either paraphrased or as a direct quote).
c. Profile/Summary of the book You should have a short paragraph that gives a summary of the book what it was about. Emphasis on SHORT (4 or 5 sentences at the most); it shouldnt be a detail by detail listing of the book.
d. Body You should discuss your observations (from the AS YOU READ section) in the body. I wont give you expectations regarding how to put these observations together or how many paragraphs, but it should be logical and well thought out. Remember that paragraphs are dividing different topics (even sub-topics) and each paragraph should have a mini-main idea. You can/should use quotes to support your points, but keep them short. (QUOTES SHOULDNT BE LONGER THAN ONE SENTENCE/LINE OF TEXT) You will have to cite any quotes or specific details you use (again, using Chicago Manual of Style as described below and in a handout on Blackboard).
e. Conclusion You should have a short paragraph that sums up your thesis/argument and major points of your review. Ideally, it should leave your reader with some ideas about whether they want to read the book or not.