Quiz 4

1. Why is the linear model good?

Linear models have efficient learning algorithms and controlled complexity. With good features and enough data, they can achieve both small $E in$ and good generalization.

2. What is the VC error bar for a d-dimensional perceptron?

Omega equals big O of the square root of d log N divided by N.

3. For the 256-dimensional digits, why were two features chosen?

To reduce model complexity and shrink the generalization error bar with the available training data:

Input dimension decreases from 256 to 2. VC dimension decreases from 257 to 3.

This helps ensure $E in is approximately E out$ .

4. For the 256-dimensional digits, why were two good features chosen?

To preserve useful information so that the classifier can still achieve a small $E in$ . Intensity and symmetry help distinguish 1s from 5s.

5. Write the model equation for linear classification.

h of x equals the sign of w transpose x.

6. Write the in-sample error for linear classification.

E in of h equals one over N times the sum, from n equals 1 to N, of the indicator that h of x n is not equal to y n.

7. Write the model equation for linear regression.

8. Write the in-sample error for linear regression.

E in of w equals one over N times the sum, from n equals 1 to N, of w transpose x n minus y n, squared.

9. Write the model equation for logistic regression.

h of x equals theta of w transpose x, which equals one divided by one plus e raised to negative w transpose x.

10. Write the in-sample error for logistic regression.

E in of w equals one over N times the sum, from n equals 1 to N, of the natural logarithm of one plus e raised to negative y n times w transpose x n.