FS19 STT481作业写作、Java程序设计作业写作

”

FS19 STT481: Homework 3
(Due: Wednesday, November 6th, beginning of the class.)
100 points total
1. (20 pts) Finish the swirl course Exploratory Data Analysis. Finish Section 1-10 (no need to do 11-15).
You can install and go to the course by using the following command lines.
library(swirl)
install_course(Exploratory_Data_Analysis)
swirl()
2. (20 pts) In this question, we are going to perform cross-validation methods in order to choose a better
logistic regression model.
Consider Weekly data set, where we want to predict Direction using Lag1 and Lag2 predictors. To load the
Weekly data set, use the following command lines.
library(ISLR)
data(Weekly)
Suppose now I have two candidate models:
(i) log P r(Direction==Up)
1P r(Direction==Up) = 0 + 1Lag1 + 2Lag2;
(ii) log P r(Direction==Up)
1P r(Direction==Up) = 0 + 1Lag1 + 2Lag2 + 3Lag12 + 4Lag22.
(a) For each model, compute the LOOCV estimate for the test error by following the steps:
Write a for loop from i = 1 to i = n, where n is the number of observations in the data set, that performs
each of the following steps:
i. Fit a logistic regression model using all but the ith observation to predict Direction using Lag1 and
Lag2 for model (i) and using Lag1, Lag2, I(Lag12), I(Lag22) for model (ii).
ii. Compute the posterior probability of the market moving up for the ith observation.
iii. Use the posterior probability for the ith observation and use the threshold 0.5 in order to predict
whether or not the market moves up.
iv. Determine whether or not an error was made in predicting the direction for the ith observation. If an
error was made, then indicate this as a 1, and otherwise indicate it as a 0.
Take the average of the n numbers obtained in iv in order to obtain the LOOCV estimate for the test error.
(b) Comment on the results. Which of the two models appears to provide the better result on this data
based on the LOOCV estimates?
(c) The cv.glm function can be used to computer the LOOCV test error estimate. Run the following
command lines and see whether the results are the same as the ones you did in (a).
library(boot)
# Since the response is a binary variable an
# appropriate cost function for glm.cv is
cost – function(r, pi = 0) mean(abs(r – pi) 0.5)
glm.fit – glm(Direction ~ Lag1 + Lag2, data = Weekly, family = binomial)
cv.error.1 – cv.glm(Weekly, glm.fit, cost, K = nrow(Weekly))$delta[1]
glm.fit – glm(Direction ~ Lag1 + Lag2 + I(Lag1^2) + I(Lag2^2), data = Weekly, family = binomial)
cv.error.2 – cv.glm(Weekly, glm.fit, cost, K = nrow(Weekly))$delta[1]
1
(d) For each model, compute the 10-fold CV estimate for the test error by following the steps:
Run the following command lines.
set.seed(1) ## the seed can be arbitrary but we use 1 for the sake of consistency
fold.index – cut(sample(1:nrow(Weekly)), breaks=10, labels=FALSE)
Write a for loop from i = 1 to i = 10 and in each loop, perform each of the following steps:
i. Fit a logistic regression model using all but the observations that satisfy fold.index==i to predict
Direction using Lag1 and Lag2 for model (i) and using Lag1, Lag2, I(Lag12), I(Lag22) for model
(ii).
ii. Compute the posterior probability of the market moving up for the observations that satisfy
fold.index==i.
iii. Use the posterior probabilities for the observations that satisfy fold.index==i and use the threshold
0.5 in order to predict whether or not the market moves up.
iv. Compute the error rate was made in predicting Direction for those observations that satisfy
fold.index==i.
Take the average of the 10 numbers obtained in iv in order to obtain the 10-fold CV estimate for the test
error.
(e) Comment on the results. Which of the two models appears to provide the better result on this data
based on the 10-fold CV estimates?
(f) cv.glm function can be used to compute the 10-fold CV test error estimate. Run the following command
lines and see whether the results are the same as the ones you did in (d). If they are not the same,
whats the reason?
library(boot)
# Since the response is a binary variable an
# appropriate cost function for glm.cv is
cost – function(r, pi = 0) mean(abs(r – pi) 0.5)
glm.fit – glm(Direction ~ Lag1 + Lag2, data = Weekly, family = binomial)
cv.error.1 – cv.glm(Weekly, glm.fit, cost, K = 10)$delta[1]
glm.fit – glm(Direction ~ Lag1 + Lag2 + I(Lag1^2) + I(Lag2^2), data = Weekly, family = binomial)
cv.error.2 – cv.glm(Weekly, glm.fit, cost, K = 10)$delta[1]
(g) Comment on the computation costs for LOOCV and 10-fold CV. Which one is faster in your implementation
in (a) and (d)?
3. (20 pts) In this question, we are going to perform cross-validation methods to determine the tuning
parameter K for KNN.
Consider Default data set, where we want to predict default using student, balance, and income predictors.
Since student is a qualitative predictor, we want to use dummy variable for it and standardize the data using
scale function. To load the Default data set and standardize the data, use the following command lines.
library(ISLR)
data(Default)
X – Default[, c(student, balance, income)]
X[,student] – ifelse(X[,student] == Yes, 1, 0)
X – scale(X)
y – Default[, default]
Suppose now the candidate tuning parameter Ks for KNN are K = 1, 5, 10, 15, 20, 25, 30.
(a) For each K, compute the LOOCV estimate for the test error by following the steps:
2
Write a for loop from i = 1 to i = n, where n is the number of observations in the data set, that performs
each of the following steps:
i. Perform KNN using all but the ith observation and predict default for the ith observation. (Hint: use
knn function and return the class. No need to compute posterior probabilities. That is, use prob =
FALSE in the knn function and then use the return class of knn).
ii. Determine whether or not an error was made in predicting the direction for the ith observation. If an
error was made, then indicate this as a 1, and otherwise indicate it as a 0.
Take the average of the n numbers obtained in ii in order to obtain the LOOCV estimate for the test error.
(b) Comment on the results. Which of the tuning parameter Ks appears to provide the best results on
this data based on the LOOCV estimates?
(c) knn.cv function can be used to perform LOOCV. Run the following command lines and see whether
the results are same as the ones you did in (a).
library(class)
for(k in c(1,5,10,15,20,25,30)){
cvknn – knn.cv(X, y, k = k) ## the little k here is the number of nearest neighbors not k-fold
print(mean(cvknn != y))
}
(d) For each K, compute the 10-fold CV estimate for the test error by following the steps:
Run the following command lines.
set.seed(10) ## the seed can be arbitrary but we use 10 for the sake of consistency
fold.index – cut(sample(1:nrow(Default)), breaks=10, labels=FALSE)
Write a for loop from i = 1 to i = 10 and in the loop, perform each of the following steps:
i. Perform KNN using all but the observations that satisfy fold.index==i and predict default for the
observations that satisfy fold.index==i. (Hint: use knn function and return the class. No need to
compute posterior probabilities. That is, use prob = FALSE in the knn function and then use the return
class of knn).
ii. Compute the error rate was made in predicting the direction for those observations that satisfy
fold.index==i.
Take the average of the 10 numbers obtained in ii in order to obtain the 10-fold CV estimate for the test error.
(e) Comment on the results. Which of the tuning parameter Ks appears to provide the best results on
this data based on the 10-fold CV estimates?
4. (10 pts) In this question, we are going to use the zipcode data in the HW2 Q10.
(a) Using the zipcode_train.csv data, perform a 10-fold cross-validation using KNNs with K = 1, 2, . . . , 30
and choose the best tuning parameter K.”