Coursera: Machine Learning-Andrew NG(Week 3) Quiz - Logistic Regression

These solutions are for reference only.

try to solve on your own

but if you get stuck in between than you can refer these solutions

there are different set of questions ,

we have provided the variations in particular question at the end.

read questions carefully before marking

-----------------------------------------------------------------------------------------

Logistic Regression

TOTAL POINTS 5

1.

Question 1

Suppose that you have trained a logistic regression classifier, and it outputs on a new example $x$ a prediction $h_\theta(x)$ = 0.7. This means (check all that apply):

1 point

Our estimate for $P(y=0|x;\theta)$ is 0.3.

Our estimate for $P(y=1|x;\theta)$ is 0.7.

Our estimate for $P(y=0|x;\theta)$ is 0.7.

Our estimate for $P(y=1|x;\theta)$ is 0.3.

EXPLANATION:

Our estimate for P(y=1|x;θ) is 0.7  =>because =>hθ(x) = 0.7

Our estimate for P(y=0|x;θ) is 0.3 =>because => P(y=0|x;θ) = 1 - P(y = 1| x; θ); the former is 1 - 0.7= 0.3

2.

Question 2

Suppose you have the following training set, and fit a logistic regression classifier $h_\theta(x) = g(\theta_0 + \theta_1x_1 + \theta_2 x_2)$.

Which of the following are true? Check all that apply.

1 point

$J(\theta)$ will be a convex function, so gradient descent should converge to the global minimum.

Adding polynomial features (e.g., instead using $h_\theta(x) = g(\theta_0 + \theta_1x_1 + \theta_2 x_2 + \theta_3 x_1^2 + \theta_4 x_1 x_2 + \theta_5 x_2^2)$ ) could increase how well we can fit the training data.

The positive and negative examples cannot be separated using a straight line. So, gradient descent will fail to converge.

Because the positive and negative examples cannot be separated using a straight line, linear regression will perform as well as logistic regression on this data.

EXPLANATION:

J(θ) will be a convex function, so gradient descent should converge to the global minimum.(true)
=>fact

Adding polynomial features (e.g., instead using hθ(x) = g(θ0 + θ1x1 + θ2x2 + θ3x2 + θ4x1x2 + θ5x2 )) could increase how well we can fit the training data (true)
=>Adding new features can only improve the fit on the training set: since setting θ3 = θ4 = θ5 = 0 makes the hypothesis the same as the original one, gradient descent will use those features (by making the corresponding non-zero) only if doing so improves the training set fit

other statements that can occur in this question:

At the optimal value of θ (e.g., found by fminunc), we will have J(θ) ≥ 0. (true)

3.

Question 3

For logistic regression, the gradient is given by $\frac{\partial}{\partial \theta_j} J(\theta) =\frac{1}{m}\sum_{i=1}^m{ (h_\theta(x^{(i)}) - y^{(i)}) x_j^{(i)}}$. Which of these is a correct gradient descent update for logistic regression with a learning rate of $\alpha$? Check all that apply.

1 point

$\theta_j := \theta_j - \alpha \frac{1}{m} \sum_{i=1}^m{ \left(\frac{1}{1 + e^{-\theta^T x^{(i)}}} - y^{(i)}\right) x_j^{(i)}}$ (simultaneously update for all $j$).

$\theta := \theta - \alpha \frac{1}{m} \sum_{i=1}^m{ \left(\theta^T x - y^{(i)}\right) x^{(i)}}$.

$\theta_j := \theta_j - \alpha \frac{1}{m} \sum_{i=1}^m{ (h_\theta(x^{(i)}) - y^{(i)}) x_j^{(i)}}$ (simultaneously update for all $j$).

$\theta_j := \theta_j - \alpha \frac{1}{m} \sum_{i=1}^m{ (h_\theta(x^{(i)}) - y^{(i)}) x^{(i)}}$ (simultaneously update for all $j$).

variation to 3rd  question is provided at the end.

4.

Question 4

Which of the following statements are true? Check all that apply.

1 point

For logistic regression, sometimes gradient descent will converge to a local minimum (and fail to find the global minimum). This is the reason we prefer more advanced optimization algorithms such as fminunc (conjugate gradient/BFGS/L-BFGS/etc).

Linear regression always works well for classification if you classify by using a threshold on the prediction made by linear regression.

The cost function $J(\theta)$ for logistic regression trained with $m \geq 1$ examples is always greater than or equal to zero.

The sigmoid function $g(z) = \frac{1}{1 + e^{-z}}$ is never greater than one ($>1$).

EXPLANATION:

The cost function J(θ) for logistic regression trained with examples is always greater than or equal to zero.(true)
=>The cost for any example x(i) is always ≥ 0 since it is the negative log of a quantity less than one. The cost function J(θ) is a summation over the cost for each eample, so the cost function itself must be greater than or equal to zero.

The sigmoid function is never greater than one(true)
=>fact

other statements that can occur in this question:

The one-vs-all technique allows you to use logistic regression for problems in which each y(i)comes from a fixed, discrete set of values. (true)
=>If each y(i) is one of k different values, we can give a label to each y(i)belongs{1,2,....,k} and use one-vs-all as described in the lecture.

5.

Question 5

Suppose you train a logistic classifier $h_\theta(x) = g(\theta_0 + \theta_1x_1 + \theta_2 x_2)$. Suppose $\theta_0 = 6, \theta_1 = -1, \theta_2 = 0$. Which of the following figures represents the decision boundary found by your classifier?

1 point

Figure:

Figure:

Figure:

Figure:

EXPLANATION:

In this figure, we transition from negative to positive when x1 goes from left of 6 to right of 6 which is true for the given values of θ.

$h_{\theta }(x)=g({\theta }_0+{\theta }_1{x}_1+{\theta }_2{x}_2)$.

${where\; \theta }_0=-6,{\theta }_1=0,{\theta }_2=1$.

-----------------------------------------------------------------------------

variations in 5 th question:

5.

Question 5

Suppose you train a logistic classifier $h_\theta(x) = g(\theta_0 + \theta_1x_1 + \theta_2 x_2)$. Suppose $\theta_0 = -6, \theta_1 = 0, \theta_2 = 1$. Which of the following figures represents the decision boundary found by your classifier?

1 point

Figure:

Figure:

Figure:

Figure:

variations in 3 th question:

3.

Question 3

For logistic regression, the gradient is given by $\frac{\partial}{\partial \theta_j} J(\theta) = \frac{1}{m}\sum_{i=1}^m{ (h_\theta(x^{(i)}) - y^{(i)}) x_j^{(i)}}$. Which of these is a correct gradient descent update for logistic regression with a learning rate of $\alpha$? Check all that apply.

1 point

$\theta_j := \theta_j - \alpha \frac{1}{m} \sum_{i=1}^m{ \left(\theta^T x - y^{(i)}\right) x_j^{(i)}}$ (simultaneously update for all $j$).

$\theta := \theta - \alpha \frac{1}{m} \sum_{i=1}^m{ \left(\frac{1}{1 + e^{-\theta^T x^{(i)}}} - y^{(i)}\right) x^{(i)}}$.

$\theta := \theta - \alpha \frac{1}{m} \sum_{i=1}^m{ (h_\theta(x^{(i)}) - y^{(i)}) x^{(i)}}$.

$\theta := \theta - \alpha \frac{1}{m} \sum_{i=1}^m{ \left(\theta^T x - y^{(i)}\right) x^{(i)}}$.

---------------------------------------------------------------------------------

reference : coursera