Machine Learning

Linear Methods For Classification

Interview questions on Linear Methods For Classification.

32 questions

All subtopics General Concepts Odds and Log-odds The Sigmoid Truly Understanding Logistic Regression The Logit Function and Entropy

General Concepts

Q1. True or False: For a fixed number of observations in a data set, introducing more variables normally generates a model that has a better fit to the data. What may be the drawback of such a model-fitting strategy?

General Concepts

Q2. Define the term odds of success both qualitatively and formally. Give a numerical example that stresses the relation between probability and odds of an event occurring.

General Concepts

Q3. Answer the following:

Define what is meant by the term "interaction", in the context of a logistic regression predictor variable.
What is the simplest form of an interaction? Write its formulae.
What statistical tests can be used to attest to the significance of an interaction term?

General Concepts

Q4. True or False: In machine learning terminology, unsupervised learning refers to the mapping of input covariates to a target response variable that is attempted to be predicted when the labels are known.

General Concepts

Q5. Complete the following sentence: In the case of logistic regression, the response variable is the log of the odds of being classified in `[...]`.

General Concepts

Q6. Describe how in a logistic regression model, a transformation to the response variable is applied to yield a probability distribution. Why is it considered a more informative representation of the response?

General Concepts

Q7. Complete the following sentence: Minimizing the negative log-likelihood also means

maximizing the [...] of selecting the [...] class.

General Concepts

Q8. Assume the probability of an event occurring is `p = 0.1`.

What are the odds of the event occurring?
What are the log odds of the event occurring?
Construct the probability of the event as a ratio that equals 0.1

Odds and Log-odds

Q9. True or False: If the odds of success in a binary response is $4 $, the corresponding probability of success is$0.8$.

Odds and Log-odds

Q10. Draw a graph of odds to probabilities, mapping the entire range of probabilities to their respective odds.

Odds and Log-odds

Q11. The logistic regression model is a subset of a broader range of machine learning models known as generalized linear models (GLMs), which also include analysis of variance (ANOVA), vanilla linear regression, etc. There are three components to a GLM; identify these three components for binary logistic regression.

Odds and Log-odds

Q12. Let us consider the logit transformation, i.e., log-odds. Assume a scenario in which the logit forms the linear decision boundary, for a given vector of systematic components X and predictor variables θ. Write the mathematical expression for the hyperplane that describes the decision boundary.$$

\log{\frac{Pr(Y = 1 | X)}{Pr(Y = 0|X)}} = \theta_0 + \theta^TX

Odds and Log-odds

Q13. True or False: The logit function and the natural logistic (sigmoid) function are inverses of each other.

The Sigmoid

Q14. Compute the derivative of the natural sigmoid function:$$

\sigma(x) = \frac{1}{1+e^{-x}}; \quad \epsilon (0, 1)

The Sigmoid

Q15. Characterize the sigmoid function when its argument approaches $0 $,$∞$ and $-∞$.

The Sigmoid

Q16. Remember that in logistic regression, the hypothesis function for some parameter vector $β$ and measurement vector $x$ is defined as:$$ h_\beta(x) = g(\beta^Tx) = \frac{1}{1+e^{-\beta^Tx}}\\ = P(y = 1|x;\beta)$$

where y holds the hypothesis value. Suppose the coefficients of a logistic regression model with independent variables are as follows: $\beta_0 = -1.5$ , $\beta_1 = 3$ , $\beta_2 = -0.5$ . Assume additionally, that we have an observation with the following values for the independent variables: $x_1 = 1$ , $x_2 = 5$ . As a result, the logit equation becomes: $\text{logit} = \beta_0 + \beta_1x_1 + \beta_2x_2$ 1. What is the value of the logit for this observation? 2. What is the value of the odds for this observation? 3. What is the value of $P(y = 1)$ for this observation?

Truly Understanding Logistic Regression

Q17. Proton therapy (PT) is a widely adopted form of treatment for many types of cancer including breast and lung cancer (Fig. 2.2). <table align='center'> <tr> <td align="center"> <img src="img/lr-1.png" alt="A multi-detector positron scanner used to locate tumors" style="max-width:70%;" /> </td> </tr> <tr> <td align="center"> Pulmonary nodules (left) and breast cancer (right) </td> </tr> </table> A PT device which was not properly calibrated is used to simulate the treatment of cancer. As a result, the PT beam does not behave normally. A data scientist collects information relating to this simulation. The covariates presented in Table 2.1 are collected during the experiment. The columns Yes and No indicate if the tumour was eradicated or not, respectively. <table align='center'> <tr> <td align="center"> <img src="img/lr-2.png" alt="A multi-detector positron scanner used to locate tumors" style="max-width:70%;" /> </td> </tr> <tr> <td align="center">Tumour eradication statistics</td> </tr> </table> Referring to Table: Answer the following questions 1. What is the explanatory variable and what is the response variable? 2. Explain the use of relative risk and odds ratio for measuring association. 3. Are the two variables positively or negatively associated? Find the direction and strength of the association using both relative risk and odds ratio. 4. Compute a 95% confidence interval (CI) for the measure of association. 5. Interpret the results and explain their significance.

Truly Understanding Logistic Regression

Q18. Consider a system for radiation therapy planning (Fig. 2.3). Given a patient with a malignant tumour, the problem is to select the optimal radiation exposure time for that patient. A key element in this problem is estimating the probability that a given tumour will be eradicated given certain covariates. A data scientist collects information relating to this radiation therapy system. <table align='center'> <tr> <td align="center"> <img src="img/lr-3.png" alt="A multi-detector positron scanner used to locate tumors"style="max-width:70%;" /> </td> </tr> <tr> <td align="center">A multi-detector positron scanner used to locate tumors</td> </tr> </table> The following covariates are collected; $X_1$ denotes time in milliseconds that a patient is irradiated with, $X_2$= holds the size of the tumour in centimeters, and$ Y $notates a binary response variable indicating if the tumour was eradicated. Assume that each response’ variable$ Y_i $is a Bernoulli random variable with success parameter$ p_i$, which holds:$$

p_i = \frac{e^{\beta_0+\beta_1x_1+\beta_2x_2}}{1+e^{\beta_0+\beta_1x_1+\beta_2x_2}} $The data scientist fits a logistic regression model to the dependent measurements and produces these estimated coefficients:$ \hat{\beta_{0}} = -6 \ \hat{\beta_{1}}= 0.05\ \hat{\beta_{2}} = 1 $$1. Estimate the probability that, given a patient who undergoes the treatment for $40 \ milliseconds$ and who is presented with a tumour sized $3.5\ cm$ , the system eradicates the tumour. 2. How many milliseconds the patient in part (a) would need to be radiated with to have exactly a $50\\%$ chance of eradicating the tumour?

Truly Understanding Logistic Regression

Q19. Recent research suggests that heating mercury containing dental amalgams may cause the release of toxic mercury fumes into the human airways. It is also presumed that drinking hot coffee, stimulates the release of mercury vapour from amalgam fillings. <table align='center'> <tr> <td align="center"> <img src="img/lr-4.png" alt="A multi-detector positron scanner used to locate tumors"style="max-width:70%;" /> </td> </tr> <tr> <td align="center"> A dental amalgam </td> </tr> </table> To study factors that affect migraines, and in particular, patients who have at least four dental amalgams in their mouth, a data scientist collects data from $200K$ users with and without dental amalgams. The data scientist then fits a logistic regression model with an indicator of a second migraine within a time frame of one hour after the onset of the first migraine, as the binary response variable (e.g., migraine=1, no migraine=0). The data scientist believes that the frequency of migraines may be related to the release of toxic mercury fumes. There are two independent variables: 1. $X_1 = 1$ if the patient has at least four amalgams; $0$ otherwise. 2. $X_2$ = coffee consumption (0 to 100 hot cups per month). The output from training a logistic regression classifier is as follows: <table align='center'> <tr> <td align="center"> <img src="img/lr-5.png" alt= "A dental amalgam" style="max-width:70%;" /> </td> </tr> <tr> <td align="center"> A dental amalgam </td> </tr> </table> 1. Using $X_1$ and $X_2$, express the odds of a patient having a migraine for a second time. 2. Calculate the probability of a second migraine for a patient that has at least four amalgams and drank 100 cups per month? 3. For users that have at least four amalgams, is high coffee intake associated with an increased probability of a second migraine? 4. Is there statistical evidence that having more than four amalgams is directly associated with a reduction in the probability of a second migraine?

Truly Understanding Logistic Regression