Tree Based Methods in Machine Learning

1. Majority Voting
2. Using K-identical base-learning algorithms
3. Using K-different base-learning algorithms
4. Using K-different data-folds
5. Using K-different random number seeds
6. A combination of all the above approaches

1. Stacking since each classier is trained on all of the available data.
2. Bagging since we can combine as many classifiers as we want by training each on a different sub-set of the training corpus.

In your opinion, which approach is being utilized?

2. True or false: The generalization accuracy of an ensemble increases with the number of well-trained models it consists of.
3. True or false: Bootstrap aggregation (or bagging), refers to a process wherein a CNN ensemble is being trained using a random subset of the training corpus.
4. True or false: Bagging assumes that if the single predictor shave independent errors, then a majority vote of their outputs should be better than the individual predictions.

1. A mixture of experts
2. Bagging
3. Ensemble averaging
4. Boosting

1. Do deep ensembles achieve a better performance on out-of-distribution uncertainty benchmarks compared with Monte-Carlo (MC)-dropout?

the number of adjustable parameters is large compared to cardinality (i.e., size of the set) of the training corpus.

2. Provide examples of ensemble methods in classical machine-learning.
3. True or false: Ensemble methods usually have stronger generalization ability.
4. Complete the sentence: Bagging is variance/bias reduction scheme while boosting reduced variance/bias.

  import torchvision.models as models ...
  models = ['resnext']
  for m in models: 
      train ...
      compute VAL loss ... 
      amend LR ...
      if (val_acc > 90.0):
          saveModel()

1. What type of ensembling can be used with this approach? Explain in detail.
2. What is the main advantage of snapshot ensembling? What are the disadvantages, if any?

  import torchvision.models as models
  import random
  import np
  ...
  models = ['resnext']
  for m in models:
      train ...
      compute loss ...
      amend LR ...
      manualSeed= draw a new random number
      random.seed(manualSeed)
      np.random.seed(manualSeed)
      torch.manual_seed(manualSeed)
      if (val_acc > 90.0):
          saveModel()

Explain in detail what would be the possible effects of adding lines 10-13.

    import torchvision.models as models 
    ...
    models = ['resnext','vgg','dense']
    for m in models: 
        train ...
        compute loss/acc 
        ... 
        if (val_acc > 90.0):
            saveModel()
    ```
    What type of ensembling is being utilized in this approach? Explain in detail.
2. Name one method by which NN models may be combined to yield a single prediction.

1. What is gradient boosting?
2. What problems is gradient boosting good for?

algorithm.

Algorithm 1: Algo 1

Data: A set of training data, Q with N elements has been established while K times do Create a random subset of N ′ data by sampling from Q containing the N samples; N′ < N; Execute algorithm Algo 2; Return all N′ back to Q

Algorithm 2: Algo 2

Choose a learner $h_m$; while K times do Pick a training set and train with $h_m$;

of which has either $1, 3, 6, 10, 15, 21, 28, 36, \hspace{0.1em}\text{or} \hspace{0.1em} 45 \hspace{0.5em} \text{balls}$.

1. We draw several samples from this corpus as presented in Fig.6.3 where in each sample is equiprobable. What type of sampling approach is being utilized here?
2. Two samples are drawn one after the other. In which of the following cases is the covariance between the two samples equals zero? - (i) Sampling without replacement - (ii) Sampling with replacement
3. During training, the corpus sampled with replacement and is divided into several folds as presented in Fig. 6.4.

1. Majority voting for binary classification
2. Weighted majority voting for binary classification
3. Majority voting for class probabilities (iv) Weighted majority class probabilities
4. An algebraic weighted average for class probabilities
5. An adaptive weighted majority voting for combining multiple classifiers

l=[]
for i,f in enumerate(filelist):
    temp = pd.read_csv(f)
    l.append(temp)
arr = np.stack(l,axis=-1)
avg_results = pd.DataFrame(arr[:,:-1,:].mean(axis=2))
avg_results['image'] = l[0]['image']
avg_results.columns = l[0].columns

fore this may lead to some instances being represented numerous times while other instances not to be represented at all.

1. What are some of the fundamental differences between bagging and boosting algorithms?
2. How are they used in deep learning?