Report#
Task 1#
1) Why should the MinMaxScaler be fitted on the training data only? (0.5 point)
2) Why is it crucial that the exact same scaler is used to transform the validation dataset? (0.5 point)
3) The train_model function tracks the MSE loss for both training and validation datasets. In a case of extreme overfitting, what would you expect to happen with the training loss? What about the validation loss? (1 point)
4) Looking at the loop in train_model, training progresses all the way until n_epochs is reached, regardless of how the validation loss is evolving. This makes the training prone to overfitting. Briefly explain in words how you could modify the code above in order to implement Early Stopping. (1 point)
Tasks 2 and 3#
5) Look at how both loss curves behave in the plot above. Is the model overfitting? Is the network learning during the training process? Is there a need to use more epochs for this particular model? (0.5 point)
6) Look at the parity plots above for the one-feature model. We see that the model is not doing well. Is the model overfitting or underfitting? Why is this happening? Consider the plotted dataset at the top of the notebook to justify your answer. (1 point)
7) Are there cities for which even this model gives reasonable flooding probability predictions? Use your parity plots to motivate your answer. (0.5 point)
Task 4#
8) Looking at the new parity plots, what suggests this model performs better than the previous one? (0.5 point)
9) Comparing training and validation parity plots, is this new model suffering from overfitting? (1 point)
Task 5#
10) Looking at all of your new plots, what are the signs this new model is suffering from overfitting? What is the root cause for the overfitting? (1.0 point)
Task 6#
11) Given a comprehensive list of layer sizes and numbers, we would in theory expect the top left region of the heatmap to have high validation errors. Why is that? (0.75 point)
12) Following up on the previous question, we would also expect the bottom right region of the heatmap to have high validation errors since those models would suffer from overfitting. Why do we not see it happening here? Think about what changed between Task 5 and Task 6. (0.75 point)
By Iuri Rocha, Delft University of Technology. CC BY 4.0, more info on the Credits page of Workbook