GA 2.6: Machine Learning¶

YOUR GROUP NAME HERE

Primary Task¶

Complete the notebook Analysis.ipynband write your answers in this document as requested in the questions below. Note that only part of the notebook results are required to be included in this report. Typically a short, one-line answer is sufficient; though a simple 'yes' or 'no' is not sufficient, please include a short reasoning, justification or argumentation.

You will be graded on the plots and answers provided in this file. You can delete the instructions and any other unnecessary text prior to submission.

Report Instructions¶

Remember to use Markdown features to clearly indicate your answers for each question below.

Importing figures into a Markdown file:

Use relative referencing only, with the git repo (working directory) as the root (this is expressed with a single dot .)
Our grading systems are case-sensitive so match the names of folders exactly
Use linux-style path separators: / rather than \.
Do not include spaces in your file path or image name; if it is unavoidable replace the space with %20, for example: ![My image](./my%20image.png)

Here are some examples:

an image located in the working directory ![My image](./imagename.ext) (where ext is any image extension).
an image located in a sub-directory called "images": ![My image](./images/imagename.ext)
an image with a space in the file name: ![My image](./images/my%20image.png)

When using Markdown to include an image, the square brackets is a text tag that is displayed in case the image does not load. Do not include a dot in the square brackets; i.e., do not do this: ![my image.](./image.svg).

Answers to Questions¶

Section 1¶

1.1) What is the purpose of splitting a dataset into training, validation, and test sets in the context of machine learning?

Your answer here.

1.2) Why should the MinMaxScaler be fitted on the training data only?

Your answer here.

1.3) Why is it crucial that the exact same scaler is used to transform the validation dataset?

Your answer here.

Section 2¶

2.1) Based on the shape of the loss curves, what can you indicate about the fitting capabilities of the model? (Is it overfitting, underfitting, or neither?)

Your answer here.

2.2) Why is the model performing so poorly? Can you give an explanation based on the physics of the problem? Is there a crack location for which this model does make a good prediction? Why is that the case?

Your answer here.

2.3) Can you explain why the model performs poorly in light of the assumptions we made for our observation model $p(t\vert x)=\mathcal{N}\left(t\lvert y(x),\beta^{-1}\right)$?

Your answer here.

Section 3¶

3.1) Based on the shape of the loss curves, what can you indicate about the fitting capabilities of the model? (Is it overfitting, underfitting, or neither?)

Your answer here.

3.2) What criterion did you use to measure the quality of your model when trying out different architectures?

Your answer here.

3.3) How well does your final model do compared to the one in Part 2? Use the parity plots you obtained to make your argument. Can you give a physical explanation for why this is the case?

Your answer here.

3.4) Can you propose explanations for the errors that still remain?

Your answer here.

Section 4¶

4.1) How does hyperparameter tuning in machine learning relate to the concept of model complexity?

Your answer here.

4.2) Given a comprehensive list of layer sizes and numbers, and given a relatively small training dataset, we expect the top left corner of the heatmap to have high validation errors. Why is that?

Your answer here.

4.3) Following up on the previous question, we also expect the bottom right corner of the heatmap to have high validation errors. Why is that?

Your answer here.

4.4) How does the performance of your final model for this part compare with the one you tweaked manually?

Your answer here.

General Comments on the Assignment [optional]¶

Use this space to let us know if you encountered any issues completing this assignment (but please keep it short!). For example, if you encountered an error that could not be fixed in your Python code, or perhaps there was a problem submitting something via GitLab. You can also let us know if the instructions were unclear. You can delete this section if you don't use it.

End of file.