EXAM August 20 - August 28 2025

The take-home exam comprises 2 questions with several subtasks each. Read through the entire exam before you start and think carefully about code-sharing between tasks to save time on execution.

It is a good strategy to attempt to answer all tasks since I will give partial marks. In addition, several of the tasks are interlinked.

Putting random results without motivation is not a good strategy. What I look for in your exams is a clear insight and motivation as to what you do to adress each task. If you do "random" things, or, even worse, "throw everything at the problem" and don't explain why you think this answers the questions, then this results in no marks. So please make sure you clearly state why you use a method/strategy/pipeline.

You should work alone for this exam. The reports will be checked with Urkund. You can discuss coding issues or bugs with other students sparingly, but don't discuss how to intepret figures or tables.

"Can I use ChatGPT?" It's been my experience that, while ChatGPT can be helpful, it is also overconfident in providing answers that are in fact wrong. It takes time to check the output from ChatGPT and, depending on your own skillset, this may result in an overall waste of time rather than a saving time, especially if you want to connect multiple tasks. Your main task is to analyze and interpret the results.

So I would say that you should use ChatGPT with caution and only for simple coding questions like generating plots, proposed functions calls (which may contain bugs), skeleton for coding pipelines, etc. Don't use ChatGPT for generating text.

GOOD LUCK!

• The report should not be longer than 1 page of text + max 1/2 page conclusion/main summary per sub-task (sub-tasks = a,b,c etc).
• You do not have to discuss the methods and assumptions unless this pertains to a result you present - i.e. you don't need to write a "lab report" with introduction, methods, analysis and conclusions. Here, it's only analysis and conclusions that I am interested in. However, you may need to refer to assumptions to explain why your results look like they do.
• Figures and Tables: maximum of 4 figures and 2 tables per sub-task (figures can contain panels but don't go overboard - I need to be able to read axes etc so tiny figures will not be appreciated).
• Make sure you provide captions for the tables and figures - these should clearly state what the figure/table contains as well as a 1-2 conclusion statement about what can be learnt from the figure/table. Refer to the figures and tables in the text!
• MOTIVATE YOUR ANSWERS!
• Don't contradict your results! If something is unexpected say so and explain why you thought you would see something different.
• Be careful about using buzzwords, guessing what I want you to write etc - just go directly from your results.
• Please think about your presentation. Spell check, grammar check. "The arm-length check": meaning, skim through the report as it appears on the screen - does it look nice and neat or is it messy with mis-alignments, cut-off sentences, plots that are tiny etc?
• No cut and paste code mixed with the text. It makes it very difficult to read.
• No, jupyter notebooks are not accepted as a report. Preferably use latex (overleaf) for neat report but word saved as a pdf is fine.
• Yes, you can use bulletpoints to present your findings. This could be especially useful for the subtask conclusion.
• Stress your main findings in the 1/2 page subtask conclusion.
• Make sure you put the figures with each subtask after the subtask discussion and before the next subtask so I don't have to scroll back and forth between tasks when I grade.

Grades: Each subtask will be scored as U, G and VG (fail, adequate pass, pass with distinction).

• Pass/3 cutoff is G on 4 out of the 6 subtasks.
• VG/4 cutoff is VG at least 2 of the subtasks and G on the rest.
• 5 cutoff is VG on at least 4 of the subtasks and G on the rest.

IMPORTANT

Wherever the exam task contains a question ("Does X differ from Y?", "Is A performing better than B") it is not sufficient to answer "Yes" or "No" - you need to explain how you arrived to this conclusion, which results provide you with these answers, as well as a statement trying explain why the results are what they are. Likewise, when a task is phrased as "Can you do X to get Y?" I, of course, mean for you to attempt to answer this question by doing something, not just hypothesizing about it.

To answer the questions it is for the most part inadequate to perform the task once on the full data set. Make use of re-sampling techniques to support your findings.

Data set

You will work with the data set found in "DataMatrix.txt". It is a data set comprising 7000 observations and 106 features. The first column is the label vector. There are 6 different classes in this data set.

Question 1

1a

Train multiple classifiers (at least 6, of varying degrees of complexity and character (parametric, nonparametric, linear, nonlinear,...) on multiple training-test splits of the data. Compare classification performance.

Pay particular attention to the following;

• imbalance between classes, how you handle this
• classification accuracy, specificity and sensitivity overall, and at the class level.

Be careful about using the full data set for any training/validation steps. For example, exploration and dimension reduction, if you choose to pursue this approach, should be done of the training data only.

Make sure to discuss:

Overall performance and how you assess this

*Which types of classifiers work best nd

Are all or only some classes well separated

Class imbalance, modeling assumptions, training metrics

The full training pipeline you set up and motivate all your choices

1b

Revisit task 1(a) and focus on model selection. Use different feature selection techniques (at least 3) and compare the results. You can use a subset of at least 3 classifiers from 1(a) for this task.

Discuss how many features are needed for good classification performance and how/if this differs between techniques.

Make sure to discuss

Stability of the feature selection - can you identify the important features with confidence? Explain why/why not?

If the important features differ between classes and how you assess this

1c

Revisit task 1(b) and perform a simulation study.

(I) Add more features to the data set where the features are noise features and uncorrelated - how does this affect the task in 1(b)?

(II) Add more features to the data set that are correlated with the original features - how does this affect the task in 1(b)?

Investigate at least 3 levels (number of features added).

Make sure to discuss:

What the impact of additional features are on task 1(b) and overall classification performance

How adding noise vs correlated features impact performance and selection stability

Classification and selection technique specific impact

Question 2

For this question you will explore the data in the context of clustering.

(2a)

Cluster the data set (remember to remove the label vector) using at least 6 different clustering techniques.

Make sure to discuss

How you select the number of clusters

If clusters overlap with the known classes for all or some of the classes and why you think that is

How stable the clustering outcome is

(2b)

Revisit task 1c in the context of clustering. How is clustering affected when you add noise features to the data set?

Make sure to discuss

How both selecting the number of clusters and cluster-class overlap and cluster stability is affected by the noise

(2c)

Finally, repeat task 1a and 1b but this time as a function sample size. That is, if you have less data to train on, how is performance affected?

Make sure to discuss

Deteriorating performance of classification as a function of sample size and compare classifiers

Impact on clusters - what happens when the sample size becomes smaller and smaller?

Compare impact on classification vs clustering