> For the complete documentation index, see [llms.txt](https://2ood.gitbook.io/2ood-knowledge-base/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://2ood.gitbook.io/2ood-knowledge-base/lecture-notes/machine_learning/2024-09-09.md).

# 2024-09-09

## Introduction

> **Summary**

> **keywords**

> **TO-DO**

> **Homework**

> **Exercise**\*

> **Next time**

***

## Bias-Variance Trade-off

$$E(y\_0+\hat{f}(x\_0))^2 = Var(\hat{f}(x\_0)) + \[Bias(\hat{f}(x\_0))]^2 + Var(e)$$

Variance occurs because we make the function with only sampled datasets.\
Bias occurs because we try to make the function as simple as possible. (generally, flexible model has small bias)

![](https://3863748873-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7lhTZlN8P2dz9Om0Ykpg%2Fuploads%2Fgit-blob-45e6e3a9411739bd6f4c7f79aa6ee988a3d8e21f%2F20240909132158.png?alt=media)\
irreducible error is shown as dotted horiziontal line.

### Classification Problems

how are we going to classsify =/ldata in the greyszone\
l. miscalculating average clock

KNN stands for K- nearest neighbours. Check k nearest neighbors and see if which sided data is more

![20240909135611.png](https://3863748873-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7lhTZlN8P2dz9Om0Ykpg%2Fuploads%2Fgit-blob-65ee3b01602a73c5703ff80e12a25ed9726e0355%2F20240909135611.png?alt=media)\
KNN : K =1 is too flexible ; it is prone to outliers\
KNN : K = 100 is too rigid ; This has bigger bias??

**How can we decide the good k value?**

* save some portion of the data for test data.
* Use the other portion of data as a training data.
* The sample percentage will affect the choosing of $k$.![20240909135703.png](https://3863748873-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7lhTZlN8P2dz9Om0Ykpg%2Fuploads%2Fgit-blob-dee509cb122a6643de2695037b67399269ac36f4%2F20240909135703.png?alt=media)

***

## Linear Regression

Assume a model $$Y = \beta\_0 + \beta\_1X +\epsilon$$\
we want to find the intercept and slope.

We introduce a residue, the difference of the model-predicted data and the real one.$$e\_i = y\_i - \hat{y}\_i$$

Lets minimise the sum of squares. RSS(residue sum of squares).

$$RSS = (y\_1-\beta\_0-\beta\_1x\_1)^2 +(y\_2-\beta\_0-\beta\_1x\_2)^2 + \ldots + (y\_n-\beta\_0-\beta\_1x\_n)^2$$

By partial differentiating with $\beta\_0$ or $\beta\_1$, we can find the point where $RSS$ is smallest.

$$β\_1 = \frac{\sum^{n}*{i=1}(x\_i - \bar{x})(y\_i - \bar{y})}{\sum^{n}*{i=1}(x\_i - \bar{x})^2}$$ $$β\_0 = y − \beta\_1x$$\
95% confidence