How to perform a classification or regression using k-NN?

Question

Suppose, you have given the following dataset where x and y are the 2 features and color Red or Blue is the target variable.

a) A new data point $x=1$ and $y=1$ is given. Using Euclidean distance in 3-NN, what you predict as the color for this data point?

Dataset

x	y	Color
-1	1	Red
0	1	Blue
0	2	Red
1	-1	Red
1	0	Blue
1	2	Blue
2	2	Red
2	3	Blue

 

b) Now assume we have the following dataset and the target value is the price. A new data point $x=1$ and $y=1$ is given. Using Euclidean distance in 3-NN. What would be the estimated price?

Dataset

x	y	Price
-1	1	$100
0	1	$50
0	2	$20
1	-1	$40
1	0	$30
1	2	$40
2	2	$70
2	3	$30

Answer 1

a) You can calculate the distances or simply visualize it:

(The code of above visualization is available here)

Based on the above visualization, 3 closest neighbors are blue points. Therefore, the predicted class is blue.

b) The 3 closest neighbors are data points: $(0,1,\$50)$, $(1,0,\$30)$, and $(1,2,\$40)$. Therefore, the estimated price is the mean of the target values $\frac{50+40+30}{3}=\$40$