How to calculate Softmax Regression probabilities in this example?

Question

The scatter plot of Iris Dataset is shown in the figure below. Assume Softmax Regression is used to classify Iris to Setosa, Versicolor, or Viriginica using just petal length and petal width. If  weights required for Softmax Regression initialized to 1 for class Setosa, 2 for class Versicolor, and 3 for Virginica,

1) What will be the probability of an iris with petal length = 4.6  and petal width = 1.7 to be classified as Virginica? 

2) What will be the probability of Virginica, if we use all features petal length = 4.6  and petal width = 1.7, sepal length = 5.5 and sepal width = 3.0 with the same weight initialization?

 

Answer 1

Please check the following answer:

Answer 2

Class 1: Setosa

Class 2: Versicolor

Class 3: Virginica

The initialized weights:

$w_{01} = w_{11}=w_{21} = 1$

$w_{02} = w_{12}=w_{22} = 2$

$w_{03} = w_{13}=w_{23} = 3$

The weight equations:

$z_1 = x_0w_{01} + x_1 w_{11} + x_2w_{21}$

$z_2 = x_0w_{02} + x_1 w_{12} + x_2w_{22}$

$z_1 = x_0w_{03} + x_1 w_{13} + x_2w_{23}$

1) $x_0 = 1 \quad x_1 = 4.6 \quad \text{and} \quad x_2 = 1.7$

$z_1 = w_{01} + x_1 w_{11} + x_2w_{21} = 1 + 4.6 + 1.7 = 7.3$

$z_2 = w_{02} + x_1 w_{12} + x_2w_{22} = 2 + 4.6(2) + 1.7(2) = 16.3$

$z_3 = w_{03} + x_1 w_{13} + x_2w_{23} = 3 + 4.6(3) + 1.7(3) = 21.9$

$e^{z_1} + e^{z_2} + e^{z_3} = e^{7.3} + e^{16.3} + e^{21.9} = 1480.3 + 11994994 + 3243763284 = 3255759758$

$p^3 = \frac{e^{z_3}}{\sum_{i=1}^3 e^{z_i}} = \frac{3243763284}{3255759758} = 0.996315307$

$\therefore$ probability to classify to virginica is 99.6%

2) $x_0 = 1 \quad x_1 = 4.6 \quad  x_2 = 1.7 \quad x_3 = 5.5 \quad x_4 = 3$
$\begin{align*} z_1 &= x_0 + w_{11}x_1 + w_{21}x_2 + w_{31}x_3 + w_{41}x_4 \\ &= 1 + 4.6 + 1.7+5.5+3\\ &=15.8\end{align*}$

$\begin{align*} z_2 &= x_0 + w_{12}x_1 + w_{22}x_2 + w_{32}x_3 + w_{42}x_4 \\ &= 2 + (2)4.6 + (2)1.7+(2)5.5+(2)3\\ &=31.6 \end{align*}$

$\begin{align*} z_3 &= x_0 + w_{13}x_1 + w_{23}x_2 + w_{33}x_3 + w_{43}x_4 \\ &= 3 + (3)4.6 + (3)1.7+(3)5.5+(3)3\\ &=47.4 \end{align*}$

$\begin{align*}\sum_{i=1}^3 e^{z_i}&=e^{z_1} + e^{z_2} + e^{z_3}\\ &= e^{15.8} + e^{31.6} + e^{47.4} \\&= 7275332 + 5.29e13 + 3.85e20 \\&= 3.850866845e20\end{align*}$

$p^3 = \frac{e^{z_3}}{\sum_{i=1}^3 e^{z_i}} = \frac{3.850866316e20}{3.850866845e20} = 0.999999863$

$\therefore$ probability to classify to virginica is 99.9%