2023-Q3-AI 4. Regression, Classification - Pytorch

4.1. Video / Materials

Video (19 Jul 2023, 13:00):

https://youtube.com/live/EvNawwH_mYU?feature=share

Jamboard: https://jamboard.google.com/d/1jEqwMfh9FZBYE_H2gYX-BUyjqMOlCl_54p6hk-5KVo4/edit?usp=sharing

Materials: https://pytorch.org/tutorials/beginner/nlp/deep_learning_tutorial.html

Priekš Youtube Live, izmantojot OBS streaming software

Youtube key: 6z53-3r17-a8tu-u50j-1xgq (cits key) RTMP: rtmp://a.rtmp.youtube.com/live2

Iepriekšējā gada

Video: https://youtu.be/Ejv260iiVfo

Jamboard: https://jamboard.google.com/d/1iiLPYbVyYwc9RM35fSTUWXH0gvh6POnZ73hveOeT7mk/edit?usp=sharing

xxxxxxxxxx
4
1
Diesel    4402
2
Petrol    3631
3
CNG         57
4
LPG         38

4.2. Implement a PyTorch version for the regression task

Use the reference video 6.1 and the template. Implement:

1. Create an Embedding Matrix for categorical input data.

2. Implement the MSE error function.

3. Implement Huber loss as an error function http://share.yellowrobot.xyz/upic/753bc664e86ca87b3c714d4215254851_1676800801.jpg

Submit a screenshot with results and source code!

Template:

http://share.yellowrobot.xyz/quick/2023-2-19-7A72ACC5-39F4-44EB-8223-374D2FF9ED8D.zip

4.3. Implement a PyTorch version for the classification task

Tasks:

1. Implement Cross-Entropy Loss (must not use the built-in loss function) - http://share.yellowrobot.xyz/upic/c83a2498563001a2e30acb1f40c97aa6_1676800812.jpg

2. Implement the Accuracy metric and chart - http://share.yellowrobot.xyz/upic/fcec10b94af5e36ab374e8d29e6811d6_1676800827.jpg

3. Implement the Confusion matrix (https://towardsdatascience.com/confusion-matrix-for-your-multi-class-machine-learning-model-ff9aa3bf7826)

4. Implement the F1-score metric - http://share.yellowrobot.xyz/upic/cb03b2e54de0e41873f94cf6ab81ed72_1676800842.jpg

Submit a screenshot with results and source code!

Template:

http://share.yellowrobot.xyz/quick/2023-2-19-13E1F9A9-F0AF-4036-A1C0-DE62F27E0FF3.zip

4.4. Implement a PyTorch version for the binary classification task

Tasks:

1. Implement Binary-Cross-Entropy Loss (must not use the built-in loss function) http://share.yellowrobot.xyz/upic/8c77d1b6f1d5e3c44febd63f0fecb9fd_1676800928.jpg

2. Implement the BatchNormalization function in PyTorch http://share.yellowrobot.xyz/upic/8b4284380b4fc8c31ce424ba10f9da4f_1676800938.jpg

Submit a screenshot with results and source code!

Template:

http://share.yellowrobot.xyz/quick/2023-2-19-9B05C46C-8CB9-4228-A190-C77F6218B051.zip

4.5. Homework - Implement a numpy version for the classification task

Tasks:

1. Implement the SoftMax function and its derivative http://share.yellowrobot.xyz/upic/7cea763fedc5123ae886830fe3622c82_1676801194.jpg

2. Implement the CCE loss function and its derivative - http://share.yellowrobot.xyz/upic/7389daf82d3fee898d5e87e68d98cb70_1676800994.jpg

Submit a screenshot with results and source code!

Template:

http://share.yellowrobot.xyz/quick/2023-2-19-0BCA422E-0120-4165-89DD-A4E8C2DBB024.zip

Accuracy formula

$acc=\frac{1}{N}\frac{\sum correct}{\sum all}$$acc = \frac{1}{N} \frac{\sum correct}{\sum all}$

Huber loss

${\mathcal{L}}_{huber}(y,y^{\prime })=\frac{1}{N}\sum {\delta }^2(\sqrt{1+(\frac{y-y^{\prime }}{\delta }{)}^2}-1)$$\mathcal{L}_{huber}(y, y') = \frac{1}{N} \sum \delta^2 (\sqrt{ 1 + (\frac{y - y'}{\delta})^2} - 1)$

CCE formula

${\mathcal{L}}_{CCE}(y,y^{\prime })=\frac{1}{N}\sum -w_{n}ylog(y^{\prime }+ϵ)$$\mathcal{L}_{CCE}(y, y') = \frac{1}{N} \sum - w_n \; y \; log(y' + \epsilon)$

F1 formula

$F1=\frac{1}{N}\sum \frac{2TP}{2TP+FP+FN}$$F1 = \frac{1}{N} \sum \frac{2TP}{2TP + FP + FN}$

BCE formula

${\mathcal{L}}_{BCE}(y,y^{\prime })=-\frac{1}{N}\sum w_{n}ylog(y^{\prime }+ϵ)+(1-y)log(1-y^{\prime }+ϵ)$$\mathcal{L}_{BCE}(y, y') = - \frac{1}{N} \sum w_n \; y \; log(y' + \epsilon) + (1 - y) \; log(1 - y' + \epsilon)$

BatchNormalization formula

Softmax formula derivative

$\begin{array}{c}SoftMax(x)=\frac{e^{x_i}}{\sum _{k=1}^K{e}^{x_k}}=\left[\begin{array}{ccccc}{a}_0& a_1& a_2& a_3& a_4\end{array}\right]\\ \frac{SoftMax(x)}{\partial x}=\left[\begin{array}{ccccc}{a}_0(1-a_0)& -a_0{a}_1& -a_0{a}_2& -a_0{a}_3& -a_0{a}_4\\ -a_1{a}_0& a_1(1-a_1)& -a_1{a}_2& -a_1{a}_3& -a_1{a}_4\\ -a_2{a}_0& -a_2{a}_1& a_2(1-a_2)& -a_2{a}_3& -a_2{a}_4\\ -a_3{a}_0& -a_3{a}_1& -a_3{a}_2& a_3(1-a_3)& -a_3{a}_4\\ -a_4{a}_0& -a_4{a}_1& -a_4{a}_2& a_4{a}_3& a_4(1-a_4)\end{array}\right]\end{array}$$SoftMax(x) = \frac{e^{x_i}}{\sum_{k=1}^K e^{x_k}} = \begin{bmatrix} a_0 & a_1 & a_2 & a_3 & a_4 \end{bmatrix} \\ \frac{SoftMax(x)}{\partial x} = \begin{bmatrix} a_0 (1 - a_0) & -a_0 a_1 & -a_0 a_2 & -a_0 a_3 & -a_0 a_4 \\ -a_1 a_0 & a_1 (1 - a_1) & -a_1 a_2 & -a_1 a_3 & -a_1 a_4 \\ -a_2 a_0 & -a_2 a_1 & a_2 (1 - a_2) & -a_2 a_3 & -a_2 a_4 \\ -a_3 a_0 & -a_3 a_1 & -a_3 a_2 & a_3 (1 - a_3) & -a_3 a_4 \\ -a_4 a_0 & -a_4 a_1 & -a_4 a_2 & a_4 a_3 & a_4 ( 1- a_4) \\ \end{bmatrix}$

CCE formula derivative

$\begin{array}{c}{\mathcal{L}}_{CCE}(y,y^{\prime })=\frac{1}{N}\sum -ylog(y^{\prime }+ϵ)\\ \frac{{\mathcal{L}}_{CCE}(y,y^{\prime })}{\partial y^{\prime }}=\frac{1}{N}\sum -y\cdot \frac{1}{y^{\prime }}=\frac{1}{N}\sum -\frac{y}{y^{\prime }}=\frac{1}{N}\sum -\frac{y}{y^{\prime }+ϵ}\end{array}$$\mathcal{L}_{CCE}(y, y') = \frac{1}{N} \sum - y \; log(y' + \epsilon)\\ \frac{\mathcal{L}_{CCE}(y, y')}{\partial y'} = \frac{1}{N} \sum -y \cdot \frac{1}{y'} = \frac{1}{N} \sum - \frac{y}{y'} = \frac{1}{N} \sum - \frac{y}{y' + \epsilon}$