2022-04-25 Meeting #33 - PPO - Contrastive Loss - Siamese Network

TODO

Youtube Video

Implement classification model based on template, complete TODOs, do not use loss function for test dataset, because it has different classes.

Implement Contrastive Loss and Center of the mass classification

using template from 1.2. task

Add on top of embedding z, projection to logits and calculate CCE composite loss. Often in practice combination of both loss functions work better.
Implement following structure of model:
http://share.yellowrobot.xyz/1636095494-dml-course-2021-q4/homework_1_4.png
For encoder implement pre-trained DenseNet from torchvision
Change metric for distances from L2 Euclidean distance to Cosine Distance using following formula (change it for training and also for class evaluation/matching): http://share.yellowrobot.xyz/1636095494-dml-course-2021-q4/cosine_dist.png

More on metric functions:
https://github.com/taki0112/Vector_Similarity
Train on Google Colab or on VeA GPU nodes, submit code and screenshots of results

Cosine similarity range: −1 meaning exactly opposite, 1 meaning exactly the same, 0 indicating orthogonality.

cosine distance

\begin{matrix} L = \sum | | z_{a} - z_{p} | |^{2} + m a x ((m - | | z_{a} - z_{n} | |^{2}), 0) - C_{c o e f} \sum l o g (y_{y_{i d x}}^{'} + ϵ) \\ D = 1 - \frac{z_{a} \cdot z_{b}}{| | z_{a} | | | | z_{b} | |} = 1 - \frac{\sum z_{a} z_{b}}{\sqrt{\sum z_{a}^{2}} \sqrt{\sum z_{b}^{2}}} \end{matrix}