2023-Q3-AI 6. Image Segmentation - UNet, YOLO

6.1. Video / Materials

Video (20 Jul 2023, 13:00):

https://youtube.com/live/dGAjhDr_crA?feature=share

Jamboard:

https://jamboard.google.com/d/1TzNY1fJUQlpdJ1qKcuNMcT5Ehj9ts3F20MD-xMppnhU/edit?usp=sharing

Materials: * UNet https://arxiv.org/pdf/1505.04597.pdf * UNet++ https://arxiv.org/pdf/1807.10165 * UNet3+ https://arxiv.org/pdf/2004.08790

Ir iedota pieeja jamboard un ar OBS jāveic screen streaming uz šādu setting

OBS key: acgp-tv3b-sz8z-mcd6-0jy2

Par katru uzdevumu dodam 100 punktus

6.2. Implement UNet Model's Forward Function with Concatenation

The source code template is available here: http://share.yellowrobot.xyz/quick/2023-3-27-0B47CEFD-1556-4BDA-85B0-F761C20418F8.zip

The equations are available here: http://share.yellowrobot.xyz/upic/1373da10bfdd8355d05dea75fdedbe04_1679908309.jpg

Submit the source code and a screenshot of the results.

6.3. Implement UNet Model's Forward Function with Addition

Use the previous task's template.

The equations are available here: http://share.yellowrobot.xyz/upic/0616992f53b1702034d5e6e8156abe5a_1679908378.jpg

Submit the source code and a screenshot of the results.

6.4. Implement YOLO-Based Image Instance Segmentation

Use the ready-made PyTorch library: https://docs.ultralytics.com/yolov5/tutorials/pytorch_hub_model_loading/#pandas-results

Submit the source code and a screenshot of the results.

6.5. Homework. Implement DICE Error and IoU Metric

1. Add DICE error to the previous task - use it as a loss function (create a composite loss function with coefficients for each part of the loss function).

   http://share.yellowrobot.xyz/upic/1ecc52854a8faeee519222fe65dd8740_1679908662.jpg

2. Add the IoU metric, visualize it graphically (Jaccard index).

   http://share.yellowrobot.xyz/upic/917ee508198be0d52f23605e033c8e15_1679908674.jpg http://share.yellowrobot.xyz/upic/3956b422a5ce3e2163bef451765a6347_1679908683.jpg

3. Train the model.

Submit screenshots and source code.

Iepriekšējais video: https://youtu.be/4AR8lbVMAoQ

Iepriekšejais jamboard: https://jamboard.google.com/d/1qj3wlfYefiaFvOtyozgizPKpXHoHT4Y-qtTOfYGu1GE/edit?usp=sharing

Saturs

1. ⚠️ Atšķirībā no iepriekšējiem video, segmentācija būs tiks veikta uz Koka dēļu bojājumu dataset https://github.com/evaldsurtans/llu-wood-defect-dataset

2. Izstāstīt par Semantic Segmentation / UNet

3. Likt pašiem implementēt UNet ar concat

4. Kad netiek galā parādīt kā implementēt

5. Likt pašiem implementēt UNet ar addition

6. Kad netiek galā parādīt kā implementēt

7. Pastāstīt atšķirības starp, semantic segmentation un instance segmentation (UNet vs YOLO)

8. Implementēt YOLO balstītu segmentāciju

   1. Pastāstīt modeļa outputs un loss function

   2. Pastāstīt Non-Maximum Surpression algo

9. Pastāstīt par mājasdarbu

Slide materials

https://manipulation.csail.mit.edu/segmentation.html

Materiāli