<aside> 📖

https://github.com/ml5js/Intro-ML-Arts-IMA-F24/tree/main/06-transformers-js

</aside>

Project Description:

For this project, I wanted to modify the depth estimation sample to be able for it to be a bit more interactive in the form modifying the depth values based on a slider, so the image slowly revelas itself

The Progress:

step 01: run the code to see what it does - after swithcing from “webgpu” to “wasm” I got it to run

(my guess is that webGPU doesn’t support apple silicon GPU-s / the METAL framework)

step02: familiarise myself with the code and occasially consult the documentation for aspects such as how the pixel array (4 values - one dimensional array) works in P5

step03: rewrite the estimateDepth function to work standalone - with input parameters and returns - as this was required for me to be able to store the depth map

step04: realising it’s very slow, so I go onto HuggingFace to find a smaller version of the model

• tried out many similar models: onnx-community/dpt-dinov2-small-nyu

The error message wasn’t very descriptive, so I couldn’t fix this issue

• tried other models as well, which did run ( Xenova/dpt-hybrid-midas ) but weren’t any faster, so I reverted back to the original