Build The latest image

  • These notes were written after notes.md

  • Now that we have a GPU on GKE standard clusters that can run models we want to see if we can build an actual image

  • Hamel recommends using cog

  • Here’s his latest honeycomb model and cog file

  • Ended up building it using

    cog build -t mistral-vllm-awq

  • I had to install docker desktop since that’s what cog uses