GPUS

Deploying Hugging Face Models Using Triton Inference Server

• Triton Inference Server

Troubleshooting

May need to set LD_LIBRARY_PATH

export LD_LIBRARY_PATH=/usr/local/nvidia/lib64

• TODO(jeremy): Its missing the CUDA location ## DeviceQuery

I think this is a sample in the CUDA code link

I think it can be used as a smoke test for GPU accessibility.

References

GPUS in Containers (2018)