Can I install this driver to the Intel TDX CVM guest kernel and use NVDIA's A100 series GPU graphics card to compute?

[fffmonkeyking-new]

Can I install this driver to the Intel TDX CVM guest kernel and use NVDIA's A100 series GPU graphics card to compute?

------------- Intel TDX CVM-------------------------
| LLM -> CUDA Toolkit ->This NVDIA Driver |PCI passthrough/vGPU-->PCIe-->| A100 GPU |

If not, what modifications do the drivers need to make?

[steven-bellock]

Can I install this driver to the Intel TDX CVM guest kernel and use NVDIA's A100 series GPU graphics card to compute?

No.

If not, what modifications do the drivers need to make?

All parts of the driver stack would need to be updated, RM/UVM/CUDA.

[steven-bellock]

Of course, I know and can accept the fact that the data is not safe after being copied from the CVM to the None CC GPU.

CPUs with Intel TDX support the provisioning of both confidential and non-confidential (legacy) VMs on the same system. Why can't you provision and use a non-confidential VM?

[fffmonkeyking-new]

Of course, I know and can accept the fact that the data is not safe after being copied from the CVM to the None CC GPU.

CPUs with Intel TDX support the provisioning of both confidential and non-confidential (legacy) VMs on the same system. Why can't you provision and use a non-confidential VM?

Because we have to let LLM run in TDX CVM to protect part of the data, and then LLM uses the other part of the data to do inference calculations on the GPU. However, due to cost reasons, we only use A100/T100 instead of Hopper series GPUs.

[steven-bellock]

The combination of encrypted VM but unencrypted PCIe will be a hard sell, especially since it requires driver changes, and introduces more test burden on QA. This would be bounce buffers without encryption / decryption, and other changes to the driver stack, including a new, somewhat insecure, mode of operation. With that said, we can register this as an informal feature request.

[fffmonkeyking-new]

The combination of encrypted VM but unencrypted PCIe will be a hard sell, especially since it requires driver changes, and introduces more test burden on QA. This would be bounce buffers without encryption / decryption, and other changes to the driver stack, including a new, somewhat insecure, mode of operation. With that said, we can register this as an informal feature request.

Thank you for your understanding of my use case.

I think it would be useful to combine Intel TDX CVM with Nvidia non-CC GPUs. It may not be entirely safe, but it could be considered as an option to Nvidia GPU more widely used.

The project https://github.com/JaewonHur/sev-snp-gpu can already allow LLM running in AMD SNP CVM to use Nvidia non-CC GPUs for inference calculations through Cuda Toolkit, but I don't know why.

I hope that with everyone's unremitting communication and efforts, we can see the LLM running in Intel TDX CVM using Nvidia non-CC GPUs for inference calculations through Cuda Toolkit as soon as possible.

[steven-bellock]

The project https://github.com/JaewonHur/sev-snp-gpu can already allow LLM running in AMD SNP CVM to use Nvidia non-CC GPUs for inference calculations through Cuda Toolkit, but I don't know why.

CUDA works with that? Interesting. It looks like it exclusively uses UVM to manage memory, since the patch in https://github.com/JaewonHur/sev-snp-gpu/blob/main/patches/open-gpu-kernel-modules-545.29.02.patch disables encryption / decryption in UVM.

[fffmonkeyking-new]

Another question, when we use Nvidia non-CC GPUs through Cuda Toolkit in Intel TDX CVM, why do we have to modify the Nvidia driver stack?
When running in CVM, what mechanism of Nvidia driver stack prevents us from successfully using non-CC GPUs?
For example, when Nvidia driver stack detects that it is running in CVM, it will perform PCIe encryption by default when doing cudaMemoryCpyHostToDevice?
I want to understand the root cause of the need to modify Nvidia driver stack

[steven-bellock]

why do we have to modify the Nvidia driver stack?

CPU-side software needs to be aware of protected and unprotected CPU memory. Data in protected memory needs to be explicitly moved to unprotected memory so that the GPU can access it.

[fffmonkeyking-new]

why do we have to modify the Nvidia driver stack?

CPU-side software needs to be aware of protected and unprotected CPU memory. Data in protected memory needs to be explicitly moved to unprotected memory so that the GPU can access it.

So in the solution of CVM passing through cc mode for Hopper series GPUs, how does the CPU-side software implement "needs to be aware of protected and unprotected CPU memory. Data in protected memory needs to be explicitly moved to unprotected memory so that the GPU can access it"?