Add support for transient textures on Vulkan and Metal

[opstic]

Connections
Fixes #8239

Description
Exposes MTLStorageMode.memoryless for Metal and vk::MemoryPropertyFlags::LAZILY_ALLOCATED for Vulkan by adding TextureUsages::TRANSIENT.

Testing
Added corresponding TextureUsage flag to relevant examples.
Also added some render tests and validation testing.

Squash or Rebase?
Squash

Checklist

• Run cargo fmt.
• Run taplo format.
• Run cargo clippy --tests. If applicable, add:
  • --target wasm32-unknown-unknown
• Run cargo xtask test to run tests.
• If this contains user-facing changes, add a CHANGELOG.md entry.

[cwfitzgerald]

Thanks for taking this on! It just so happened we just talked about wanting this in the WebGPU face to face, so super happy to have this! There's a few things that need to be added, but I don't think it will end up being too much work:

1. At the face to face, we've decided that transient textures wouldn't be a feature. It would be available everywhere, but a no-op on unsupported platforms. If we wanted to expose to the user if this is supported, we can do this in the AdapterInfo as a hint.
2. We need two pieces of validation:
   a. If you add the transient attachment usage, the usage must be exactly RENDER_ATTACHMENT | TRANSIENT
   b. When writing a renderpass to these attachments, the StoreOp must be discard.
3. With point 1 implemented, we should stick these flags on as many of the examples as relevant as we really want to encourage its use.
4. On VK, we need to add VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT to the memory so that it may be lazily allocated. I think you may be getting this already implicitly without explicitly asking for it, if you're seeing memory improvements.
5. We should add some tests
   a. For all the validation I mentioned, these can be simple tests in wgpu-validation.
   b. We should have at least one test in the wgpu-gpu tests which ensures we don't get validation errors from vk/metal.

Let me know if you have any questions or if any of this is unclear!

[opstic]

Thanks for your input @cwfitzgerald!

For:

2. I added some validation and error types specific to the situations you mentioned, please check it again!

3. The most relevant looks to only be the msaa_line example, I looked through all of the other examples and it appears they don't use multisampling at all. I'll be adding it now.

4. Something I found was that none of the desktop GPUs I have supports memory types with LAZILY_ALLOCATED, the memory decrease must just have been run-to-run difference😭, I'll be proceeding by testing more carefully and on mobile from now on.

[cwfitzgerald]

Re 3: shadow can use it for the forward depth buffer, and I suspect there might be an extra example or two which only ever uses depth in a memoryless compatible way.

[cwfitzgerald]

Something I found was that none of the desktop GPUs I have supports memory types with LAZILY_ALLOCATED, the memory decrease must just have been run-to-run difference😭, I'll be proceeding by testing more carefully and on mobile from now on.

Yeah this is unfortunately a mobile only feature really, but hopefully the validation layers can save us on desktop. Apple is the main one that really benefits from it on "desktop".

[opstic]

Yeah this is unfortunately a mobile only feature really, but hopefully the validation layers can save us on desktop. Apple is the main one that really benefits from it on "desktop".

I did make it so that a if a memory type with the LAZILY_ALLOCATED bit was not found, it would just proceed normally with the TextureUsages::TRANSIENT being disregarded, just making sure that this behavior should be the intended one.

[cwfitzgerald]

Some comments and needs tests, but moving in the right direction!

[opstic]

Re 3: shadow can use it for the forward depth buffer, and I suspect there might be an extra example or two which only ever uses depth in a memoryless compatible way.

Alright, I looked through it more carefully again, we should be able to apply TRANSIENT on msaa_line's resolve texture and shadow, skybox's depth texture. Adding them now.

[JMS55]

What's the use case for this?

One thing I do in Bevy is use a dummy render target for a render pass, since I use storage atomics for my output. I have wanted render passes with no render attachments for this reason, so that we don't need to waste memory on a dummy render target. Would this be a suitable replacement?

[cwfitzgerald]

What's the use case for this?

Tilers where you need an attachment but never actually read from it from memory. You can avoid allocating that memory.

One thing I do in Bevy is use a dummy render target for a render pass, since I use storage atomics for my output. I have wanted render passes with no render attachments for this reason, so that we don't need to waste memory on a dummy render target. Would this be a suitable replacement?

This is better than nothing, but not really. It would only actually reduce memory on tilers.

[opstic]

One final thing, you should do a pass over the backends and make sure transient is advertised https://github.com/gfx-rs/wgpu/blob/trunk/wgpu-hal/src/lib.rs#L670-L673 wherever RENDER_ATTACHMENT is advertised.

I checked and it appears the only usage of texture_format_capabilities seems to be in get_texture_format_features, which is then used in other places, so I think it should be sufficient to advertise it there.

[cwfitzgerald]

Sure, that works

[cwfitzgerald]

Great stuff!