-
Notifications
You must be signed in to change notification settings - Fork 5
[BUG] not enough memory, while I have plenty #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Can you try my new f16 deployment? https://github.com/wpmed92/stable-diffusion-tinygrad-f16 |
hI @wpmed92, Chrome Versie 131.0.6778.205 (Officiële build) (arm64) gives me the following:
at
The AI feature of Chrome Developer Console suggests:
executing the suggested code above gives:
The code I posted earlier const k1Gig = 1024 * 1024 * 1024 * 2;
const adapter = await navigator.gpu?.requestAdapter();
const device = adapter?.requestDevice({
requiredLimits: { maxBufferSize: k1Gig },
requiredFeatures: [ 'float32-filterable' ],
}); gives this information GPUSupportLimits
maxBindGroups: 4
maxBindGroupsPlusVertexBuffers: 24
maxBindingsPerBindGroup: 1000
maxBufferSize: 2147483648
maxColorAttachmentBytesPerSample: 32
maxColorAttachments: 8
maxComputeInvocationsPerWorkgroup: 256
maxComputeWorkgroupSizeX: 256
maxComputeWorkgroupSizeY: 256
maxComputeWorkgroupSizeZ: 64
maxComputeWorkgroupStorageSize: 16384
maxComputeWorkgroupsPerDimension: 65535
maxDynamicStorageBuffersPerPipelineLayout: 4
maxDynamicUniformBuffersPerPipelineLayout: 8
maxInterStageShaderComponents: 64
maxInterStageShaderVariables: 16
maxSampledTexturesPerShaderStage: 16
maxSamplersPerShaderStage: 16
maxStorageBufferBindingSize: 134217728
maxStorageBuffersPerShaderStage: 8
maxStorageTexturesPerShaderStage: 4
maxSubgroupSize: 4294967295
maxTextureArrayLayers: 256
maxTextureDimension1D: 8192
maxTextureDimension2D: 8192
maxTextureDimension3D: 2048
maxUniformBufferBindingSize: 65536
maxUniformBuffersPerShaderStage: 12
maxVertexAttributes: 16
maxVertexBufferArrayStride: 2048
maxVertexBuffers: 8
minStorageBufferOffsetAlignment: 256
minSubgroupSize: 4294967295
minUniformBufferOffsetAlignment: 256 Does this help? |
Can you try commenting out the max storage binding size? |
@wpmed92 do you mean |
Yes, curious if removing that solves your problem |
@wpmed92 Is that possible? |
@wpmed92 I've now installed a local web server and deleted the line of code: requiredLimits.maxStorageBufferBindingSize = maxBufferSizeInSDModel; It downloaded the 2.5GB safetensor file. It executes fast: And gives lots of errors in the console. First error:
Last error:
|
Yeah that step time is not relevant since you have validation errors. Unfortunately the buffers used by some kernels are larger than supported on your system. This requires a refactor in tinygrad to handle it somehow i think. |
@wpmed92 const maxBufferSizeInSDModel = 134217728;
requiredLimits.maxStorageBufferBindingSize = maxBufferSizeInSDModel;
requiredLimits.maxBufferSize = maxBufferSizeInSDModel; It re-downloads the safetensor file (so it's not in a PWA cache?)
|
Hmm.. while 128MB is actually the default setting of WebGPU. Looking at https://chat.webllm.ai/ this works.
that refers to https://chat.webllm.ai/sw.js these lines if (1073741824 > A.limits.maxStorageBufferBindingSize && (console.log(`Requested maxStorageBufferBindingSize exceeds limit.
requested=${computeMB(g)},
limit=${computeMB(A.limits.maxStorageBufferBindingSize)}.
WARNING: Falling back to ${computeMB(134217728)}...`),
g = 134217728, Does this help? |
@wpmed92 that project https://github.com/mlc-ai/web-llm-chat even works on mobile phones. Is ot possible to do similar with this project? |
@wpmed92 So if I understand you remark correctly, the code in the .js files is automatically generated using tinygrad and is very specific for the model. If you want an other SD model, even if it's fl16, the code needs to be regenerated, because lots of things are hardcoded or not dynamically coded (like the file size of the safetensors file and some derivitives of them). An thus, updating the code to enable support for 128MB maxStorageBufferBindingSize needs to be a change in tinygrad. Thanks! |
Uh oh!
There was an error while loading. Please reload this page.
I have the same issue as mentioned here:
https://www.reddit.com/r/webgpu/comments/1bzzul0/binding_size_141557760_of_buffer_is_larger_than/
But code like this
Works without a problem.
Resulting in a maxBufferSize of 2147483648 ~= 2048MB ~= 2GB available (2x the memory need by the model) instead of the 128MB the error claims is the maximum.
Can you fix the memory reservation?
I would love to use your tool.
By the way
shader-f16
is available in my Chrome 131.0.6778.109 (Officiële build) (arm64), when WebGPU is enabled.Testen on
Qualcomm X Elite, 32GB (with 16GB allocated to the NPU/GPU) => architecture: "adreno-8xx"
Chrome 131.0.6778.109 (Officiële build) (arm64), with WebGPU is enabled.
The text was updated successfully, but these errors were encountered: