Skip to content

Driver fails to allocate memory for decoder #848

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
1 of 2 tasks
x5f3759df opened this issue May 5, 2025 · 6 comments
Open
1 of 2 tasks

Driver fails to allocate memory for decoder #848

x5f3759df opened this issue May 5, 2025 · 6 comments
Labels
bug Something isn't working

Comments

@x5f3759df
Copy link

NVIDIA Open GPU Kernel Modules Version

575.51.03

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

  • I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Fedora Linux 42 (KDE Plasma Desktop Edition)

Kernel Release

Linux fedora 6.14.4-300.fc42.x86_64

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

  • I am running on a stable kernel release.

Hardware: GPU

GPU 0: NVIDIA GeForce RTX 5070 (UUID: GPU-0c71241b-6112-2e53-8d60-a2c6e634e234)

Describe the bug

Driver fails to allocate any memory for hw decoding. Even is memory is free. Logs from dmesg says that.

[  +0,000004] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000017] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[мая 5 10:37] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[  +0,000005] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000025] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[мая 5 10:43] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[  +0,000005] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000010] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[  +6,016231] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[  +0,000012] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000014] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[  +4,283316] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[  +0,000004] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000009] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[мая 5 10:45] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[  +0,000004] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000012] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[ +26,625894] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[  +0,000003] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000012] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[мая 5 10:58] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[  +0,000005] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000007] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[  +0,073746] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[  +0,000004] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[  +0,000007] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[  +0,106133] NVRM: GPU at PCI:0000:01:00: GPU-0c71241b-6112-2e53-8d60-a2c6e634e234
[  +0,000006] NVRM: GPU Board Serial Number: 0
[  +0,000001] NVRM: Xid (PCI:0000:01:00): 32, pid=22827, name=vo, Channel ID 0000003d intr0 00000000 intr1 80000000
[  +0,016731] NVRM: Xid (PCI:0000:01:00): 32, pid=22827, name=vo, Channel ID 0000003d intr0 00000000 intr1 80000000

For example - mpv

[vaapi] libva: /usr/lib64/dri/nvidia_drv_video.so init failed
[ffmpeg] av_log callback called with bad parameters (NULL AVClass).
[ffmpeg] This is a bug in one of FFmpeg libraries used.
[ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST
[ffmpeg/video] h264: get_buffer() failed
[ffmpeg/video] h264: decode_slice_header error
[ffmpeg/video] h264: no frame!
Error while decoding frame (hardware decoding)!
[ffmpeg] av_log callback called with bad parameters (NULL AVClass).
[ffmpeg] This is a bug in one of FFmpeg libraries used.

BUT next try it will be okay. Issue is not consistent. Sometimes it may happen right after suspend wake up.

To Reproduce

Try to use player with hardware decoding support (mpv, firefox, etc).

Bug Incidence

Sometimes

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

No response
@x5f3759df x5f3759df added the bug Something isn't working label May 5, 2025
@LesterLyu
Copy link

LesterLyu commented May 16, 2025 •
edited
Loading

Got the same random issue on 5060Ti on 575.51.02 when triggering ffmpeg decoding.

[52595.758098] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[52595.759943] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[52595.760899] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[52611.858258] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[52611.860142] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[52611.861108] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647
[52620.465122] NVRM: nvCheckOkFailedNoLog: Check failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from _memdescAllocInternal(pMemDesc) @ mem_desc.c:1353
[52620.466954] NVRM: sysmemConstruct_IMPL: *** Cannot allocate sysmem through fb heap
[52620.468136] NVRM: nvAssertOkFailedNoLog: Assertion failed: Out of memory [NV_ERR_NO_MEMORY] (0x00000051) returned from pRmApi->Alloc(pRmApi, device->session->handle, isSystemMemory ? device->handle : device->subhandle, &physHandle, isSystemMemory ? NV01_MEMORY_SYSTEM : NV01_MEMORY_LOCAL_USER, &memAllocParams, sizeof(memAllocParams)) @ nv_gpu_ops.c:4647

nvidia-bug-report.log.gz

@pinkycollie
Copy link

pinkycollie commented May 16, 2025 via email

@x5f3759df
Copy link
Author

My VRAM was free for that moment. At least more than 8+ gigs I guess.

@LesterLyu
Copy link

I’ve found a setup that greatly improves stability:

  • Boot in UEFI mode with CSM disabled, and enable Resizable BAR.
  • Add the kernel parameters cma=2048M nvidia_drm.modeset=1 to GRUB.
  • If you use ZFS, cap ARC to a reasonable size for your RAM—for example, zfs.zfs_arc_max=8589934592 (8 GiB).
  • Before you activate the GPU, keep at least 6 GiB of free system RAM. The driver often fails when caches and buffers consume most of the memory, suggesting it has low allocation priority. When memory is tight, running echo 3 > /proc/sys/vm/drop_caches frequently resolves the issue.
  • And this is not a VRAM issue.

@Cloudwalk9
Copy link

The driver itself seems to be running out of kernel memory. Definitely not a VRAM issue.

@x5f3759df
Copy link
Author

Looks like not a VRAM issue indeed. Not good news, because I have 32 gigs of RAM - this is strange

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Cloudwalk9 @x5f3759df @LesterLyu @pinkycollie