Error Running Crash Analyzer in OSS-Fuzz-Gen Experiments

[AmPaschal]

Summary

I have noticed the following errors when using the crash analyzer during OSS-Fuzz-Gen experiments. These errors are frequent and commonly lead to incorrect root cause analysis of the crash.

1. GDB complains that it cannot insert breakpoint because it cannot access memory at the breakpoint's address.
   https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-16-1148-pamusuo-analyzer-tests-1/sample/output-libraw-_zn6libraw17crxloaddecodeloopepvi/08.html
   https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-16-1148-pamusuo-analyzer-tests-1/sample/output-libraw-_zn6libraw17crxloaddecodeloopepvi/03.html

2. GDB complains that the artifact directory was not found.
   https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-16-1148-pamusuo-analyzer-tests-1/sample/output-astc-encoder-_z20symbolic_to_physicalrk21block_size_descriptorrk25symbolic_compressed_blockph/03.html
   https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-16-1148-pamusuo-analyzer-tests-1/sample/output-libsndfile-sf_open/10.html (crash in cycle 1)

3. The LLM hallucinates GDB interactions and uses this hallucinated interaction to derive its conclusion.
   https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-16-1148-pamusuo-analyzer-tests-1/sample/output-mosh-_zn8terminal8emulator6resizeemm/10.html
   https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2025-07-16-1148-pamusuo-analyzer-tests-1/sample/output-quickjs-js_parsejson2/07.html

Why these issues are important

Agents in OSS-Fuzz-Gen, specifically the Context Analyzer and the Enhancer, rely on the root cause analysis from the crash analyzer to either determine the feasibility of the crash or modify the fuzz driver to prevent similar crashes. Hence, when the crash analyzer is wrong, the context analyzer is usually also wrong (and reports false crashes as being feasible or vice versa) or the enhancer fails to fix the crash in the next cycle.

Root Causes

After some investigation and debugging (with the help of LLM and the new capability of running individual agents), I think these are the root causes (but I may be wrong).

Error 1: GDB complains that it cannot insert the breakpoint because it cannot access memory at the breakpoint's address.

There are two factors that cause this error:

1. The oss-fuzz project binary is not compiled using debug flags.
   While there are two lines of code that add debug flags to the compile flags, the docker exec command used to add these debug flags creates a new terminal instance during each invocation. Hence, the modified CFLAGS and CXXFLAGS are not persisted when the compile command is executed.

2. The oss-fuzz project with GDB that is created by the create_ossfuzz_project_with_gdb function, and which has the necessary debug flags set as environment variables, is not actually used by the GDB tool. Instead, a new oss-fuzz project image is created in the _prepare_project_image function, without the necessary debug flag.

Error 2: GDB complains that the artifact directory was not found.

There are also two factors that cause this error:

1. The artifact (and the fuzz driver) is not copied into the oss-fuzz directory. This occurs because of the error above where the oss-fuzz project that is created for the GDB tool is not actually used by the Crash Analyzer. Instead, in the actual oss-fuzz project that is used, both the artifact and the fuzz driver that caused the crash are not copied into the docker container.

2. The oss-fuzz project created by the create_ossfuzz_project_with_gdb function correctly copies the artifact into a /artifact/ directory in the docker container provided to the GDB tool. However, the prompt for the GDB tool still references the artifact path in the experiment's working directory (/experiment/results directory). Hence, even if the artifact exists in the docker container, the LLM will try to run the compiled oss-fuzz project using the wrong artifact path.

Error 3: The LLM hallucinates GDB interactions and uses this hallucinated interaction to derive its conclusion.

The LLM hallucinates by nature when generating its response. There is currently no validation to prevent this hallucination or ensure the LLM is not producing a response that contains supposed outputs from the GDB tool.