Compare the performance of ffmpeg and pyav

[Globefishp]

I am using pyav to help me encode raw video stream from a industrial camera. The camera API can output frames (numpy array) and timestamp to keep video sync to the actual time. That is, although my camera is set to 60fps, the actual fps would always be slightly faster than 60 (60.47 maybe? but it's hard to predict the behaviour of my camera). PyAV gave me the chance to set pts, which I found it's hard to set using ffmpeg directly.

But I noticed that when encoding, the CPU usage is only 35%, while 50% when using ffmpeg. And the encoding speed is also ~75% of the newest ffmpeg(7.1.1). I am using libx264 with medium preset, and the threads are set to 0(auto). I am using i9-14900K with dual channel DDR5 4800MHz.
Here's example code for PyAV encoding:

                for frame_batch_ndarray in frames:
                    # Iterate through each frame in the current ndarray batch
                    for i in range(frame_batch_ndarray.shape[0]):
                        total_frames_processed += 1
                        single_frame_data = frame_batch_ndarray[i] # This is (H, W, C)
                        
                        # Time VideoFrame.from_ndarray()
                        start_time = time.perf_counter()
                        av_frame = av.VideoFrame.from_ndarray(single_frame_data, format='bgr24')
                        total_from_ndarray_time += (time.perf_counter() - start_time)

                        # Time stream.encode()
                        start_time = time.perf_counter()
                        packets = list(self._stream.encode(av_frame)) # Materialize to measure encode time accurately
                        total_encode_time += (time.perf_counter() - start_time)

                        # Time muxing
                        start_time = time.perf_counter()
                        for packet in packets:
                            # Mux the packet into the container
                            self._container.mux(packet)
                        total_mux_time += (time.perf_counter() - start_time)

When encode 1000 random noise image, the result shows the encode method is the bottleneck (95% time).

Total time in VideoFrame.from_ndarray: 0.4021s (Avg: 0.000402s/frame)
Total time in stream.encode:          13.2390s (Avg: 0.013239s/frame)
Total time in container.mux:          0.3457s (Avg: 0.000346s/frame)
Combined internal avg per frame:      0.013987s/frame

Compare to the performace of ffmpeg, for the same 1000 random noise image, it only cost 9s.

I wonder if the performance comparison of PyAV and ffmpeg is expected? Are there any other optimization I can do to increase the CPU usage in order to get better performance? Since the bottleneck is stream.encode, I would expected that the PyAV can have similar encode performance compared with ffmpeg.

Thanks!

[Globefishp]

I have solved my original question (timecode-aware encoding) by combining x264 (cli) and mp4fpsmod tool. This discussion could be closed.