Use jemalloc open source

Summary: Internally, we default to jemalloc, so all our optimisations and profiling is done with jemalloc. Use the same on linux/mac. Improves performance on `time pyrefly check torch  -o /dev/null` from 3.6s to 1.6s.

Reviewed By: SamChou19815

Differential Revision: D73927808

fbshipit-source-id: ec41ff5fca3bb9d0d1c3f6cfac9a1daaeb96d019

Author: ndmitchell

pyrefly/Cargo.toml

@@ -74,6 +74,7 @@ tar = "0.4.44"
 zstd = { version = "0.13", features = ["experimental", "zstdmt"] }
 
 [target.'cfg(not(target_arch = "wasm32"))'.dependencies]
+jemallocator = "0.5.4"
 watchman_client = "0.9.0"
 which = "4.2.4"
 zstd = { version = "0.13", features = ["experimental", "zstdmt"] }

pyrefly/bin/main.rs

@@ -36,6 +36,13 @@ use pyrefly::library::library::library::library;
 use tracing::debug;
 use tracing::info;
 
+// fbcode likes to set its own allocator in fbcode.default_allocator
+// So when we set our own allocator, buck build buck2 or buck2 build buck2 often breaks.
+// Making jemalloc the default only when we do a cargo build.
+#[global_allocator]
+#[cfg(all(any(target_os = "linux", target_os = "macos"), not(fbcode_build)))]
+static ALLOC: jemallocator::Jemalloc = jemallocator::Jemalloc;
+
 #[derive(Debug, Parser)]
 #[command(name = "pyrefly")]
 #[command(about = "Next generation of Pyre type checker", long_about = None)]