Distance based latency #858

thomasywang · 2025-08-13T18:17:32Z

Summary:
Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency.

Latency is randomly sample from a beta distribution where the min and max for each distance is configured

Implementation details (follow along numbers in comments):

In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet
When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between
We determine distance between 2 coordinates by identifying the most major dimension in which they differ
We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance.
We use the identified distance to get a sample for what the latency should be for that send
We pass in that latency to the MessageDeliveryEvent to use as its duration
The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs
Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency

test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance

Differential Revision: D80141665

facebook-github-bot · 2025-08-13T18:17:56Z

This pull request was exported from Phabricator. Differential Revision: D80141665

Summary: Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Differential Revision: D80141665

facebook-github-bot · 2025-08-14T00:07:19Z

This pull request was exported from Phabricator. Differential Revision: D80141665

Summary: Pull Request resolved: meta-pytorch#858 Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Differential Revision: D80141665

Summary: Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Differential Revision: D80141665

facebook-github-bot · 2025-08-14T15:01:11Z

This pull request was exported from Phabricator. Differential Revision: D80141665

Summary: Pull Request resolved: meta-pytorch#858 Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Differential Revision: D80141665

Summary: Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Differential Revision: D80141665

facebook-github-bot · 2025-08-18T23:02:21Z

This pull request was exported from Phabricator. Differential Revision: D80141665

Summary: Pull Request resolved: meta-pytorch#858 Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Differential Revision: D80141665

Summary: Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Differential Revision: D80141665

facebook-github-bot · 2025-08-19T19:20:55Z

This pull request was exported from Phabricator. Differential Revision: D80141665

Summary: Pull Request resolved: meta-pytorch#858 Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Differential Revision: D80141665

Summary: Previously we had to use u64 for serialization reasons but those reasons no longer exist Differential Revision: D80556690

Summary: There was an open TODO to remove the global mailbox for SimClock. We don't actually even need mailboxes for sim clock and a oneshot works just fine Differential Revision: D80029571

Summary: Pull Request resolved: meta-pytorch#854 When we increase the number of actors in our simulation it takes longer for all the events at a certain time to complete so we need to wait for longer. If we wait to long then the simulation just runs slower than it needs to so its nice to make this configurable. In the long term we will come up with a more robust solution to this but in the meantime that is not a priority. See EX528476 to understand the underlying problem the debounce is remedying Differential Revision: D80137965 Reviewed By: pablorfb-meta

Summary: The sim allocator will now register the location (region, dc, zone, rack, host, gpu) of every ProcId upon creation with the simnet. Differential Revision: D80137963

Summary: Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Reviewed By: pablorfb-meta Differential Revision: D80141665

Summary: Pull Request resolved: meta-pytorch#858 Now that the simnet has awareness of which compute resource each ProcId maps to, when messages are being sent we can simply look at the sender and destination ProcIds and compute the distance the message is being sent in order to determine the latency. Latency is randomly sample from a beta distribution where the min and max for each distance is configured Implementation details (follow along numbers in comments): 1. In the previous diff when Procs were allocated, their coordinates (region, dc, zone, rack, host, gpu) were registered to the Simnet 2. When SimTx posts a message, we can safely assume that it is a MessageEnvelope. MessageEnvelopes contain information about the sender and receiver so we can determine which ProcIds the message is being sent between, which in turn means we can identify which coordinates they are being sent between 3. We determine distance between 2 coordinates by identifying the most major dimension in which they differ 4. We create a struct called LatencyConfig which holds a distribution for sampling, as well as minimum and maximum values for each distance. 5. We use the identified distance to get a sample for what the latency should be for that send 6. We pass in that latency to the MessageDeliveryEvent to use as its duration 7. The old network configuration which was an all-to-all map of edges with latencies between nodes has been removed along with all related structs 8. Unit tests have been refactored such that when we need a particular message to be sent with a particular latency, we register the ProcIds with the appropriate coordinates, and configure the interdistance latency test_allocator_registers_resources in alloc/sim.rs demonstrates that when we allocate a ProcMesh using the sim allocator, our Procs are registered as compute resources and the latencies are computed based on distance Reviewed By: pablorfb-meta Differential Revision: D80141665

facebook-github-bot · 2025-08-20T18:11:57Z

This pull request was exported from Phabricator. Differential Revision: D80141665

facebook-github-bot · 2025-08-21T01:21:58Z

This pull request has been merged in 112091d.

meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Aug 13, 2025

facebook-github-bot added the fb-exported label Aug 13, 2025

thomasywang force-pushed the export-D80141665 branch from a44e6bf to 21ec77b Compare August 14, 2025 00:03

thomasywang force-pushed the export-D80141665 branch from 21ec77b to 667531b Compare August 14, 2025 00:07

thomasywang force-pushed the export-D80141665 branch from 667531b to d4889c9 Compare August 14, 2025 14:58

thomasywang force-pushed the export-D80141665 branch 2 times, most recently from 85d1018 to ba31c31 Compare August 18, 2025 22:59

thomasywang force-pushed the export-D80141665 branch from ba31c31 to 9ad771c Compare August 18, 2025 23:02

thomasywang force-pushed the export-D80141665 branch from 9ad771c to 8858e2d Compare August 19, 2025 19:16

thomasywang force-pushed the export-D80141665 branch from 8858e2d to bb0d755 Compare August 19, 2025 19:21

thomasywang added 4 commits August 20, 2025 11:04

Use tokio::time::Duration and tokio::time::Instant for timekeeping

d524452

Summary: Previously we had to use u64 for serialization reasons but those reasons no longer exist Differential Revision: D80556690

Use tokio::oneshot for sim clock

082ee4a

Summary: There was an open TODO to remove the global mailbox for SimClock. We don't actually even need mailboxes for sim clock and a oneshot works just fine Differential Revision: D80029571

Compute resource aware simulation

7fff3cb

Summary: The sim allocator will now register the location (region, dc, zone, rack, host, gpu) of every ProcId upon creation with the simnet. Differential Revision: D80137963

thomasywang force-pushed the export-D80141665 branch from bb0d755 to 56fcf78 Compare August 20, 2025 18:07

thomasywang force-pushed the export-D80141665 branch from 56fcf78 to f8a69db Compare August 20, 2025 18:12

facebook-github-bot closed this in 112091d Aug 21, 2025

facebook-github-bot added the Merged label Aug 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Distance based latency #858

Distance based latency #858

thomasywang commented Aug 13, 2025

Uh oh!

facebook-github-bot commented Aug 13, 2025

Uh oh!

facebook-github-bot commented Aug 14, 2025

Uh oh!

facebook-github-bot commented Aug 14, 2025

Uh oh!

facebook-github-bot commented Aug 18, 2025

Uh oh!

facebook-github-bot commented Aug 19, 2025

Uh oh!

facebook-github-bot commented Aug 20, 2025

Uh oh!

facebook-github-bot commented Aug 21, 2025

Uh oh!

Uh oh!

Distance based latency #858

Distance based latency #858

Conversation

thomasywang commented Aug 13, 2025

Uh oh!

facebook-github-bot commented Aug 13, 2025

Uh oh!

facebook-github-bot commented Aug 14, 2025

Uh oh!

facebook-github-bot commented Aug 14, 2025

Uh oh!

facebook-github-bot commented Aug 18, 2025

Uh oh!

facebook-github-bot commented Aug 19, 2025

Uh oh!

facebook-github-bot commented Aug 20, 2025

Uh oh!

facebook-github-bot commented Aug 21, 2025

Uh oh!

Uh oh!