Skip to content

[WIP]: Adding device message handling #169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

bechols97
Copy link
Collaborator

This is a feature to store messages and call a callback with the message as arguments at a later time. The main goal with this feature is to work across the host and device. For example, storing a log message on device with printing out to a file after the kernel is complete.

This is currently a work in progress with open questions about the design.

  • Can there be multiple readers from the host?
  • Can the host post messages while the device is posting?
  • Currently, messages that go over the capacity of the buffer are lost. Is this behavior desired?
  • Currently, atomic increment is supported as a host device function. This requires an ifdef to determine if device side vs. host side. To avoid this, it may be better to have message_queue have a different version based on the type of resource. Or, this may be better in another library

@trws
Copy link
Member

trws commented Mar 20, 2025

We should find a time to chat about this in person or on a call I think, rather than try to hash everything out in the PR. Some immediate thoughts/questions:

  • The atomic should use the ones provided by desul, since we already built the infrastructure for supporting system-level atomics there and it can be reused.
  • For the system level atomics to work reliably, these would need to be allocated in pinned or managed memory or similar depending on the target system, so it may need to be a factory function to ensure that's done correctly.
  • Is the goal for this to be MPMC, MPSC or what kind of queue are you going for, and with what combination of host/device readers/writers concurrently?

As to where this goes, my first thought is it would probably make more sense in RAJA or similar since CAMP doesn't depend on desul, but I could see arguments either way.

@rhornung67
Copy link
Member

@trws it could go in RAJA to simplify the dependencies. IIRC, we briefly discussed that as an option in a recent RAJA meeting.

@bechols97 bechols97 marked this pull request as draft March 20, 2025 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants