Skip to content

feat: add support for Intel XPU backend in device selection #1553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

xiangyang-95
Copy link

@xiangyang-95 xiangyang-95 commented Jul 21, 2025

This pull request enhances the device compatibility of the lerobot utility functions by adding support for Intel XPU. The changes include updates to device selection, validation, and data type handling to ensure smooth integration of the XPU backend.

Added support for Intel XPU:

  • Device selection and logging:

    • Updated auto_select_torch_device to detect and select the XPU backend when available.
    • Enhanced get_safe_torch_device to handle "xpu" as a valid device option.
  • Data type compatibility:

    • Modified get_safe_dtype to check XPU device capabilities for float64 support. If unsupported, it falls back to float32 with appropriate warnings.
  • Device availability checks:

    • Extended is_torch_device_available to validate the availability of the XPU backend.
  • AMP (Automatic Mixed Precision) support:

    • Updated is_amp_available to include XPU as a device supporting AMP.This pull request enhances the src/lerobot/utils/utils.py file by adding support for Intel XPU devices. The changes ensure that the utility functions can detect, validate, and handle the XPU backend alongside existing backends like CUDA, MPS, and CPU.

Added support for Intel XPU:

  • auto_select_torch_device: Added logic to detect and select the XPU backend when available.
  • get_safe_torch_device: Extended the function to handle "xpu" as a valid device option, ensuring proper assertions and device creation.
  • get_safe_dtype: Introduced checks for XPU devices to handle cases where torch.float64 is unsupported, falling back to torch.float32 if necessary.
  • is_torch_device_available: Updated to include detection of XPU availability and adjusted the error message to list XPU as a supported device.

How to checkout & try? (for the reviewer)

Provide a simple way for the reviewer to try out your changes.

Examples:

Install environment

python3 -m venv .venv
source .venv/bin/activate
git clone https://github.com/huggingface/lerobot.git
cd lerobot
pip install -e ".[smolvla]" --extra-index-url https://download.pytorch.org/whl/xpu

Run training

Tested on Intel B580 GPU

python3 src/lerobot/scripts/train.py \
  --policy.path=lerobot/smolvla_base \
  --policy.push_to_hub=false \
  --policy.device=xpu \
  --dataset.repo_id=lerobot/svla_so100_stacking \
  --batch_size=32 \
  --steps=20000 \
  --output_dir=outputs/train/my_smolvla \
  --job_name=my_smolvla_training \
  --wandb.enable=false

SECTION TO REMOVE BEFORE SUBMITTING YOUR PR

Note: Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. Try to avoid tagging more than 3 people.

Note: Before submitting this PR, please read the contributor guideline.

@Copilot Copilot AI review requested due to automatic review settings July 21, 2025 01:55
Copilot

This comment was marked as outdated.

@xiangyang-95 xiangyang-95 requested a review from Copilot July 21, 2025 02:06
Copilot

This comment was marked as outdated.

Co-authored-by: Copilot <[email protected]>
Signed-off-by: Lim Xiang Yang <[email protected]>
@xiangyang-95 xiangyang-95 requested a review from Copilot July 21, 2025 02:09
Copilot

This comment was marked as outdated.

Co-authored-by: Copilot <[email protected]>
Signed-off-by: Lim Xiang Yang <[email protected]>
@xiangyang-95 xiangyang-95 requested a review from Copilot July 21, 2025 02:12
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for Intel XPU backend to the lerobot utility functions, expanding device compatibility beyond the existing CUDA, MPS, and CPU backends. The changes ensure that XPU devices can be automatically detected, validated, and properly configured with appropriate data type handling.

  • Added XPU device detection and selection in auto device selection logic
  • Extended device validation and creation functions to handle XPU devices
  • Implemented XPU-specific data type compatibility checks with float64 fallback handling

@CarolinePascal CarolinePascal added enhancement Suggestions for new features or improvements policies Items related to robot policies labels Jul 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Suggestions for new features or improvements policies Items related to robot policies
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants