Skip to content

[CAS] Add a new API in ObjectStore to import a CAS tree #10839

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: next
Choose a base branch
from

Conversation

cachemeifyoucan
Copy link

@cachemeifyoucan cachemeifyoucan commented Jun 13, 2025

Add a new API to ObjectStore that can import a cas tree from another CAS. The two ObjectStores don't have to share the same hashing algorithm since all the objects will be rehashed and inserted into the new database.

As part of the better testing support, the test plugin CAS library now uses SHA1 hashing which is different from default BLAKE3 hasher as builtin CAS. The test plugin library can be used to test interaction of CAS of different schemas.

@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-cas-import-tree branch from 487eced to 9292e05 Compare June 13, 2025 20:40
Copy link

@benlangmuir benlangmuir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add a unit test that imports between two CAS with different hash schema

@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-cas-import-tree branch 2 times, most recently from 474a066 to 7e4fe68 Compare June 16, 2025 17:33
@cachemeifyoucan
Copy link
Author

@swift-ci please test llvm

Copy link

@benlangmuir benlangmuir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, with some minor nits.

@@ -125,6 +125,55 @@ bool llcas_cas_options_set_option(llcas_cas_options_t c_opts, const char *name,

namespace {

using HasherT = SHA1;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akyrtzi any concerns about changing the cas plugin test dylib to use SHA1 so that it differs from the builtin CAS?

Add a new API to ObjectStore that can import a cas tree from another
CAS. The two ObjectStores don't have to share the same hashing
algorithm since all the objects will be rehashed and inserted into the
new database.

As part of the better testing support, the test plugin CAS library now
uses SHA1 hashing which is different from default BLAKE3 hasher as
builtin CAS. The test plugin library can be used to test interaction of
CAS of different schemas.
@cachemeifyoucan cachemeifyoucan force-pushed the eng/PR-cas-import-tree branch from 7e4fe68 to 41bdb56 Compare June 16, 2025 19:35
@cachemeifyoucan
Copy link
Author

@swift-ci please test llvm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants