Skip to content

Adding upsert for adopt-or-create #184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Adding upsert for adopt-or-create #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
54b924b
adding upsert for adopt-or-create
JonathanGraniero May 13, 2025
e9ac7ea
upsert test
JonathanGraniero May 13, 2025
935fdf3
update comments and deep copy desired for latest
JonathanGraniero May 13, 2025
4839be0
updating test to validate that the latest and desired are different
JonathanGraniero May 13, 2025
b8ad462
removed unnecessary lines from code review
JonathanGraniero May 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions pkg/runtime/reconciler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -333,13 +333,22 @@ func (r *resourceReconciler) handlePopulation(
}

populated := desired.DeepCopy()
err = populated.PopulateResourceFromAnnotation(adoptionFields)
// maybe don't return errors when it's adopt-or-create?
//
// TODO (michaelhtm) change PopulateResourceFromAnnotation to understand
// adopt-or-create, and validate Spec fields are not nil...
if err != nil && adoptionPolicy != AdoptionPolicy_AdoptOrCreate {
return nil, err

// For adopt-or-create, we want to:
// 1. Strictly parse and validate adoption fields
// 2. If found, adopt the resource
// 3. Use spec values for updates
if adoptionPolicy == AdoptionPolicy_AdoptOrCreate {
err = populated.PopulateResourceFromAnnotation(adoptionFields)
if err != nil {
rlog.Debug("Ignoring adoption field population error for adopt-or-create", "error", err)
}
} else {
// For regular adoption, maintain existing behavior
err = populated.PopulateResourceFromAnnotation(adoptionFields)
if err != nil {
return nil, err
}
Comment on lines +341 to +351
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this one, we were planning on implementing a different method.
Instead of PopulateResourceFromAnnotation we can give the method context of whether it is adopt or adopt-or-create. This will allow it to either Populate both Spec and Status, or only Status.

Honestly i'm not sure what the expected Error should be if you need to provide a required field in Status on adopt-or-create but you fail to do so. Should that be a terminal error? Or just return nothing, and just create the resource...
cc: @gfaraj @havard024

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to suggestions here, what I was trying to solve for here is that in the process of doing my migration, I found that I had some resources that were setup incorrectly and were not standardized. So I want to standardize them in one shot rather than doing first getting the reconciler to accept the adopted policy, then go and remove the extra code that I was just doing to be able to adopt the resource.

If this is in line with what y'all need to, or you need me to add some more branching logic, totally up to adding that in here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you test your use case with the latest released controllers? I believe this issue came up before, and we just cut releases for it today..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also create an issue describing the issue you encountered? just in case we also have it in other controllers

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! aws-controllers-k8s/community#2481 I'll also update that in the PR description

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @JonathanGraniero
just responded on the Github issue. I believe this was resolved in the latest release

}

return populated, nil
Expand Down
88 changes: 88 additions & 0 deletions pkg/runtime/reconciler_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -457,6 +457,92 @@ func TestReconcilerAdoptOrCreateResource_Adopt(t *testing.T) {
rm.AssertNumberOfCalls(t, "Create", 0)
}

func TestReconcilerAdoptOrCreateResource_Upsert(t *testing.T) {
require := require.New(t)

ctx := context.TODO()

desired, _, metaObj := resourceMocks()
desired.On("ReplaceConditions", mock.AnythingOfType("[]*v1alpha1.Condition")).Return()
metaObj.SetAnnotations(map[string]string{
ackv1alpha1.AnnotationAdoptionPolicy: "adopt-or-create",
ackv1alpha1.AnnotationAdoptionFields: "{\"arn\": \"my-adopt-book-arn\"}",
})
// Initialize the spec field in the desired object
metaObj.Object = map[string]interface{}{
"spec": map[string]interface{}{
"A": "val1",
},
}

ids := &ackmocks.AWSResourceIdentifiers{}
delta := ackcompare.NewDelta()
delta.Add("Spec.A", "val1", "val2")

latest, latestRTObj, latestMetaObj := resourceMocks()
latest.On("Identifiers").Return(ids)
latest.On("Conditions").Return([]*ackv1alpha1.Condition{})
// Initialize the spec field in the latest object
latestMetaObj.Object = map[string]interface{}{
"spec": map[string]interface{}{
"A": "val2",
},
}
latest.On("MetaObject").Return(metav1.ObjectMeta{
Annotations: map[string]string{
ackv1alpha1.AnnotationAdoptionPolicy: "adopt-or-create",
ackv1alpha1.AnnotationAdoptionFields: "{\"arn\": \"my-adopt-book-arn\"}",
},
})
latest.On("Conditions").Return([]*ackv1alpha1.Condition{})
latest.On(
"ReplaceConditions",
mock.AnythingOfType("[]*v1alpha1.Condition"),
).Return()

rm := &ackmocks.AWSResourceManager{}
rm.On("ResolveReferences", ctx, nil, desired).Return(
desired, false, nil,
).Times(2)
rm.On("ClearResolvedReferences", desired).Return(desired)
rm.On("ClearResolvedReferences", latest).Return(latest)
rm.On("ReadOne", ctx, desired).Return(
latest, nil,
).Once()
rm.On("Update", ctx, desired, latest, delta).Return(
latest, nil,
).Once()
rm.On("IsSynced", ctx, latest).Return(true, nil)
rmf, rd := managedResourceManagerFactoryMocks(desired, latest)

rm.On("LateInitialize", ctx, latest).Return(latest, nil)
rd.On("IsManaged", desired).Return(true)
rd.On("Delta", desired, latest).Return(
delta,
).Once()
rd.On("Delta", desired, latest).Return(ackcompare.NewDelta())
rd.On("Delta", latest, latest).Return(ackcompare.NewDelta())

r, kc, scmd := reconcilerMocks(rmf)
rm.On("EnsureTags", ctx, desired, scmd).Return(nil)
statusWriter := &ctrlrtclientmock.SubResourceWriter{}
kc.On("Status").Return(statusWriter)
kc.On("Patch", ctx, latestRTObj, mock.AnythingOfType("*client.mergeFromPatch")).Return(nil)
statusWriter.On("Patch", ctx, latestRTObj, mock.AnythingOfType("*client.mergeFromPatch")).Return(nil)

// Test that we can update the resource even with non-matching adoption fields
_, err := r.Sync(ctx, rm, desired)
require.Nil(err)
rm.AssertNumberOfCalls(t, "ReadOne", 1)
rm.AssertCalled(t, "Update", ctx, desired, latest, delta)
rd.AssertCalled(t, "Delta", desired, latest)
rm.AssertNumberOfCalls(t, "Create", 0)

rm.AssertCalled(t, "Update", ctx, desired, latest, mock.MatchedBy(func(d *ackcompare.Delta) bool {
return d.DifferentAt("Spec.A")
}))
}

func TestReconcilerCreate_UnManagedResource_CheckReferencesResolveOnce(t *testing.T) {
require := require.New(t)

Expand Down Expand Up @@ -1331,6 +1417,8 @@ func TestReconcilerUpdate_ErrorInLateInitialization(t *testing.T) {
rm.AssertCalled(t, "Update", ctx, desired, latest, delta)
// No difference in desired, latest metadata and spec
kc.AssertNotCalled(t, "Patch", ctx, latestRTObj, mock.AnythingOfType("*client.mergeFromPatch"))
// Only the HandleReconcilerError wrapper function ever calls patchResourceStatus
kc.AssertNotCalled(t, "Status")
rm.AssertCalled(t, "LateInitialize", ctx, latest)
rm.AssertCalled(t, "EnsureTags", ctx, desired, scmd)
}
Expand Down