feat: `emitStringData` boolean for secretGenerator #5894

stealthybox · 2025-04-22T06:05:32Z

This allows users to pass emitStringData: true to the secretGenerator, alongside the type option.
When enabled, UTF-8 strings are output in plainText stringData, and non-UTF strings and any binary data fallback to the default behavior of being base64 encoded in data.

This is very similar to the default, kyaml behavior of loading values into ConfigMap's data and binaryData fields.

This feature provides a general U/X improvement for people using kustomize interactively, and also allows Flux users to template into generated secrets.
(Flux users already can and do template into ConfigMaps, so we want to give people more secure mechanisms in kustomize-controller)

test: add secretGenerator testcases
feat: support emitStringData in secretGenerator
test: test emitStringData in secretGenerator

resolves #5142 #1444 #1261 #793

k8s-ci-robot · 2025-04-22T06:05:35Z

This PR has multiple commits, and the default merge method is: merge.
You can request commits to be squashed using the label: tide/merge-method-squash

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

koba1t · 2025-04-30T14:38:51Z

/assign

matheuscscp

LGTM 🚀

k8s-ci-robot · 2025-05-03T13:20:18Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: matheuscscp, stealthybox
Once this PR has been reviewed and has the lgtm label, please ask for approval from koba1t. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

stealthybox · 2025-06-09T21:25:32Z

I've just done a fresh rebase.
@koba1t, is there any way I can help out in reviewing this patch?
Thanks :)

koba1t · 2025-06-10T08:12:05Z

@stealthybox
So, Sorry for the late review.

The field name stringData: true is associated with the field name of a secret resource, but is it possible to change it to a more descriptive field name, such as noEncode: true?
As a test scenario, could you verify that when adding a secondary secretGenerator with stringData: false to a secretGenerator configured with stringData: true using the behavior: merge option, the resulting data is properly base64 encoded?

matheuscscp · 2025-06-10T08:14:19Z

How about outputStringData?

koba1t · 2025-06-11T00:02:35Z

@matheuscscp
What I'm trying to say is that field names should be clear enough that users can immediately understand how they'll behave when seeing them in this PR.
The name stringData likely comes from the existence of a stringData field in secret resources, but it doesn't actually explain the behavior in kustomize.

For example, in the case of stringData, it's not clear whether this refers to string input or what exactly is being stringified, or in the case of outputStringData, it's not clear whether this controls the output destination or something else.

stealthybox · 2025-07-24T18:18:34Z

could you verify that when adding a secondary secretGenerator with stringData: false to a secretGenerator configured with stringData: true using the behavior: merge option, the resulting data is properly base64 encoded?

@koba1t, thanks for pointing out this out.
I've added tests for the following cases in test: field overrides for configMapGenerator + secretGenerator
67fc288:

1. merge encoded data fields onto a generated secret with stringData
1. override encoded data fields with stringData for secretGenerator
  producing a Secret that successfully uses the new value in the Kubernetes API
1. override stringData fields with encoded data for secretGenerator
  producing a Secret that silently fails to use the new value in the Kubernetes API
1. override encoded binaryData fields with data for configMapGenerator
  producing an invalid ConfigMap
1. override data fields with encoded binaryData for configMapGenerator
  producing an invalid ConfigMap

The 4 cases where fields override using a different encoding output duplicate fields.
This is existing ConfigMapGenerator behavior that is undesirable.
This existing buggy behavior becomes new Secret-gen behavior.
I can fix both of these in a followup PR as discussed privately in our Slack dm.

Here is an example of what that fix could look like:
stealthybox@e95e854

stealthybox · 2025-07-24T19:05:28Z

Regarding the field name, I understand the concern of stringData as a bool not fully expressing the behavior.
I would hope the doc-string helps clarify this beyond just the name.

In the most recent issue,
an end-user requests a stringData bool in the generatorOptions. #5142 (comment),
@seh and @annasong20 confirm the boolean field stringData should be added to SecretArgs and exposed directly on secretGenerator. #5142 (comment)

A separate end-user suggests the boolean field should be called useStringData. #5142 (comment)

The title of unrelated issue (#1261) is "Support StringData option in SecretArgs".

Considering end-user, contributor, and maintainer responses, stringData seems to be the agreed upon field name that is most natural. I think the word "stringData" must at least be contained within the field name.

noEncode implies that there isn't a fallback encoding behavior when there is.
I don't think noEncode works.

I'd also challenge that stringData doesn't make sense.
Most of the fields on generator options across the project are passthrough fields that end up in the output:

labels
annotations
immutable
type (for Secrets only -- defaults to "Opaque" when not supplied)

The new field isn't a passthrough value field -- it changes generator behavior. disableNameSuffixHash is a similar behavioral field that describes the generated output implicitly.
That's why secretGenerator[].stringData = true in my head automatically talks about the output of the generator.

in the case of stringData, it's not clear whether this refers to string input or what exactly is being stringified

This is fair. there are a lot of fields when you consider input fields like env, envs ,files, and literals mixing with output fields like type and annotations and behavioral fields like behavior and disableNameSuffixHash.
Disambiguation would need to come from the doc-string and examples.
I'm happy to add examples to:

https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kustomize/#create-a-secret.
site/content/en/docs/Tasks/secret_generator.md

in the case of outputStringData, it's not clear whether this controls the output destination or something else.

I disagree that outputStringData is unclear. There is only one output of a generator.

I think these are decent options for field names:

stringData
outputStringData
generateStringData
setStringData
preferStringDataOutput
disableBase64ForStringData

The default behavior of a configMapGenerator already includes heuristic auto-encoding behavior for binaryData, and that doesn't show up in any field names. It's just the way a kustomize generator works.

Thanks for considering these points.
I don't want to be overly pedantic, and I know the behavior is complex.
Naming this field is tricky.
I think examples and the doc-strings on the field name should help people understand what it does exactly.
I don't think we should give this field any name that inaccurately implies that it performs or prevents behaviors that it does not. Because the behavior is complex, the name either needs to be somewhat ambiguous, or very detailed and lengthy.

Do you like stringData or any of the other names I suggested?

matheuscscp · 2025-07-24T19:08:58Z

Yeah this field has to explain itself, it has to have stringData on its name somehow. Anything else would be garbage for explaining what it does.

stealthybox · 2025-07-24T19:21:16Z

unrelated to field names, here's one way to fix the existing broken configMapGenerator behavior:
stealthybox@e95e854
I would send this in a followup PR.

seh · 2025-07-24T19:39:00Z

I think "stringData" would be a deceiving name for this new field. My suggestion today is "emitStringData".

stealthybox · 2025-07-24T20:33:38Z

I've proactively renamed the field to emitStringData in the types, docs, and test names.
Tests are passing.
Now that the field name is disambiguated from the stringData fields in the test output and docstrings, it's easy to rename it to whatever we decide 👍.

stealthybox · 2025-08-04T18:47:11Z

@koba1t and I considered in our Slack dm's whether a string value field could work:

  field: "data" # "stringData" | "binaryData"

This could be a generic way to solve this problem for both ConfigMap and Secret generators, but it also creates more error states and is harder to grep for in existing code + discover in auto-complete.

I prefer the emitStringData boolean field.
If we want support for explicit encoding in ConfigMap generator, I propose we do so by adding emitBinaryData as a boolean in another PR.

stealthybox · 2025-08-08T18:46:25Z

Hi friends, if there's no further contest on the field name, I'd love to get this reviewed for a merge.
I'm happy to write some docs examples.

I want to draft a change for the merge behavior bugfix in stealthybox@e95e854 .

After these changes make their way into kustomize, kubectl needs to update kustomize before we can pull this feature into Flux.
I'd like to be able to do that before KubeCon NA and the Flux 2.7 release.

cc @koba1t @seh

seh

I have not reviewed the secrets_test.go file yet.

seh · 2025-08-21T13:44:10Z

api/internal/generators/secret.go

+// A Secret's `stringData` field is similar to ConfigMap's `data` field.
+// `stringData` allows specifying non-binary, UTF-8 secret data in string form.
+// It is provided as a write-only input field for convenience.
+// All keys and values are merged into the data field on write, overwriting any
+// existing values. The stringData field is never output when reading from the API.


Nice. Thank you for this.

seh · 2025-08-21T13:47:44Z

api/internal/generators/secret.go

+		}
+	} else {
+		if err = rn.LoadMapIntoSecretData(m); err != nil {
+			return nil, fmt.Errorf("Failed to load map into Secret data: %w", err)


Likewise with the capitalization.

seh · 2025-08-21T13:47:45Z

api/internal/generators/secret.go

-		return nil, err
+	if args.EmitStringData {
+		if err = rn.LoadMapIntoSecretStringData(m); err != nil {
+			return nil, fmt.Errorf("Failed to load map into Secret stringData: %w", err)


I'm not sure whether kustomize eschews the general advice for Go programs to not capitalize the first word in an error message, assuming that this message might wind up serving as a suffix for more prefixes to be added onto it higher up in the call stack. If this capitalization isn't us playing along with the rest of the code base, consider this alternative:

Suggested change

return nil, fmt.Errorf("Failed to load map into Secret stringData: %w", err)

return nil, fmt.Errorf(`loading map into Secret "stringData" field: %w`, err)

The message adopts the advice from Preslav Rachev's essay Go's Error Handling Is a Form of Storytelling.

seh · 2025-08-21T14:00:15Z

api/internal/generators/secret_test.go

+`,
+			},
+		},
+		"construct secret from a binary file and fallback to data from stringData": {


Suggested change

"construct secret from a binary file and fallback to data from stringData": {

"construct secret from a binary file and fall back to data from stringData": {

api/krusty/configmaps_test.go

seh · 2025-08-21T14:18:26Z

kyaml/yaml/datamap.go

 }

+func (rn *RNode) LoadMapIntoSecretStringData(m map[string]string) error {
+	for _, k := range SortedMapKeys(m) {


What is the significance of walking the keys in lexicographical order? Is that to avoid any nondeterminism? Since the key names must be unique, they can't conflict with one another in the "m" map, but perhaps we're wary of what the SetField function will do against the RNode's existing fields. I'm still not sure if that's a problem, though.

If you walked the map by its keys and values together, you could avoid the index expression on the next line.

seh · 2025-08-21T14:24:23Z

kyaml/yaml/datamap.go

+	return nil
+}
+
+// valid UTF-8 strings can be stringData, but fallback to Base64 for non UTF-8


Note that the documentation capitalizes "Base64" inconsistently, often writing it as all-lowercase.

Suggested change

// valid UTF-8 strings can be stringData, but fallback to Base64 for non UTF-8

// makeSecretStringValueRNode creates a scalar node for one of two potential fields.

// If the supplied node's value is a valid UTF-8 string, it use the "stringData" field, but

// otherwise falls back to placing the Base64-encoded value in the "data" field for

// non-UTF-8 strings.

seh · 2025-08-21T14:33:20Z

kyaml/yaml/rnode.go

+	if err != nil {
+		return nil
+	}
+	result := map[string]string{}


We'd like to know how many fields are present ahead of time. We should not call (*RNode).Fields, because (*RNode).VisitFields calls it immediately, and Fields allocates.

Should we estimate a likely count here, so that we could write the following?

Suggested change

result := map[string]string{}

result := make(map[string]string, 10) // Estimate field count

Perhaps the cost of starting with a too-small map and needing it to grow later isn't a real problem.

I see that this is mimicking the longstanding (*RNode).GetDataMap method, so if it's not a problem there, it's probably not a problem here either.

seh · 2025-08-21T14:37:06Z

kyaml/yaml/rnode.go


+func (rn *RNode) SetStringDataMap(m map[string]string) {
+	if rn == nil {
+		log.Fatal("cannot set stringData map on nil Rnode")


If we change this here, we should change it in (*RNode).SetDataMap and (*RNode).SetBinaryDataMap consistently.

Suggested change

log.Fatal("cannot set stringData map on nil Rnode")

log.Fatal("cannot set stringData map on nil RNode")

seh · 2025-08-21T14:38:16Z

site/content/en/docs/Reference/API/included/secretargs.md

+
+* **emitStringData** (bool), optional
+
+	  emitStringData if true generates a v1/Secret with plain-text stringData fields


I recommend that we adapt the proposed struct field documentation here. I see that you made them consistent originally, so I'm just reminding you to update this paragraph if you decide to update the struct field documentation.

seh · 2025-08-21T14:43:23Z

api/krusty/secrets_test.go

+  - year=2025
+  - crisis=true
+`)
+	th.WriteF("service.yaml", `


What is the significance of including the Service in this kustomization?

I think just that the generator properly appends to other resources in the output.
This test is a functional copy of TestGeneratorIntVsStringNoMerge from configmaps_test.go but for secrets. All of the copied test-cases are added in their own commit before the new ones from this patch are added.

seh · 2025-08-21T14:45:28Z

api/krusty/secrets_test.go

+`)
+}
+
+// Generate Secrets similar to TestGeneratorBasics with emitStringData enabled and


Did you intend to finish this sentence? It ends with a dangling "and".

seh · 2025-08-21T14:47:19Z

api/krusty/secrets_test.go

+	// The resulting Secret will have a duplicate key in both data and stringData
+	// The stringData override will work, because the kube API considers stringData authoritative and write-only


Suggested change

// The resulting Secret will have a duplicate key in both data and stringData

// The stringData override will work, because the kube API considers stringData authoritative and write-only

// The resulting Secret will have a duplicate key in both its "data" and "stringData" fields.

// The "stringData" field's override will work, because the Kubernetes API considers the "stringData" field to be authoritative.

seh · 2025-08-21T14:49:11Z

api/krusty/secrets_test.go

+	// The resulting Secret will have a duplicate key in both data and stringData
+	// The data override will fail, because the kube API considers the older value in stringData authoritative and write-only


Suggested change

// The resulting Secret will have a duplicate key in both data and stringData

// The data override will fail, because the kube API considers the older value in stringData authoritative and write-only

// The resulting Secret will have a duplicate key in both its "data" and "stringData" fields.

// The "data" field's override will fail, because the Kubernetes API considers the sibling value in the "stringData" field to be authoritative.

seh · 2025-08-21T14:51:46Z

api/krusty/secrets_test.go

+// TODO: This should be an error instead. However, we can't strict unmarshal until we have a yaml
+// lib that support case-insensitive keys and anchors.
+// See https://github.com/kubernetes-sigs/kustomize/issues/5061


Suggested change

// TODO: This should be an error instead. However, we can't strict unmarshal until we have a yaml

// lib that support case-insensitive keys and anchors.

// See https://github.com/kubernetes-sigs/kustomize/issues/5061

// TODO: This should be an error instead. However, we can't unmarshal strictly until

// we have a YAML library that support case-insensitive keys and anchors.

// See https://github.com/kubernetes-sigs/kustomize/issues/5061.

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Apr 22, 2025

k8s-ci-robot requested review from koba1t and varshaprasad96 April 22, 2025 06:05

k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 22, 2025

stealthybox force-pushed the stringdata-secret-gen branch from 14da6ae to 905d18d Compare April 22, 2025 06:24

stealthybox changed the title ~~Support outputting stringData from secretGenerator~~ feat: Support outputting stringData from secretGenerator Apr 23, 2025

k8s-ci-robot assigned koba1t Apr 30, 2025

matheuscscp approved these changes May 3, 2025

View reviewed changes

stealthybox force-pushed the stringdata-secret-gen branch from 905d18d to 9e14bdb Compare June 9, 2025 19:09

stealthybox force-pushed the stringdata-secret-gen branch from 9e14bdb to 2e90352 Compare July 24, 2025 17:56

test: add secretGenerator testcases

ef57ad8

stealthybox force-pushed the stringdata-secret-gen branch 2 times, most recently from bab4e87 to 67fc288 Compare July 24, 2025 18:03

stealthybox added 3 commits July 24, 2025 14:11

feat: support emitStringData in secretGenerator

2652351

test: test emitStringData in secretGenerator

072ae78

test: field overrides for configMapGenerator + secretGenerator

c0b3af1

stealthybox force-pushed the stringdata-secret-gen branch from 67fc288 to c0b3af1 Compare July 24, 2025 20:23

stealthybox changed the title ~~feat: Support outputting stringData from secretGenerator~~ feat: emitStringData boolean for secretGenerator Jul 24, 2025

adoramshoval mentioned this pull request Aug 20, 2025

secretGenerator overwrites existing stringData on merge #5955

Open

seh reviewed Aug 21, 2025

View reviewed changes

	return nil, fmt.Errorf("Failed to load map into Secret stringData: %w", err)
	return nil, fmt.Errorf(`loading map into Secret "stringData" field: %w`, err)

	"construct secret from a binary file and fallback to data from stringData": {
	"construct secret from a binary file and fall back to data from stringData": {

-// valid UTF-8 strings can be stringData, but fallback to Base64 for non UTF-8
+// makeSecretStringValueRNode creates a scalar node for one of two potential fields.
+// If the supplied node's value is a valid UTF-8 string, it use the "stringData" field, but
+// otherwise falls back to placing the Base64-encoded value in the "data" field for
+// non-UTF-8 strings.

	result := map[string]string{}
	result := make(map[string]string, 10) // Estimate field count

	log.Fatal("cannot set stringData map on nil Rnode")
	log.Fatal("cannot set stringData map on nil RNode")


		* emitStringData (bool), optional

		emitStringData if true generates a v1/Secret with plain-text stringData fields

		// The resulting Secret will have a duplicate key in both data and stringData
		// The stringData override will work, because the kube API considers stringData authoritative and write-only

		// The resulting Secret will have a duplicate key in both data and stringData
		// The data override will fail, because the kube API considers the older value in stringData authoritative and write-only

feat: emitStringData boolean for secretGenerator #5894

Are you sure you want to change the base?

feat: emitStringData boolean for secretGenerator #5894

Conversation

stealthybox commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Apr 22, 2025

Uh oh!

koba1t commented Apr 30, 2025

Uh oh!

matheuscscp left a comment

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented May 3, 2025

Uh oh!

stealthybox commented Jun 9, 2025

Uh oh!

koba1t commented Jun 10, 2025

Uh oh!

matheuscscp commented Jun 10, 2025

Uh oh!

koba1t commented Jun 11, 2025

Uh oh!

stealthybox commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stealthybox commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matheuscscp commented Jul 24, 2025

Uh oh!

stealthybox commented Jul 24, 2025

Uh oh!

seh commented Jul 24, 2025

Uh oh!

stealthybox commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stealthybox commented Aug 4, 2025

Uh oh!

stealthybox commented Aug 8, 2025

Uh oh!

seh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

feat: `emitStringData` boolean for secretGenerator #5894

feat: `emitStringData` boolean for secretGenerator #5894

stealthybox commented Apr 22, 2025 •

edited

Loading

stealthybox commented Jul 24, 2025 •

edited

Loading

stealthybox commented Jul 24, 2025 •

edited

Loading

stealthybox commented Jul 24, 2025 •

edited

Loading