[fix][client]Failure to load encryption key should not prevent creation of producer on client side

[vraulji567]

Fixes #24280

Motivation

Failure to load the encryption public key while creating the producer creates a producer on the server side but returns an exception on client side. If the client application is retrying producer creation due to the failure on client side, this will create too many producer connection on the broker side and can bring down the broker.
I expected that the producer creation to be successful (client should not see failure if there is a failure to load the encryption key) and eventually send should fail with the failure to load the encryption key.

Modifications

• Updated the producer creation to not throw an exception when there is a failure to load the encryption key.

Verifying this change

• Added tests to verify the change

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository:

[vraulji567]

Can anyone review my PR?