Skip to content

[fix][sec] Update dependencies to use snakeyaml 2.0 #20222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

eugene-cheverda
Copy link

@eugene-cheverda eugene-cheverda commented May 4, 2023

Fixes #20224

Motivation

Fixes https://avd.aquasec.com/nvd/cve-2022-1471 caused by snakeyaml by updating all dependencies bringing it into the project

Modifications

Updated jackson and prometheus dependencies, updated code to use non-deprecated EnumResolver functions

Verifying this change

  • Make sure that the change passes the CI checks.

This change is already covered by existing tests, such as FieldParserTest.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: eugene-cheverda#1

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label May 4, 2023
@eugene-cheverda eugene-cheverda marked this pull request as ready for review May 4, 2023 18:08
@eugene-cheverda eugene-cheverda changed the title [improve][security] Update dependencies to use snakeyaml 2.0 [improve][client] Update dependencies to use snakeyaml 2.0 May 4, 2023
@eugene-cheverda eugene-cheverda force-pushed the snakeyaml_security_fix branch 3 times, most recently from 9d88eac to 91d5210 Compare May 4, 2023 19:05
@eugene-cheverda eugene-cheverda marked this pull request as draft May 4, 2023 19:28
@eugene-cheverda eugene-cheverda force-pushed the snakeyaml_security_fix branch 3 times, most recently from b4300fa to 2772601 Compare May 4, 2023 21:16
@eugene-cheverda eugene-cheverda changed the title [improve][client] Update dependencies to use snakeyaml 2.0 [fix][sec] Update dependencies to use snakeyaml 2.0 May 4, 2023
@eugene-cheverda eugene-cheverda marked this pull request as ready for review May 4, 2023 23:50
Copy link
Member

@mattisonchao mattisonchao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a corner regression by Jackson 2.15. It will affect our json schema. We will need to talk about it this in the dev mail list.
FasterXML/jackson-databind#3874

@mattisonchao
Copy link
Member

@codelipenghui @Technoboy- @lhotari @michaeljmarshall @tisonkun would you mind taking a look?

@eugene-cheverda
Copy link
Author

eugene-cheverda commented May 5, 2023

There's a corner regression by Jackson 2.15. It will affect our json schema. We will need to talk about it this in the dev mail list. FasterXML/jackson-databind#3874

@mattisonchao So is my understanding correct that wherever @JsonIgnore is used with private transient fields we need to make sure that they're not getting serialized with 2.15.0?

UPD: please see 12f5421
I made a search over private transient fields with @JsonIgnore attribute and added tests to verify they're not present in serialized output. Please let me know if it's a right direction.

@eugene-cheverda eugene-cheverda force-pushed the snakeyaml_security_fix branch 2 times, most recently from 92dc454 to 12f5421 Compare May 5, 2023 17:06
@eugene-cheverda
Copy link
Author

Here's the gist, for future discussion, that reproduces regression for regular class, lombok.Data annotated class and lombok.Value annotated class https://gist.github.com/eugene-cheverda/e3370304895ada1048fc210d0261d853

@eugene-cheverda
Copy link
Author

Regression changes behavior in the following cases:

  1. For regular class, @Data annotated the following definition of json property is not serialized in 2.14.2, gets serialized in 2.15.0:
@JsonIgnore
private transient String field4;

public String getField4() {
    return field4;
}

public void setField4(String field4) {
    this.field4 = field4;
}
  1. For @Value annotated class the following definitions of json property are not serialized in 2.14.2, get serialized in 2.15.0:
@JsonIgnore
private transient String field3;

@JsonIgnore
private transient String field4;

public String getField4() {
    return field4;
}

The most notable discrepancy in behavior between @Data and @Value annotated classes. The following property will be excluded for @Data and will be included in @Value class:

@JsonIgnore
private transient String field3;

i.e. for @Data annotated class @JsonIgnore is being propagated to the generated property, for @Value class it is not.

Speaking about usages of @JsonIgnore and private transient field in pulsar code, please see the list below:

All of the above occurrences are in @Data annotated classes, hence the behavior matches to the one in 2.14.2, the absence of mentioned properties is covered in tests in PR

If this PR gets approved/merged, I'll cherry-pick additional changes into PR agains branch-3.0

@eugene-cheverda
Copy link
Author

Hi @mattisonchao did you have a chance to discuss this in dev mail group? TIA

@eugene-cheverda eugene-cheverda force-pushed the snakeyaml_security_fix branch 3 times, most recently from 93f03f7 to ab7cfb9 Compare May 11, 2023 15:25
@tisonkun
Copy link
Member

tisonkun commented Jun 8, 2023

I remember that the client doesn't use snakeyaml actually. Maybe you can try exclude the dependency from shaded result so we get rid of all these things.

@github-actions
Copy link

github-actions bot commented Jul 9, 2023

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label Jul 9, 2023
@tisonkun
Copy link
Member

Resolved by #20085.

@tisonkun tisonkun closed this Jul 27, 2023
@tisonkun tisonkun reopened this Jul 27, 2023
@github-actions github-actions bot removed the Stale label Jul 28, 2023
@Technoboy- Technoboy- added this to the 3.2.0 milestone Jul 31, 2023
@github-actions
Copy link

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label Aug 31, 2023
@Technoboy- Technoboy- modified the milestones: 3.2.0, 3.3.0 Dec 22, 2023
@coderzc coderzc modified the milestones: 3.3.0, 3.4.0 May 8, 2024
@lhotari lhotari modified the milestones: 4.0.0, 4.1.0 Oct 14, 2024
@lhotari
Copy link
Member

lhotari commented Nov 29, 2024

already resolved with #20085

@lhotari lhotari closed this Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Security] Pulsar client uses snakeyaml version and dependent on it libraries that have CVE-2022-1471
6 participants