Skip to content

Conversation

fuatbasik
Copy link

@fuatbasik fuatbasik commented Sep 1, 2025

Description of PR

This PR enables Analytics Stream to retry connection failures on stream from S3 to AAL up to fs.s3a.retry.limit times.
Note on idempotancy: As AAL maintains an internal state of stream, it handles re-opening stream from the blocks that are not filled yet on its own. This PR adds that logic to PhysicalIO: awslabs/analytics-accelerator-s3#340

How was this patch tested?

Added new tests with FlakyAsynClient that throws exception twice and succeeds at the third call.

For code changes:

There is no Jira item for this task.

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [N/A ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [N/A ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@fuatbasik fuatbasik changed the title Add Retries for InputStream exceptions Add Retries for InputStream exceptions on Analytics Stream Sep 1, 2025
@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 21m 17s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 9m 2s Maven dependency ordering for branch
+1 💚 mvninstall 38m 42s trunk passed
+1 💚 compile 17m 42s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 compile 15m 9s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 checkstyle 4m 39s trunk passed
+1 💚 mvnsite 1m 35s trunk passed
+1 💚 javadoc 1m 33s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 23s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+0 🆗 spotbugs 0m 41s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 42m 10s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 34s Maven dependency ordering for patch
+1 💚 mvninstall 0m 47s the patch passed
+1 💚 compile 17m 2s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javac 17m 2s the patch passed
+1 💚 compile 15m 10s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 javac 15m 10s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 4m 35s /results-checkstyle-root.txt root: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
+1 💚 mvnsite 1m 34s the patch passed
+1 💚 javadoc 1m 29s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 26s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+0 🆗 spotbugs 0m 38s hadoop-project has no data from spotbugs
+1 💚 shadedclient 41m 46s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 37s hadoop-project in the patch passed.
+1 💚 unit 3m 34s hadoop-aws in the patch passed.
+1 💚 asflicense 1m 6s The patch does not generate ASF License warnings.
251m 36s
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7923/1/artifact/out/Dockerfile
GITHUB PR #7923
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname Linux 0ca0a9974eb2 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / a6e2f21
Default Java Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7923/1/testReport/
Max. process+thread count 537 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-tools/hadoop-aws U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7923/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants