Python: Add S3 Batch scenario with CloudFormation integration #7531
Conversation
github-actions
bot
added
the
Python
brmur
added
the
Feature Scenario
|
@aBurmeseDev is this ready for re-review? |
There was a problem hiding this comment.
@aBurmeseDev , this is looking really close now - only a few questions and small issues here, once those are resolved and I can successfully run the scenario and tests, I'll approve it.
| excerpts: | ||
| - description: Learn S3 Batch Basics Scenario. | ||
| snippet_tags: | ||
| - python.example_code.s3control.Batch.scenario |
There was a problem hiding this comment.
This tag doesn't seem to exist in the code.
| excerpts: | ||
| - description: | ||
| snippet_tags: | ||
| - python.example_code.s3control.update_job_status |
There was a problem hiding this comment.
This tag doesn't seem to exist in the code.
| To run this workflow, pull AWS tokens and run the command below: | ||
|
|
||
| ```bash | ||
| python s3_batch.py |
There was a problem hiding this comment.
Should it be s3_batch_scenario.py now?
|
|
||
| ## Additional resources | ||
|
|
||
| - [Amazon S3 Developer Guide](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) |
There was a problem hiding this comment.
This is linking to an object-lock page, did you mean to link to something for S3 Batch?
| print("\n" + self.DASHES) | ||
| print("3. Cancel the S3 Batch job") | ||
| cancel_job = q.ask("Do you want to cancel the Batch job? (y/n): ", q.is_yesno) | ||
| if cancel_job: |
There was a problem hiding this comment.
If I choose y here, I get an error and am unable to continue the scenario. Does that happen for you?
| self.cfn_helper.destroy_cloudformation_stack(self.STACK_NAME) | ||
|
|
||
| except Exception as e: | ||
| print(f"An error occurred: {e}") |
There was a problem hiding this comment.
If the scenario fails, we should try to clean up whatever exists: bucket, stack, etc.
| import pytest | ||
| from moto import mock_s3, mock_s3control, mock_sts | ||
|
|
||
| from test_s3_batch_stubber import S3BatchStubber |
There was a problem hiding this comment.
You should be able to put the stubber in test_tools, and then add an import for it. Take a look at the test setup for the conditional_requests scenario. You may then want to run the tests from a higher directory, or let me know how you're running them so I can verify.
This PR adds code example that demonstrates how to use the AWS SDK for Python (boto3) to work with S3 Batch Operations. The scenario covers various operations such as creating an AWS Batch compute environment, creating a job queue, creating a job definition, and submitting a job.
The example includes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.