TC Software Update 2.7 Verify events during update #40656
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jtrejoespinoza-grid
changed the title
jtrejoespinoza-grid
changed the title
|
Progress update: Completed:
Worked in several apps configuration that allows how to trigger events for idle, applying and consent. Example for user consent Example for Delayed Events still missing to trigger
Overal actions missing:
|
Removed manual yaml test.
…from 1-4. Missing download error and AwaitNextAction. Removed manual test case Test_TC_SU_2_7
|
PR #40656: Size comparison from f1631b6 to 2c9d8f7 Full report (37 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, nxp, psoc6, qpg, realtek, stm32, telink)
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #40656 +/- ##
==========================================
+ Coverage 50.78% 50.81% +0.03%
==========================================
Files 1358 1364 +6
Lines 99515 100190 +675
Branches 12882 13011 +129
==========================================
+ Hits 50537 50913 +376
- Misses 48978 49277 +299 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…rom SU 2.7 Steps 1 - 2 work continuously. Steps 3 need to fix issue of the state recorded by the requestor that will needed for following steps.
jtrejoespinoza-grid
changed the title
…onApplied after restart
juandediosg
reviewed
Sep 11, 2025
src/python_testing/matter_testing_infrastructure/matter/testing/event_attribute_reporting.py
Show resolved
Hide resolved
juandediosg
reviewed
Sep 11, 2025
juandediosg
reviewed
Sep 11, 2025
src/python_testing/TC_SU_2_7.py
Outdated
| asserts.assert_equal(event_report.newState, self.ota_req.Enums.UpdateStateEnum.kQuerying) | ||
| event_report = state_transition_event_handler.wait_for_event_report(self.ota_req.Events.StateTransition, timeout_sec=60*5) | ||
| logger.info(f"Event response : {event_report}") | ||
| self._verify_event_transition_status(event_report, previous_state=self.ota_req.Enums.UpdateStateEnum.kQuerying, |
There was a problem hiding this comment.
This verifies a transition from Querying -> Idle.
According to Matters_V_1_4_TestPlanVerification, the expected transition should be from Downloading (state 4) -> Idle (state 1). The approach should be killing the OTA Provider app before issuing the QueryImage.
Wondering if there is a way to simulate the same condition to be aligned with the test, or if the expected flow has changed?
There was a problem hiding this comment.
Follow-up: After re-checking, the expected transition Downloading -> Idle transition can't be validated right now due to the known limitation (OTA: Provider cannot gracefully cancel a BDX transfer during OTA update #685):
- If the Provider is killed mid-BDX transfer, the Requestor stays stuck in
Downloadingand won’t return toIdleuntil the 6-hour timeout. - This is caused by the current implementation where there's no graceful way to cancel BDX (the Requestor needs a
StatusReportfrom the Provider, which can't be sent once it's killed).
Current approach Querying -> Idle is a reasonable workaround given the known issue, but I suggest checking with client whether this step should be skipped until the bug is resolved, or kept as is.
juandediosg
reviewed
Sep 11, 2025
| asserts.assert_equal(event_report.newState, self.ota_req.Enums.UpdateStateEnum.kDownloading) | ||
| # Wait some time to let it download some data and then Kill the current process and remove kvs files | ||
| logger.info("Wait 10 seconds to allow download some data") | ||
| sleep(10) |
There was a problem hiding this comment.
I noticed that sleep is used instead of await asyncio.sleep throughout the test. Is there a specific reason?
asirko-soft
approved these changes
Sep 25, 2025
|
|
||
| def __init__(self, app: str, storage_dir: str, discriminator: int, | ||
| passcode: int, ota_source: Union[OtaImagePath, ImageListPath], | ||
| port: int = 5541, extra_args: list[str] = [], f_stdout: BinaryIO = stdout.buffer, f_stderr: BinaryIO = stdout.buffer): |
There was a problem hiding this comment.
just thinking, maybe we can avoid signature duplicating defaults already defined in AppServerSubprocess, by passing **kwargs? This way child class pass arguments up to the parent without re-defining defaults that are already set there.
|
PR #40656: Size comparison from b0f2fa3 to df7189b Full report (3 builds for realtek, stm32)
|
This reverts commit 183cd17ce11d79045a71ad4c5df06e0b2e999691.
|
PR #40656: Size comparison from 876fcf5 to 8312097 Full report (5 builds for cc32xx, realtek, stm32)
|
|
PR #40656: Size comparison from 876fcf5 to a108d35 Full report (3 builds for realtek, stm32)
|
…tead of event to then wait for Event DelayedOnApply
|
PR #40656: Size comparison from 2c0b7fc to 00e8549 Full report (35 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, nxp, psoc6, qpg, realtek, stm32, telink)
|
…ine in run_python_test to kill the previous app before start to avoid port collision issues.
|
PR #40656: Size comparison from 2c0b7fc to 58a5f61 Full report (35 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, nxp, psoc6, qpg, realtek, stm32, telink)
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR contains the test case TC_SU_2.7 which verify the events during the software update.
This scripts closes and open the provider_app and the requestor_app as needed this is required because different configuration for both apps is required.
This script also requires different ota_versions ( 2 -4 ) in order to run properly as the updates are completed.
Related issues
#Fixes project-chip/matter-test-scripts#657
Testing
Terminal 1
Terminal 2: