Skip to content

Templated dts e2e tests #863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: develop
Choose a base branch
from
Open

Templated dts e2e tests #863

wants to merge 6 commits into from

Conversation

m-iwanicki
Copy link
Contributor

@m-iwanicki m-iwanicki commented Jun 5, 2025 •
edited
Loading

  • Update our fork of robotframework to 7.x
  • Test
  • Fix failing tests

As of now template generates this tests:

robot -t "Print All Test Cases To Be Generated" dts/dts-e2e.robot
==============================================================================
Dts-E2E
==============================================================================
Print All Test Cases To Be Generated
.msi-pro-z690-a-ddr5 Initial Deployment - Community Version
msi-pro-z690-a-ddr5 Initial Deployment - DPP
msi-pro-z690-a-ddr5 Update - Community Version
msi-pro-z690-a-ddr5 Update - DPP
msi-pro-z690-a-ddr5 Heads Transition - DPP
msi-pro-z690-a-wifi-ddr4 Initial Deployment - Community Version
msi-pro-z690-a-wifi-ddr4 Initial Deployment - DPP
msi-pro-z690-a-wifi-ddr4 Update - Community Version
msi-pro-z690-a-wifi-ddr4 Update - DPP
msi-pro-z690-a-wifi-ddr4 Heads Transition - DPP
msi-pro-z790-p-ddr5 Initial Deployment - DPP
msi-pro-z790-p-ddr5 Update - DPP
msi-pro-z790-p-ddr5 Heads Transition - DPP
novacustom-ns50mu Initial Deployment - Community Version
novacustom-ns50mu Update - Community Version
novacustom-ns50pu Initial Deployment - Community Version
novacustom-ns50pu Update - Community Version
novacustom-ns70mu Initial Deployment - Community Version
novacustom-ns70mu Update - Community Version
novacustom-ns70pu Initial Deployment - Community Version
novacustom-ns70pu Update - Community Version
novacustom-nv41mb Initial Deployment - Community Version
novacustom-nv41mb Update - Community Version
novacustom-nv41mz Initial Deployment - Community Version
novacustom-nv41mz Update - Community Version
novacustom-nv41pz Initial Deployment - Community Version
novacustom-nv41pz Update - Community Version
novacustom-nv41pz Heads Transition - DPP
novacustom-v540tnd Initial Deployment - Community Version
novacustom-v540tnd Update - Community Version
novacustom-v540tnd Heads Transition - DPP
novacustom-v540tu Initial Deployment - Community Version
novacustom-v540tu Update - Community Version
novacustom-v540tu Heads Transition - DPP
novacustom-v560tnd Initial Deployment - Community Version
novacustom-v560tnd Update - Community Version
novacustom-v560tnd Heads Transition - DPP
novacustom-v560tne Initial Deployment - Community Version
novacustom-v560tne Update - Community Version
novacustom-v560tne Heads Transition - DPP
novacustom-v560tu Initial Deployment - Community Version
novacustom-v560tu Update - Community Version
novacustom-v560tu Heads Transition - DPP
optiplex-7010 Initial Deployment - DPP
optiplex-7010 Update - DPP
optiplex-9010 Initial Deployment - DPP
optiplex-9010 Update - DPP
pcengines-apu2 Initial Deployment - DPP
pcengines-apu2 Update - DPP
pcengines-apu3 Initial Deployment - DPP
pcengines-apu3 Update - DPP
pcengines-apu4 Initial Deployment - DPP
pcengines-apu4 Update - DPP
pcengines-apu6 Initial Deployment - DPP
pcengines-apu6 Update - DPP
Print All Test Cases To Be Generated                                  | PASS |
------------------------------------------------------------------------------
Dts-E2E                                                               | PASS |
1 test, 1 passed, 0 failed
==============================================================================

with exports that'll be set in shell for each one:

Details 

==============================================================================
Dts-E2E                                                                       
==============================================================================
Print All Test Cases To Be Generated                                  | PASS |
------------------------------------------------------------------------------
E2E001: msi-pro-z690-a-ddr5,Initial Deployment,Community Version      ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI (MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E002: msi-pro-z690-a-ddr5,Initial Deployment,DPP                    ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI (MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E003: msi-pro-z690-a-ddr5,Update,Community Version                  ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI (MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E004: msi-pro-z690-a-ddr5,Update,DPP                                ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI (MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E005: msi-pro-z690-a-ddr5,Heads Transition,DPP                      ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI (MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E006: msi-pro-z690-a-wifi-ddr4,Initial Deployment,Community Version ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI DDR4(MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E007: msi-pro-z690-a-wifi-ddr4,Initial Deployment,DPP               ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI DDR4(MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E008: msi-pro-z690-a-wifi-ddr4,Update,Community Version             ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI DDR4(MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E009: msi-pro-z690-a-wifi-ddr4,Update,DPP                           ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI DDR4(MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E010: msi-pro-z690-a-wifi-ddr4,Heads Transition,DPP                 ""
export TEST_SYSTEM_MODEL=MS-7D25
export TEST_BOARD_MODEL=PRO Z690-A WIFI DDR4(MS-7D25)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E011: msi-pro-z790-p-ddr5,Initial Deployment,DPP                    ""
export TEST_SYSTEM_MODEL=MS-7E06
export TEST_BOARD_MODEL=PRO Z790-P WIFI (MS-7E06)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E012: msi-pro-z790-p-ddr5,Update,DPP                                ""
export TEST_SYSTEM_MODEL=MS-7E06
export TEST_BOARD_MODEL=PRO Z790-P WIFI (MS-7E06)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E013: msi-pro-z790-p-ddr5,Heads Transition,DPP                      ""
export TEST_SYSTEM_MODEL=MS-7E06
export TEST_BOARD_MODEL=PRO Z790-P WIFI (MS-7E06)
export TEST_SYSTEM_VENDOR=Micro-Star International Co., Ltd.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E014: novacustom-ns50mu,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=NS50_70MU
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E015: novacustom-ns50mu,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=NS50_70MU
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E016: novacustom-ns50pu,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=NS5x_NS7xPU
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E017: novacustom-ns50pu,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=NS5x_NS7xPU
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E018: novacustom-ns70mu,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=NS50_70MU
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E019: novacustom-ns70mu,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=NS50_70MU
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E020: novacustom-ns70pu,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=NS5x_NS7xPU
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E021: novacustom-ns70pu,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=NS5x_NS7xPU
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E022: novacustom-nv41mb,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=NV4XMB,ME,MZ
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E023: novacustom-nv41mb,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=NV4XMB,ME,MZ
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E024: novacustom-nv41mz,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=NV4XMB,ME,MZ
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E025: novacustom-nv41mz,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=NV4XMB,ME,MZ
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E026: novacustom-nv41pz,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=NV4xPZ
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E027: novacustom-nv41pz,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=NV4xPZ
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E028: novacustom-nv41pz,Heads Transition,DPP                        ""
export TEST_SYSTEM_MODEL=NV4xPZ
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E029: novacustom-v540tnd,Initial Deployment,Community Version       ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V540TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export TEST_NOVACUSTOM_MODEL=V540TNx
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E030: novacustom-v540tnd,Update,Community Version                   ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V540TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V540TNx
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E031: novacustom-v540tnd,Heads Transition,DPP                       ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V540TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V540TNx
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E032: novacustom-v540tu,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=V54x_6x_TU
export TEST_BOARD_MODEL=V540TU
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export TEST_NOVACUSTOM_MODEL=V540TU
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E033: novacustom-v540tu,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=V54x_6x_TU
export TEST_BOARD_MODEL=V540TU
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V540TU
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E034: novacustom-v540tu,Heads Transition,DPP                        ""
export TEST_SYSTEM_MODEL=V54x_6x_TU
export TEST_BOARD_MODEL=V540TU
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V540TU
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E035: novacustom-v560tnd,Initial Deployment,Community Version       ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V560TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export TEST_NOVACUSTOM_MODEL=V560TNx
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E036: novacustom-v560tnd,Update,Community Version                   ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V560TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V560TNx
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E037: novacustom-v560tnd,Heads Transition,DPP                       ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V560TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V560TNx
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E038: novacustom-v560tne,Initial Deployment,Community Version       ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V560TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export TEST_NOVACUSTOM_MODEL=V560TNx
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E039: novacustom-v560tne,Update,Community Version                   ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V560TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V560TNx
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E040: novacustom-v560tne,Heads Transition,DPP                       ""
export TEST_SYSTEM_MODEL=V5xTNC_TND_TNE
export TEST_BOARD_MODEL=V560TNx
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V560TNx
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E041: novacustom-v560tu,Initial Deployment,Community Version        ""
export TEST_SYSTEM_MODEL=V54x_6x_TU
export TEST_BOARD_MODEL=V560TU
export TEST_SYSTEM_VENDOR=Notebook
export TEST_BIOS_VERSION=0.0.0
export TEST_NOVACUSTOM_MODEL=V560TU
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E042: novacustom-v560tu,Update,Community Version                    ""
export TEST_SYSTEM_MODEL=V54x_6x_TU
export TEST_BOARD_MODEL=V560TU
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V560TU
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E043: novacustom-v560tu,Heads Transition,DPP                        ""
export TEST_SYSTEM_MODEL=V54x_6x_TU
export TEST_BOARD_MODEL=V560TU
export TEST_SYSTEM_VENDOR=Notebook
export TEST_NOVACUSTOM_MODEL=V560TU
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E044: optiplex-7010,Initial Deployment,DPP                          ""
export TEST_SYSTEM_MODEL=OptiPlex 7010
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Dell Inc.
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E045: optiplex-7010,Update,DPP                                      ""
export TEST_SYSTEM_MODEL=OptiPlex 7010
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Dell Inc.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E046: optiplex-9010,Initial Deployment,DPP                          ""
export TEST_SYSTEM_MODEL=OptiPlex 7010
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Dell Inc.
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E047: optiplex-9010,Update,DPP                                      ""
export TEST_SYSTEM_MODEL=OptiPlex 7010
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=Dell Inc.
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E048: pcengines-apu2,Initial Deployment,DPP                         ""
export TEST_SYSTEM_MODEL=APU2
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=PC Engines
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E049: pcengines-apu2,Update,DPP                                     ""
export TEST_SYSTEM_MODEL=APU2
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=PC Engines
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E050: pcengines-apu3,Initial Deployment,DPP                         ""
export TEST_SYSTEM_MODEL=APU3
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=PC Engines
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E051: pcengines-apu3,Update,DPP                                     ""
export TEST_SYSTEM_MODEL=APU3
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=PC Engines
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E052: pcengines-apu4,Initial Deployment,DPP                         ""
export TEST_SYSTEM_MODEL=APU4
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=PC Engines
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E053: pcengines-apu4,Update,DPP                                     ""
export TEST_SYSTEM_MODEL=APU4
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=PC Engines
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E054: pcengines-apu6,Initial Deployment,DPP                         ""
export TEST_SYSTEM_MODEL=APU6
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=PC Engines
export TEST_BIOS_VERSION=0.0.0
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
E2E055: pcengines-apu6,Update,DPP                                     ""
export TEST_SYSTEM_MODEL=APU6
export TEST_BOARD_MODEL=
export TEST_SYSTEM_VENDOR=PC Engines
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
| PASS |
------------------------------------------------------------------------------
Dts-E2E                                                               | PASS |
56 tests, 56 passed, 0 failed
==============================================================================

@m-iwanicki m-iwanicki self-assigned this Jun 5, 2025
@m-iwanicki m-iwanicki mentioned this pull request Jun 5, 2025
Open
@m-iwanicki
Copy link
Contributor Author

Current state:

56 tests, 43 passed, 13 failed

Most of those if not all can be fixed with tweaking platform config (e.g. to set correct version string when updating)

Details 

==============================================================================
Dts-E2E
==============================================================================
Create tests                                                          | PASS |
------------------------------------------------------------------------------
E2E001: msi-pro-z690-a-ddr5 Initial Deployment - Community Version    | PASS |
------------------------------------------------------------------------------
E2E002: msi-pro-z690-a-ddr5 Initial Deployment - DPP                  | PASS |
------------------------------------------------------------------------------
E2E003: msi-pro-z690-a-ddr5 Update - Community Version                | PASS |
------------------------------------------------------------------------------
E2E004: msi-pro-z690-a-ddr5 Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E005: msi-pro-z690-a-ddr5 Heads Transition - DPP                    | PASS |
------------------------------------------------------------------------------
E2E006: msi-pro-z690-a-wifi-ddr4 Initial Deployment - Community Ve... | PASS |
------------------------------------------------------------------------------
E2E007: msi-pro-z690-a-wifi-ddr4 Initial Deployment - DPP             | PASS |
------------------------------------------------------------------------------
E2E008: msi-pro-z690-a-wifi-ddr4 Update - Community Version           | PASS |
------------------------------------------------------------------------------
E2E009: msi-pro-z690-a-wifi-ddr4 Update - DPP                         | PASS |
------------------------------------------------------------------------------
E2E010: msi-pro-z690-a-wifi-ddr4 Heads Transition - DPP               | PASS |
------------------------------------------------------------------------------
E2E011: msi-pro-z790-p-ddr5 Initial Deployment - DPP                  | PASS |
------------------------------------------------------------------------------
E2E012: msi-pro-z790-p-ddr5 Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E013: msi-pro-z790-p-ddr5 Heads Transition - DPP                    | PASS |
------------------------------------------------------------------------------
E2E014: novacustom-ns50mu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E015: novacustom-ns50mu Update - Community Version                  | FAIL |
No match found for 'Does\ it\ match\ your\ actual\ specification\?\ \(Y\|n\)|Would\ you\ like\ to\ switch\ to\ Dasharo\ heads\ firmware\?\ \(Y\|n\)' in 2 minutes
Output:
2^J
Detected NovaCustom hardware with version < 1.3.0
Need to perform EC transition after which the platform will turn off
Then, please power it on and proceed with update again
EC transition procedure will start in 5 seconds
Checking for Open Source Embedded Controller firmware...
Device has already Open Source Embedded Controller firmware, aborting...
Could not perform EC transition : (1)
Press Enter to continue.root@DasharoToolsSuite:~# .
------------------------------------------------------------------------------
E2E016: novacustom-ns50pu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E017: novacustom-ns50pu Update - Community Version                  | PASS |
------------------------------------------------------------------------------
E2E018: novacustom-ns70mu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E019: novacustom-ns70mu Update - Community Version                  | FAIL |
No match found for 'Does\ it\ match\ your\ actual\ specification\?\ \(Y\|n\)|Would\ you\ like\ to\ switch\ to\ Dasharo\ heads\ firmware\?\ \(Y\|n\)' in 2 minutes
Output:
2^J
Detected NovaCustom hardware with version < 1.3.0
Need to perform EC transition after which the platform will turn off
Then, please power it on and proceed with update again
EC transition procedure will start in 5 seconds
Checking for Open Source Embedded Controller firmware...
Device has already Open Source Embedded Controller firmware, aborting...
Could not perform EC transition : (1)
Press Enter to continue.root@DasharoToolsSuite:~# .
------------------------------------------------------------------------------
E2E020: novacustom-ns70pu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E021: novacustom-ns70pu Update - Community Version                  | PASS |
------------------------------------------------------------------------------
E2E022: novacustom-nv41mb Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E023: novacustom-nv41mb Update - Community Version                  | FAIL |
No match found for 'Does\ it\ match\ your\ actual\ specification\?\ \(Y\|n\)|Would\ you\ like\ to\ switch\ to\ Dasharo\ heads\ firmware\?\ \(Y\|n\)' in 2 minutes
Output:
2^J
Detected NovaCustom hardware with version < 1.3.0
Need to perform EC transition after which the platform will turn off
Then, please power it on and proceed with update again
EC transition procedure will start in 5 seconds
Checking for Open Source Embedded Controller firmware...
Device has already Open Source Embedded Controller firmware, aborting...
Could not perform EC transition : (1)
Press Enter to continue.root@DasharoToolsSuite:~# .
------------------------------------------------------------------------------
E2E024: novacustom-nv41mz Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E025: novacustom-nv41mz Update - Community Version                  | FAIL |
No match found for 'Does\ it\ match\ your\ actual\ specification\?\ \(Y\|n\)|Would\ you\ like\ to\ switch\ to\ Dasharo\ heads\ firmware\?\ \(Y\|n\)' in 2 minutes
Output:
2^J
Detected NovaCustom hardware with version < 1.3.0
Need to perform EC transition after which the platform will turn off
Then, please power it on and proceed with update again
EC transition procedure will start in 5 seconds
Checking for Open Source Embedded Controller firmware...
Device has already Open Source Embedded Controller firmware, aborting...
Could not perform EC transition : (1)
Press Enter to continue.root@DasharoToolsSuite:~# .
------------------------------------------------------------------------------
E2E026: novacustom-nv41pz Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E027: novacustom-nv41pz Update - Community Version                  | PASS |
------------------------------------------------------------------------------
E2E028: novacustom-nv41pz Heads Transition - DPP                      | FAIL |
No match found for 'Would you like to switch to Dasharo heads firmware? (Y|n)' in 2 minutes
Output:

 Dasharo Tools Suite Script 2.4.0
 (c) Dasharo <[email protected]>
 Report issues at: https://github.com/Dasharo/dasharo-issues
*********************************************************
**                HARDWARE INFORMATION
*********************************************************
**    System Inf.: Notebook NV4xPZ
** Baseboard Inf.: Notebook
**       CPU Inf.:
*********************************************************
**                FIRMWARE INFORMATION
*********************************************************
** BIOS Inf.: 3mdeb Dasharo (coreboot+UEFI) v0.0.0
*********************************************************
**                DPP credentials
    [ Message content over the limit has been removed. ]

Please verify detected hardware!

Board vendor: Notebook
System model: NV4xPZ

Does it match your actual specification? (Y|n)
Following firmware will be used to deploy Dasharo:
Dasharo BIOS firmware:
  - link: https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_nv4x_adl/v1.7.2/novacustom_nv4x_adl_v1.7.2.rom
  - hash: 00b6338389cc5d020b641629971aac6d4047be6134c6e8d0228140edc42584f6
Dasharo EC firmware:
  - link: https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_nv4x_adl/v1.7.2/novacustom_nv4x_adl_ec_v1.7.2.rom
  - hash: adbd651646789f5ac4c9d1835f0d2bf502192fd9476cd77b9b22fa077c5403e8

You can learn more about this release on: https://docs.dasharo.com/

Do you want to deploy this Dasharo Firmware on your platform (Y|n) .
------------------------------------------------------------------------------
E2E029: novacustom-v540tnd Initial Deployment - Community Version     | FAIL |
No match found for 'c) Community version' in 2 minutes
Output:
(if you answer "yes", then command hw-probe --all --upload will be ran, in
order to participate)

[N/y] Please consider contributing to the "Hardware for Linux" project in the future.
    All you have to do is run this command:
    hw-probe --all --upload

Gathering flash chip and chipset information...
Flash information: Test Flash Chip
Flash size: 2M
Waiting for network connection ...
Y
Network connection have been established!
Checking if board is Dasharo compatible.
Choose your board model:
  0. None below
  1: V540TNx
  2: V560TNx

Enter an option:
/usr/sbin/dts-functions.sh: line 201: [: Y: integer expression expected
/usr/sbin/dts-functions.sh: line 205: [: Y: integer expression expected
Choose your board model:
  0. None below
  1: V540TNx
  2: V560TNx

Enter an option: .
------------------------------------------------------------------------------
E2E030: novacustom-v540tnd Update - Community Version                 | PASS |
------------------------------------------------------------------------------
E2E031: novacustom-v540tnd Heads Transition - DPP                     | FAIL |
No match found for 'Would you like to switch to Dasharo heads firmware? (Y|n)' in 2 minutes
Output:

 Dasharo Tools Suite Script 2.4.0
 (c) Dasharo <[email protected]>
 Report issues at: https://github.com/Dasharo/dasharo-issues
*********************************************************
**                HARDWARE INFORMATION
*********************************************************
**    System Inf.: Notebook V5xTNC_TND_TNE
** Baseboard Inf.: Notebook V540TNx
**       CPU Inf.:
*********************************************************
**                FIRMWARE INFORMATION
*********************************************************
** BIOS Inf.: 3mdeb Dasharo (coreboot+UEFI) v0.0.0
*********************************************************
**                DPP credentials
    [ Message content over the limit has been removed. ]
Please verify detected hardware!

Board vendor: Notebook
System model: V5xTNC_TND_TNE
Board model: V540TNx

Does it match your actual specification? (Y|n)
Following firmware will be used to deploy Dasharo:
Dasharo BIOS firmware:
  - link: https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v54x_mtl/v0.9.1/novacustom_v54x_mtl_v0.9.1.rom
  - hash: 430c0f7a099ef068a20de80e22b58d94e2614b782990dc5a91e1d8affc25cbf1
Dasharo EC firmware:
  - link: https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v54x_mtl/v0.9.1/novacustom_v54x_mtl_ec_v0.9.1.rom
  - hash: 8ea33760138e5d7338d375d2c1bbed7d5be55441dfec0599872855ae6d6018e2

You can learn more about this release on: https://docs.dasharo.com/

Do you want to deploy this Dasharo Firmware on your platform (Y|n) .
------------------------------------------------------------------------------
E2E032: novacustom-v540tu Initial Deployment - Community Version      | FAIL |
No match found for 'c) Community version' in 2 minutes
Output:
(if you answer "yes", then command hw-probe --all --upload will be ran, in
order to participate)

[N/y] Please consider contributing to the "Hardware for Linux" project in the future.
    All you have to do is run this command:
    hw-probe --all --upload

Gathering flash chip and chipset information...
Flash information: Test Flash Chip
Flash size: 2M
Waiting for network connection ...
Y
Network connection have been established!
Checking if board is Dasharo compatible.
Choose your board model:
  0. None below
  1: V540TU
  2: V560TU

Enter an option:
/usr/sbin/dts-functions.sh: line 201: [: Y: integer expression expected
/usr/sbin/dts-functions.sh: line 205: [: Y: integer expression expected
Choose your board model:
  0. None below
  1: V540TU
  2: V560TU

Enter an option: .
------------------------------------------------------------------------------
E2E033: novacustom-v540tu Update - Community Version                  | PASS |
------------------------------------------------------------------------------
E2E034: novacustom-v540tu Heads Transition - DPP                      | PASS |
------------------------------------------------------------------------------
E2E035: novacustom-v560tnd Initial Deployment - Community Version     | FAIL |
No match found for 'c) Community version' in 2 minutes
Output:
(if you answer "yes", then command hw-probe --all --upload will be ran, in
order to participate)

[N/y] Please consider contributing to the "Hardware for Linux" project in the future.
    All you have to do is run this command:
    hw-probe --all --upload

Gathering flash chip and chipset information...
Flash information: Test Flash Chip
Flash size: 2M
Waiting for network connection ...
Y
Network connection have been established!
Checking if board is Dasharo compatible.
Choose your board model:
  0. None below
  1: V540TNx
  2: V560TNx

Enter an option:
/usr/sbin/dts-functions.sh: line 201: [: Y: integer expression expected
/usr/sbin/dts-functions.sh: line 205: [: Y: integer expression expected
Choose your board model:
  0. None below
  1: V540TNx
  2: V560TNx

Enter an option: .
------------------------------------------------------------------------------
E2E036: novacustom-v560tnd Update - Community Version                 | PASS |
------------------------------------------------------------------------------
E2E037: novacustom-v560tnd Heads Transition - DPP                     | FAIL |
No match found for 'Would you like to switch to Dasharo heads firmware? (Y|n)' in 2 minutes
Output:

 Dasharo Tools Suite Script 2.4.0
 (c) Dasharo <[email protected]>
 Report issues at: https://github.com/Dasharo/dasharo-issues
*********************************************************
**                HARDWARE INFORMATION
*********************************************************
**    System Inf.: Notebook V5xTNC_TND_TNE
** Baseboard Inf.: Notebook V560TNx
**       CPU Inf.:
*********************************************************
**                FIRMWARE INFORMATION
*********************************************************
** BIOS Inf.: 3mdeb Dasharo (coreboot+UEFI) v0.0.0
*********************************************************
**                DPP credentials
    [ Message content over the limit has been removed. ]
Please verify detected hardware!

Board vendor: Notebook
System model: V5xTNC_TND_TNE
Board model: V560TNx

Does it match your actual specification? (Y|n)
Following firmware will be used to deploy Dasharo:
Dasharo BIOS firmware:
  - link: https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v56x_mtl/v0.9.1/novacustom_v56x_mtl_v0.9.1.rom
  - hash: d06adae481b8939e912af4c76a194ecf04cf8417520df29cd3cdaf2d621d268e
Dasharo EC firmware:
  - link: https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v56x_mtl/v0.9.1/novacustom_v56x_mtl_ec_v0.9.1.rom
  - hash: 54e4573b1fcd832a6cc72d4fd14670e2daa816c77133008c0942b05f6b1de960

You can learn more about this release on: https://docs.dasharo.com/

Do you want to deploy this Dasharo Firmware on your platform (Y|n) .
------------------------------------------------------------------------------
E2E038: novacustom-v560tne Initial Deployment - Community Version     | FAIL |
No match found for 'c) Community version' in 2 minutes
Output:
(if you answer "yes", then command hw-probe --all --upload will be ran, in
order to participate)

[N/y] Please consider contributing to the "Hardware for Linux" project in the future.
    All you have to do is run this command:
    hw-probe --all --upload

Gathering flash chip and chipset information...
Flash information: Test Flash Chip
Flash size: 2M
Waiting for network connection ...
Y
Network connection have been established!
Checking if board is Dasharo compatible.
Choose your board model:
  0. None below
  1: V540TNx
  2: V560TNx

Enter an option:
/usr/sbin/dts-functions.sh: line 201: [: Y: integer expression expected
/usr/sbin/dts-functions.sh: line 205: [: Y: integer expression expected
Choose your board model:
  0. None below
  1: V540TNx
  2: V560TNx

Enter an option: .
------------------------------------------------------------------------------
E2E039: novacustom-v560tne Update - Community Version                 | PASS |
------------------------------------------------------------------------------
E2E040: novacustom-v560tne Heads Transition - DPP                     | FAIL |
No match found for 'Would you like to switch to Dasharo heads firmware? (Y|n)' in 2 minutes
Output:

 Dasharo Tools Suite Script 2.4.0
 (c) Dasharo <[email protected]>
 Report issues at: https://github.com/Dasharo/dasharo-issues
*********************************************************
**                HARDWARE INFORMATION
*********************************************************
**    System Inf.: Notebook V5xTNC_TND_TNE
** Baseboard Inf.: Notebook V560TNx
**       CPU Inf.:
*********************************************************
**                FIRMWARE INFORMATION
*********************************************************
** BIOS Inf.: 3mdeb Dasharo (coreboot+UEFI) v0.0.0
*********************************************************
**                DPP credentials
    [ Message content over the limit has been removed. ]
Please verify detected hardware!

Board vendor: Notebook
System model: V5xTNC_TND_TNE
Board model: V560TNx

Does it match your actual specification? (Y|n)
Following firmware will be used to deploy Dasharo:
Dasharo BIOS firmware:
  - link: https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v56x_mtl/v0.9.1/novacustom_v56x_mtl_v0.9.1.rom
  - hash: d06adae481b8939e912af4c76a194ecf04cf8417520df29cd3cdaf2d621d268e
Dasharo EC firmware:
  - link: https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v56x_mtl/v0.9.1/novacustom_v56x_mtl_ec_v0.9.1.rom
  - hash: 54e4573b1fcd832a6cc72d4fd14670e2daa816c77133008c0942b05f6b1de960

You can learn more about this release on: https://docs.dasharo.com/

Do you want to deploy this Dasharo Firmware on your platform (Y|n) .
------------------------------------------------------------------------------
E2E041: novacustom-v560tu Initial Deployment - Community Version      | FAIL |
No match found for 'c) Community version' in 2 minutes
Output:
(if you answer "yes", then command hw-probe --all --upload will be ran, in
order to participate)

[N/y] Please consider contributing to the "Hardware for Linux" project in the future.
    All you have to do is run this command:
    hw-probe --all --upload

Gathering flash chip and chipset information...
Flash information: Test Flash Chip
Flash size: 2M
Waiting for network connection ...
Y
Network connection have been established!
Checking if board is Dasharo compatible.
Choose your board model:
  0. None below
  1: V540TU
  2: V560TU

Enter an option:
/usr/sbin/dts-functions.sh: line 201: [: Y: integer expression expected
/usr/sbin/dts-functions.sh: line 205: [: Y: integer expression expected
Choose your board model:
  0. None below
  1: V540TU
  2: V560TU

Enter an option: .
------------------------------------------------------------------------------
E2E042: novacustom-v560tu Update - Community Version                  | PASS |
------------------------------------------------------------------------------
E2E043: novacustom-v560tu Heads Transition - DPP                      | PASS |
------------------------------------------------------------------------------
E2E044: optiplex-7010 Initial Deployment - DPP                        | PASS |
------------------------------------------------------------------------------
E2E045: optiplex-7010 Update - DPP                                    | PASS |
------------------------------------------------------------------------------
E2E046: optiplex-9010 Initial Deployment - DPP                        | PASS |
------------------------------------------------------------------------------
E2E047: optiplex-9010 Update - DPP                                    | PASS |
------------------------------------------------------------------------------
E2E048: pcengines-apu2 Initial Deployment - DPP                       | PASS |
------------------------------------------------------------------------------
E2E049: pcengines-apu2 Update - DPP                                   | PASS |
------------------------------------------------------------------------------
E2E050: pcengines-apu3 Initial Deployment - DPP                       | PASS |
------------------------------------------------------------------------------
E2E051: pcengines-apu3 Update - DPP                                   | PASS |
------------------------------------------------------------------------------
E2E052: pcengines-apu4 Initial Deployment - DPP                       | PASS |
------------------------------------------------------------------------------
E2E053: pcengines-apu4 Update - DPP                                   | PASS |
------------------------------------------------------------------------------
E2E054: pcengines-apu6 Initial Deployment - DPP                       | PASS |
------------------------------------------------------------------------------
E2E055: pcengines-apu6 Update - DPP                                   | PASS |
------------------------------------------------------------------------------
Dts-E2E                                                               | FAIL |
56 tests, 43 passed, 13 failed
==============================================================================

Also found a bug in DTS

Enter an option:
/usr/sbin/dts-functions.sh: line 201: [: Y: integer expression expected
/usr/sbin/dts-functions.sh: line 205: [: Y: integer expression expected
Choose your board model:
  0. None below
  1: V540TNx
  2: V560TNx

probably when trying to pass non-numeric value.

@m-iwanicki
Copy link
Contributor Author

Had to update robotframework to make this work, so this PR would have to be tested to make sure there are no regressions but would also resolve #606

@macpijan macpijan closed this Jul 3, 2025
@macpijan macpijan deleted the template-dts-e2e branch July 3, 2025 13:46
@m-iwanicki m-iwanicki restored the template-dts-e2e branch July 3, 2025 14:06
@m-iwanicki m-iwanicki reopened this Jul 3, 2025
@DaniilKl DaniilKl mentioned this pull request Jul 9, 2025
Open
@m-iwanicki m-iwanicki force-pushed the template-dts-e2e branch 3 times, most recently from 4bfb866 to 5261b4c Compare July 23, 2025 17:02
@m-iwanicki m-iwanicki marked this pull request as ready for review July 23, 2025 17:02
@m-iwanicki
Copy link
Contributor Author

m-iwanicki commented Jul 23, 2025 •
edited
Loading

To make sure all tests work boot into DTS with newest dts-scripts changes, otherwise SeaBIOS tests will fail

robot -b cmd_logs.txt -v rte_ip:127.0.0.1 -v config:qemu \
            -L TRACE \
            -v snipeit:no \
            -v boot_dts_from_ipxe_shell:True \
            -v dts_ipxe_link:http://localhost/dts.ipxe \
            -v dpp_email:'***' \
            -v dpp_password:'***' -t "Create tests" dts/dts-e2e.robot

Output:

Details 

==============================================================================
Dts-E2E
==============================================================================
Create tests                                                          | PASS |
------------------------------------------------------------------------------
E2E001: msi-pro-z690-a-ddr5 Initial Deployment - Community Version    | PASS |
------------------------------------------------------------------------------
E2E002: msi-pro-z690-a-ddr5 Initial Deployment - DPP                  | PASS |
------------------------------------------------------------------------------
E2E003: msi-pro-z690-a-ddr5 UEFI Update - Community Version           | PASS |
------------------------------------------------------------------------------
E2E004: msi-pro-z690-a-ddr5 UEFI Update - DPP                         | PASS |
------------------------------------------------------------------------------
E2E005: msi-pro-z690-a-ddr5 UEFI->Heads Transition - DPP              | PASS |
------------------------------------------------------------------------------
E2E006: msi-pro-z690-a-wifi-ddr4 Initial Deployment - Community Ve... | PASS |
------------------------------------------------------------------------------
E2E007: msi-pro-z690-a-wifi-ddr4 Initial Deployment - DPP             | PASS |
------------------------------------------------------------------------------
E2E008: msi-pro-z690-a-wifi-ddr4 UEFI Update - Community Version      | PASS |
------------------------------------------------------------------------------
E2E009: msi-pro-z690-a-wifi-ddr4 UEFI Update - DPP                    | PASS |
------------------------------------------------------------------------------
E2E010: msi-pro-z690-a-wifi-ddr4 UEFI->Heads Transition - DPP         | PASS |
------------------------------------------------------------------------------
E2E011: msi-pro-z790-p-ddr5 Initial Deployment - DPP                  | PASS |
------------------------------------------------------------------------------
E2E012: msi-pro-z790-p-ddr5 UEFI Update - DPP                         | PASS |
------------------------------------------------------------------------------
E2E013: msi-pro-z790-p-ddr5 UEFI->Heads Transition - DPP              | PASS |
------------------------------------------------------------------------------
E2E014: novacustom-ns50mu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E015: novacustom-ns50mu UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E016: novacustom-ns50pu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E017: novacustom-ns50pu UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E018: novacustom-ns70mu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E019: novacustom-ns70mu UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E020: novacustom-ns70pu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E021: novacustom-ns70pu UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E022: novacustom-nv41mb Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E023: novacustom-nv41mb UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E024: novacustom-nv41mz Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E025: novacustom-nv41mz UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E026: novacustom-nv41pz Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E027: novacustom-nv41pz UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E028: novacustom-nv41pz UEFI->Heads Transition - DPP                | PASS |
------------------------------------------------------------------------------
E2E029: novacustom-v540tnd Initial Deployment - Community Version     | PASS |
------------------------------------------------------------------------------
E2E030: novacustom-v540tnd UEFI Update - Community Version            | PASS |
------------------------------------------------------------------------------
E2E031: novacustom-v540tu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E032: novacustom-v540tu UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E033: novacustom-v540tu UEFI->Heads Transition - DPP                | PASS |
------------------------------------------------------------------------------
E2E034: novacustom-v560tnd Initial Deployment - Community Version     | PASS |
------------------------------------------------------------------------------
E2E035: novacustom-v560tnd UEFI Update - Community Version            | PASS |
------------------------------------------------------------------------------
E2E036: novacustom-v560tne Initial Deployment - Community Version     | PASS |
------------------------------------------------------------------------------
E2E037: novacustom-v560tne UEFI Update - Community Version            | PASS |
------------------------------------------------------------------------------
E2E038: novacustom-v560tu Initial Deployment - Community Version      | PASS |
------------------------------------------------------------------------------
E2E039: novacustom-v560tu UEFI Update - Community Version             | PASS |
------------------------------------------------------------------------------
E2E040: novacustom-v560tu UEFI->Heads Transition - DPP                | PASS |
------------------------------------------------------------------------------
E2E041: odroid-h4-plus Initial Deployment - DPP                       | PASS |
------------------------------------------------------------------------------
E2E042: odroid-h4-plus UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E043: optiplex-7010 Initial Deployment - DPP                        | PASS |
------------------------------------------------------------------------------
E2E044: optiplex-7010 UEFI Update - DPP                               | PASS |
------------------------------------------------------------------------------
E2E045: optiplex-9010 Initial Deployment - DPP                        | PASS |
------------------------------------------------------------------------------
E2E046: optiplex-9010 UEFI Update - DPP                               | PASS |
------------------------------------------------------------------------------
E2E047: pcengines-apu2 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E048: pcengines-apu2 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E049: pcengines-apu2 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
E2E050: pcengines-apu3 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E051: pcengines-apu3 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E052: pcengines-apu3 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
E2E053: pcengines-apu4 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E054: pcengines-apu4 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E055: pcengines-apu4 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
E2E056: pcengines-apu6 UEFI Update - DPP                              | PASS |
------------------------------------------------------------------------------
E2E057: pcengines-apu6 SeaBIOS Update - DPP                           | PASS |
------------------------------------------------------------------------------
E2E058: pcengines-apu6 SeaBIOS->UEFI Transition - DPP                 | PASS |
------------------------------------------------------------------------------
Dts-E2E                                                               | PASS |
59 tests, 59 passed, 0 failed
==============================================================================
Debug:   /home/miwanicki/projects/open-source-firmware-validation/cmd_logs.txt
Output:  /home/miwanicki/projects/open-source-firmware-validation/output.xml
Log:     /home/miwanicki/projects/open-source-firmware-validation/log.html
Report:  /home/miwanicki/projects/open-source-firmware-validation/report.html

To test make sure to create venv from requirements-rf7.txt.

To list all tests that'll be generated you can use (without starting QEMU):

robot -t "E2EH003*" dts/dts-e2e-helper.robot
Details 

==============================================================================
Dts-E2E-Helper
==============================================================================
E2EH003.001 Print names of test cases to be generated :: Print out...
..msi-pro-z690-a-ddr5 Initial Deployment - Community Version
msi-pro-z690-a-ddr5 Initial Deployment - DPP
msi-pro-z690-a-ddr5 UEFI Update - Community Version
msi-pro-z690-a-ddr5 UEFI Update - DPP
msi-pro-z690-a-ddr5 UEFI->Heads Transition - DPP
msi-pro-z690-a-wifi-ddr4 Initial Deployment - Community Version
msi-pro-z690-a-wifi-ddr4 Initial Deployment - DPP
msi-pro-z690-a-wifi-ddr4 UEFI Update - Community Version
msi-pro-z690-a-wifi-ddr4 UEFI Update - DPP
msi-pro-z690-a-wifi-ddr4 UEFI->Heads Transition - DPP
msi-pro-z790-p-ddr5 Initial Deployment - DPP
msi-pro-z790-p-ddr5 UEFI Update - DPP
msi-pro-z790-p-ddr5 UEFI->Heads Transition - DPP
novacustom-ns50mu Initial Deployment - Community Version
novacustom-ns50mu UEFI Update - Community Version
novacustom-ns50pu Initial Deployment - Community Version
novacustom-ns50pu UEFI Update - Community Version
novacustom-ns70mu Initial Deployment - Community Version
novacustom-ns70mu UEFI Update - Community Version
novacustom-ns70pu Initial Deployment - Community Version
novacustom-ns70pu UEFI Update - Community Version
novacustom-nv41mb Initial Deployment - Community Version
novacustom-nv41mb UEFI Update - Community Version
novacustom-nv41mz Initial Deployment - Community Version
novacustom-nv41mz UEFI Update - Community Version
novacustom-nv41pz Initial Deployment - Community Version
novacustom-nv41pz UEFI Update - Community Version
novacustom-nv41pz UEFI->Heads Transition - DPP
novacustom-v540tnd Initial Deployment - Community Version
novacustom-v540tnd UEFI Update - Community Version
novacustom-v540tu Initial Deployment - Community Version
novacustom-v540tu UEFI Update - Community Version
novacustom-v540tu UEFI->Heads Transition - DPP
novacustom-v560tnd Initial Deployment - Community Version
novacustom-v560tnd UEFI Update - Community Version
novacustom-v560tne Initial Deployment - Community Version
novacustom-v560tne UEFI Update - Community Version
novacustom-v560tu Initial Deployment - Community Version
novacustom-v560tu UEFI Update - Community Version
novacustom-v560tu UEFI->Heads Transition - DPP
odroid-h4-plus Initial Deployment - DPP
odroid-h4-plus UEFI Update - DPP
optiplex-7010 Initial Deployment - DPP
optiplex-7010 UEFI Update - DPP
optiplex-9010 Initial Deployment - DPP
optiplex-9010 UEFI Update - DPP
pcengines-apu2 UEFI Update - DPP
pcengines-apu2 SeaBIOS Update - DPP
pcengines-apu2 SeaBIOS->UEFI Transition - DPP
pcengines-apu3 UEFI Update - DPP
pcengines-apu3 SeaBIOS Update - DPP
pcengines-apu3 SeaBIOS->UEFI Transition - DPP
pcengines-apu4 UEFI Update - DPP
pcengines-apu4 SeaBIOS Update - DPP
pcengines-apu4 SeaBIOS->UEFI Transition - DPP
pcengines-apu6 UEFI Update - DPP
pcengines-apu6 SeaBIOS Update - DPP
pcengines-apu6 SeaBIOS->UEFI Transition - DPP
E2EH003.001 Print names of test cases to be generated :: Print out... | PASS |
------------------------------------------------------------------------------
Dts-E2E-Helper                                                        | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Output:  /home/miwanicki/projects/open-source-firmware-validation/output.xml
Log:     /home/miwanicki/projects/open-source-firmware-validation/log.html
Report:  /home/miwanicki/projects/open-source-firmware-validation/report.html

To print out exports used during every test, please run test E2EH001*. There is also E2EH002* which prints out exports generated for all tests for only one platform. Useful for debugging:

robot -v config:novacustom-v540tnd -t "E2EH002*" dts/dts-e2e-helper.robot
Details 

==============================================================================
Dts-E2E-Helper
==============================================================================
E2EH002.001 Print names and exports of test cases to be generated ...
.--------------------------------------------------
novacustom-v540tnd
--------------------------------------------------
---------------
novacustom-v540tnd Initial Deployment - Community Version
---------------
export TEST_SYSTEM_MODEL="V5xTNC_TND_TNE"
export TEST_BOARD_MODEL="V540TNx"
export TEST_SYSTEM_VENDOR="Notebook"
export TEST_BIOS_VENDOR="proprietary"
export DTS_TESTING="true"
export TEST_NOVACUSTOM_MODEL="V540TNx"
export TEST_BIOS_VERSION="v0.0.0"
---------------
novacustom-v540tnd UEFI Update - Community Version
---------------
export TEST_SYSTEM_MODEL="V5xTNC_TND_TNE"
export TEST_BOARD_MODEL="V540TNx"
export TEST_SYSTEM_VENDOR="Notebook"
export TEST_BIOS_VENDOR="3mdeb"
export DTS_TESTING="true"
export TEST_NOVACUSTOM_MODEL="V540TNx"
export TEST_BIOS_VERSION="Dasharo (coreboot+UEFI) v0.0.0"
export TEST_USING_OPENSOURCE_EC_FIRM="true"
E2EH002.001 Print names and exports of test cases to be generated ... | PASS |
------------------------------------------------------------------------------
Dts-E2E-Helper                                                        | PASS |
1 test, 1 passed, 0 failed
==============================================================================

@m-iwanicki m-iwanicki requested a review from macpijan July 23, 2025 17:19
@m-iwanicki
dts: Add TemplateSplit and PlatformParser libraries
56dcfa1 
* TemplateSplit - split each template keyword into separate tests
* PlatformParser - parse platform config and return variables defined
  for that platform in a dict

Signed-off-by: Michał Iwanicki <[email protected]>
@m-iwanicki
requirements.txt: update to robotframework 7.3
f4d810b 
Signed-off-by: Michał Iwanicki <[email protected]>
@m-iwanicki
dts: replace tests with template generated ones
20ff403 
Signed-off-by: Michał Iwanicki <[email protected]>
@m-iwanicki
dts-lib: remove unused keyword
c64c56e 
Signed-off-by: Michał Iwanicki <[email protected]>
@m-iwanicki
dts-lib: Add some improvements and new keywords
b5bd51f 
Signed-off-by: Michał Iwanicki <[email protected]>
@m-iwanicki
platform-configs: add config used with templated E2E tests
1710a07 
Signed-off-by: Michał Iwanicki <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@macpijan macpijan Awaiting requested review from macpijan

Assignees

@m-iwanicki m-iwanicki

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@m-iwanicki @macpijan