Skip to content

[minor] mas rollback #1610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 26 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
0a361e7
[minor] add rollback support into cli
anilprajapatiibm May 21, 2025
537fffb
[patch] add python-devops tar
anilprajapatiibm May 21, 2025
f0039e5
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm May 22, 2025
199ec65
[patch] add local python-devops
anilprajapatiibm May 22, 2025
776a447
[patch] add rollback command at top level
anilprajapatiibm May 23, 2025
d1e7f11
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm May 23, 2025
f2aed9d
[patch] add rollback option and message
anilprajapatiibm May 23, 2025
ccd37f9
Merge branch 'MASCORE-7021-mas-rollback' of https://github.com/ibm-ma…
anilprajapatiibm May 23, 2025
7852ef1
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm May 23, 2025
ee4bb8e
[patch] fix action to store_true
anilprajapatiibm May 23, 2025
b39145c
[patch] add py-devops tar
anilprajapatiibm May 23, 2025
fde38fc
[patch] change variable names
anilprajapatiibm May 23, 2025
f57873c
[patch] remove codition wait_for_install
anilprajapatiibm May 23, 2025
fa38c89
[patch] remove wait_for_install condition
anilprajapatiibm May 23, 2025
9d3be23
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm May 23, 2025
4d168a9
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm May 26, 2025
40bd394
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm May 27, 2025
fe50550
[patch] add py devops
anilprajapatiibm May 27, 2025
1a4951d
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm Jun 12, 2025
0de7030
[patch] local tar py-devops
anilprajapatiibm Jun 12, 2025
56d9c44
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm Jul 8, 2025
584ebc1
[patch] tmp disable mongo update
anilprajapatiibm Jul 8, 2025
756b5fe
[patch] add python-devops
anilprajapatiibm Jul 8, 2025
6b2fea1
[patch] add python tar
anilprajapatiibm Jul 8, 2025
2c15964
Merge branch 'master' into MASCORE-7021-mas-rollback
anilprajapatiibm Jul 14, 2025
0ed0876
Revert "[patch] tmp disable mongo update"
anilprajapatiibm Jul 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion image/cli/app-root/src/.bashrc
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ fi

echo " - ${TEXT_BOLD}${COLOR_GREEN}mas must-gather${TEXT_RESET} to perform must-gather against the target cluster"
echo " - ${TEXT_BOLD}${COLOR_GREEN}mas uninstall${TEXT_RESET} to uninstall a MAS instance"

echo " - ${TEXT_BOLD}${COLOR_GREEN}mas rollback${TEXT_RESET} to rollback a MAS instance"
# None of these functions are tested/supported on s390x /ppc64le yet
if [ $arch != "s390x" ] && [ $arch != "ppc64le" ]; then
echo " - ${TEXT_BOLD}${COLOR_GREEN}mas configtool-oidc${TEXT_RESET} to configure OIDC integration"
Expand Down
Binary file added image/cli/install/mas_devops.tar.gz
Binary file not shown.
11 changes: 11 additions & 0 deletions image/cli/mascli/mas
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,17 @@ case $1 in
mas-cli uninstall "$@"
;;

rollback)
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" >> $LOGFILE
echo "!! rollback !!" >> $LOGFILE
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" >> $LOGFILE
# Take the first parameter off (it will be "rollback")
shift
# Run the new Python-based upgrade
mas-cli rollback "$@"
;;


mustgather|must-gather)
echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" >> $LOGFILE
echo "!! must-gather !!" >> $LOGFILE
Expand Down
5 changes: 5 additions & 0 deletions python/src/mas-cli
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ from mas.cli.aiservice.install.app import AiServiceInstallApp
from mas.cli.update.app import UpdateApp
from mas.cli.upgrade.app import UpgradeApp
from mas.cli.uninstall.app import UninstallApp
from mas.cli.rollback.app import RollbackApp

from prompt_toolkit import HTML, print_formatted_text
from urllib3.exceptions import MaxRetryError
Expand All @@ -43,6 +44,7 @@ def usage():
+ " - <ForestGreen>mas-cli update</ForestGreen> Apply updates and security fixes\n" # noqa: W503
+ " - <ForestGreen>mas-cli upgrade</ForestGreen> Upgrade to a new MAS release\n" # noqa: W503
+ " - <ForestGreen>mas-cli uninstall</ForestGreen> Remove MAS from the cluster\n" # noqa: W503
+ " - <ForestGreen>mas-cli rollback</ForestGreen> Rollback MAS from the cluster\n" # noqa: W503

))
print_formatted_text(HTML("For usage information run <ForestGreen>mas-cli [action] --help</ForestGreen>\n"))
Expand All @@ -67,6 +69,9 @@ if __name__ == '__main__':
elif function == "upgrade":
app = UpgradeApp()
app.upgrade(argv[2:])
elif function == "rollback":
app = RollbackApp()
app.rollback(argv[2:])
elif function in ["-h", "--help"]:
usage()
exit(0)
Expand Down
11 changes: 11 additions & 0 deletions python/src/mas/cli/rollback/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# *****************************************************************************
# Copyright (c) 2024 IBM Corporation and other Contributors.
#
# All rights reserved. This program and the accompanying materials
# are made available under the terms of the Eclipse Public License v1.0
# which accompanies this distribution, and is available at
# http://www.eclipse.org/legal/epl-v10.html
#
# *****************************************************************************

from ..cli import BaseApp # noqa: F401
177 changes: 177 additions & 0 deletions python/src/mas/cli/rollback/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
#!/usr/bin/env python
# *****************************************************************************
# Copyright (c) 2024 IBM Corporation and other Contributors.
#
# All rights reserved. This program and the accompanying materials
# are made available under the terms of the Eclipse Public License v1.0
# which accompanies this distribution, and is available at
# http://www.eclipse.org/legal/epl-v10.html
#
# *****************************************************************************

import logging
import logging.handlers
from halo import Halo
from prompt_toolkit import print_formatted_text, HTML

from openshift.dynamic.exceptions import ResourceNotFoundError

from ..cli import BaseApp
from ..validators import StorageClassValidator
from .argParser import rollbackArgParser

from mas.devops.ocp import createNamespace, getConsoleURL
from mas.devops.mas import listMasInstances, getCurrentCatalog
from mas.devops.tekton import preparePipelinesNamespace, installOpenShiftPipelines, updateTektonDefinitions, launchRollbackPipeline


logger = logging.getLogger(__name__)


class RollbackApp(BaseApp):

def rollback(self, argv):
"""
Rollback MAS instance
"""
self.args = rollbackArgParser.parse_args(args=argv)
self.noConfirm = self.args.no_confirm
self.devMode = self.args.dev_mode

if self.args.mas_catalog_version:
# Non-interactive mode
logger.debug("Maximo Operator Catalog version is set, so we assume already connected to the desired OCP")
requiredParams = ["mas_catalog_version", "mas_instance_id", "mas_core_version", "mas_app_manage_version", "mas_app_iot_version"]
optionalParams = [
"skip_pre_check",
"dev_mode",
# Dev Mode
"artifactory_username",
"artifactory_token"

]
for key, value in vars(self.args).items():
# These fields we just pass straight through to the parameters and fail if they are not set
if key in requiredParams:
if value is None:
self.fatalError(f"{key} must be set")
self.setParam(key, value)

# These fields we just pass straight through to the parameters
elif key in optionalParams:
if value is not None:
self.setParam(key, value)

# Arguments that we don't need to do anything with
elif key in ["no_confirm", "help"]:
pass

# Fail if there's any arguments we don't know how to handle
else:
print(f"Unknown option: {key} {value}")
self.fatalError(f"Unknown option: {key} {value}")
else:
# Interactive mode
self.printH1("Set Target OpenShift Cluster")
# Connect to the target cluster
self.connect()

if self.dynamicClient is None:
self.fatalError("The Kubernetes dynamic Client is not available. See log file for details")

# Perform a check whether the cluster is set up for airgap install, this will trigger an early failure if the cluster is using the now
# deprecated MaximoApplicationSuite ImageContentSourcePolicy instead of the new ImageDigestMirrorSet
self.isAirgap()
self.reviewCurrentCatalog()
self.reviewMASInstance()

if self.args.mas_catalog_version is None:
# Interactive mode
self.chooseCatalog()

# Validations
if not self.devMode:
self.validateCatalog()

print()

self.printH1("Review Settings")
self.printDescription([
"Connected to:",
f" - <u>{getConsoleURL(self.dynamicClient)}</u>"
])

self.printH2("IBM Maximo Operator Catalog")
self.printSummary("Installed Catalog", self.installedCatalogId)
self.printSummary("Rollback Catalog", self.getParam("mas_catalog_version"))
self.printSummary("Current Instance ID", self.getParam("mas_instance_id"))
self.printSummary("Rollback Channel for Core Platform", self.getParam("mas_core_version"))

self.printSummary("Rollback Channel for Maximo Manage", self.getParam("mas_app_manage_version"))

self.printSummary("Rollback Channel for Maximo IoT", self.getParam("mas_app_iot_version"))

if not self.noConfirm:
print()
self.printDescription([
"Please carefully review your choices above, correcting mistakes now is much easier than after the update has begun"
])
continueWithUpdate = self.yesOrNo("Proceed with these settings")
# Prepare the namespace and launch the installation pipeline
if self.noConfirm or continueWithUpdate:
self.createTektonFileWithDigest()

self.printH1("Launch Rollback")
pipelinesNamespace = f"mas-{self.getParam('mas_instance_id')}-pipelines"

with Halo(text='Validating OpenShift Pipelines installation', spinner=self.spinner) as h:
installOpenShiftPipelines(self.dynamicClient)
h.stop_and_persist(symbol=self.successIcon, text="OpenShift Pipelines Operator is installed and ready to use")

with Halo(text=f'Preparing namespace ({pipelinesNamespace})', spinner=self.spinner) as h:
createNamespace(self.dynamicClient, pipelinesNamespace)
preparePipelinesNamespace(dynClient=self.dynamicClient)
h.stop_and_persist(symbol=self.successIcon, text=f"Namespace is ready ({pipelinesNamespace})")

with Halo(text=f'Installing latest Tekton definitions (v{self.version})', spinner=self.spinner) as h:
updateTektonDefinitions(pipelinesNamespace, self.tektonDefsPath)
h.stop_and_persist(symbol=self.successIcon, text=f"Latest Tekton definitions are installed (v{self.version})")

with Halo(text="Submitting PipelineRun for MAS Rollback", spinner=self.spinner) as h:
pipelineURL = launchRollbackPipeline(dynClient=self.dynamicClient, params=self.params)
if pipelineURL is not None:
h.stop_and_persist(symbol=self.successIcon, text="PipelineRun for MAS rollback submitted")
print_formatted_text(HTML(f"\nView progress:\n <Cyan><u>{pipelineURL}</u></Cyan>\n"))
else:
h.stop_and_persist(symbol=self.failureIcon, text="Failed to submit PipelineRun for MAS rollback, see log file for details")
print()

def reviewCurrentCatalog(self) -> None:
catalogInfo = getCurrentCatalog(self.dynamicClient)
self.installedCatalogId = None
if catalogInfo is None:
self.fatalError("Unable to locate existing install of the IBM Maximo Operator Catalog")
elif catalogInfo["catalogId"] is None:
self.printWarning("Unable to determine identity & version of currently installed ibm-maximo-operator-catalog")
else:
self.installedCatalogId = catalogInfo["catalogId"]
self.printH1("Review Installed Catalog")
self.printDescription([
f"The currently installed Maximo Operator Catalog is <u>{catalogInfo['displayName']}</u>",
f" <u>{catalogInfo['image']}</u>"
])

def reviewMASInstance(self) -> None:
self.printH1("Review MAS Instances")
self.printDescription(["The following MAS intances are installed on the target cluster and will be affected by the catalog rollback:"])
try:
suites = listMasInstances(self.dynamicClient)
for suite in suites:
self.printDescription([f"- <u>{suite['metadata']['name']}</u> v{suite['status']['versions']['reconciled']}"])
except ResourceNotFoundError:
self.fatalError("No MAS instances were detected on the cluster (Suite.core.mas.ibm.com/v1 API is not available). See log file for details")

def validateCatalog(self) -> None:
if self.installedCatalogId is not None and self.installedCatalogId < self.getParam("mas_catalog_version"):
self.fatalError(f"Selected catalog is newer than the currently installed catalog. Unable to rollback catalog from {self.installedCatalogId} to {self.getParam('mas_catalog_version')}")

111 changes: 111 additions & 0 deletions python/src/mas/cli/rollback/argParser.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# *****************************************************************************
# Copyright (c) 2024 IBM Corporation and other Contributors.
#
# All rights reserved. This program and the accompanying materials
# are made available under the terms of the Eclipse Public License v1.0
# which accompanies this distribution, and is available at
# http://www.eclipse.org/legal/epl-v10.html
#
# *****************************************************************************

import argparse

from .. import __version__ as packageVersion
from ..cli import getHelpFormatter

rollbackArgParser = argparse.ArgumentParser(
prog='mas rollback',
description="\n".join([
f"IBM Maximo Application Suite Admin CLI v{packageVersion}",
"Rollback the IBM Maximo Operator Catalog, and related MAS dependencies by configuring and launching the MAS Rollback Tekton Pipeline.\n",
"Interactive Mode:",
"Omitting the --catalog option will trigger an interactive prompt"
]),
epilog="Refer to the online documentation for more information: https://ibm-mas.github.io/cli/",
formatter_class=getHelpFormatter(),
add_help=False
)

masArgGroup = rollbackArgParser.add_argument_group('MAS Basic Configuration')
masArgGroup.add_argument(
"-i", "--mas-instance-id",
dest='mas_instance_id',
required=False,
help="MAS Instance ID"
)

masArgGroup = rollbackArgParser.add_argument_group('Catalog Selection')
masArgGroup.add_argument(
'-c', '--catalog',
dest='mas_catalog_version',
required=False,
help="Maximo Operator Catalog Version (e.g. v9-240625-amd64)"
)

masArgGroup.add_argument(
"--mas-version",
dest='mas_core_version',
required=False,
help="Subscription channel for the Core Platform"
)


masArgGroup.add_argument(
"--manage-version",
dest='mas_app_manage_version',
required=False,
help="Subscription channel for Maximo Manage"
)

masArgGroup.add_argument(
"--iot-version",
required=False,
dest='mas_app_iot_version',
help="Subscription channel for Maximo IoT"
)


# Development Mode
# -----------------------------------------------------------------------------
devArgGroup = rollbackArgParser.add_argument_group("Development Mode")
devArgGroup.add_argument(
"--artifactory-username",
required=False,
help="Username for access to development builds on Artifactory"
)
devArgGroup.add_argument(
"--artifactory-token",
required=False,
help="API Token for access to development builds on Artifactory"
)

# More Options
# -----------------------------------------------------------------------------
otherArgGroup = rollbackArgParser.add_argument_group('More')
otherArgGroup.add_argument(
"--dev-mode",
required=False,
action="store_true",
default=False,
help="Configure installation for development mode",
)
otherArgGroup.add_argument(
'--no-confirm',
required=False,
action='store_true',
default=False,
help="Launch the upgrade without prompting for confirmation",
)
otherArgGroup.add_argument(
'--skip-pre-check',
required=False,
action='store_true',
default=False,
help="Skips the 'pre-update-check' and 'post-update-verify' tasks in the update pipeline",
)
otherArgGroup.add_argument(
'-h', "--help",
action='help',
default=False,
help="Show this help message and exit",
)
Loading
Loading