Compatibility with Magento Cloud

[kbalazy-creatuity]

In the latest version (1.3.2), a cache.xml file has been added which adds cache and is to the problem on Magento Cloud.
When setup:upgrade is executed, the files from generated/staticcache are deleted and generated. This is not executable on Magento Cloud, and we get an error during deployment.

INFO: Running setup upgrade.
INFO: Set flag: var/.deploy_is_failed
ERROR: [126] The command "/bin/bash -c "set -o pipefail; php ./bin/magento setup:upgrade --keep-generated --ansi --no-interaction -vvv | tee -a /app/xxxxxxxx_stg/var/log/install_upgrade.log"" failed. Cache types config flushed successfully
  Warning: unlink(/app/xxxxxxxx_stg/generated/staticcache/global_primary_adminhtml_compiled_plugins.php): Read-only file system in /app/xxxxxxxx_stg/vendor/creatuity/magento2-interceptors/Generator/FileCache.php on line 121
INFO: Set flag: var/.deploy_is_failed
  W: 
  W: In Processor.php line 129:
  W:                                                                                
  W:   The command "/bin/bash -c "set -o pipefail; php ./bin/magento setup:upgrade  
  W:    --keep-generated --ansi --no-interaction -vvv | tee -a /app/xxxxxxxx_  
  W:   stg/var/log/install_upgrade.log"" failed. Cache types config flushed s  
  W:   uccessfully                                                             
  W:   Warning: unlink(/app/xxxxxxxx_stg/generated/staticcache/global  
  W:   _primary_adminhtml_compiled_plugins.php): Read-only file system in /app/xxxxxxxx  
  W:   _stg/vendor/creatuity/magento2-interceptors/Generator/FileCache.p  
  W:   hp on line 121                                                       
  W:                                                                                
  W: 
  W: In InstallUpdate.php line 100:
  W:                                                                                
  W:   The command "/bin/bash -c "set -o pipefail; php ./bin/magento setup:upgrade  
  W:    --keep-generated --ansi --no-interaction -vvv | tee -a /app/xxxxxxxx_  
  W:   stg/var/log/install_upgrade.log"" failed. Cache types config flushed s  
  W:   uccessfully                                                             
  W:   Warning: unlink(/app/xxxxxxxx_stg/generated/staticcache/global  
  W:   _primary_adminhtml_compiled_plugins.php): Read-only file system in /app/xxxxxxxx  
  W:   _stg/vendor/creatuity/magento2-interceptors/Generator/FileCache.p  
  W:   hp on line 121                                                       
  W:                                                                                
  W: 
  W: In Setup.php line 58:
  W:                                                                                
  W:   The command "/bin/bash -c "set -o pipefail; php ./bin/magento setup:upgrade  
  W:    --keep-generated --ansi --no-interaction -vvv | tee -a /app/xxxxxxxx_  
  W:   stg/var/log/install_upgrade.log"" failed. Cache types config flushed s  
  W:   uccessfully                                                             
  W:   Warning: unlink(/app/xxxxxxxx_stg/generated/staticcache/global  
  W:   _primary_adminhtml_compiled_plugins.php): Read-only file system in /app/xxxxxxxx  
  W:   _stg/vendor/creatuity/magento2-interceptors/Generator/FileCache.p  
  W:   hp on line 121                                                       
  W:                                                                                
  W: 
  W: In Shell.php line 86:
  W:                                                                                
  W:   The command "/bin/bash -c "set -o pipefail; php ./bin/magento setup:upgrade  
  W:    --keep-generated --ansi --no-interaction -vvv | tee -a /app/xxxxxxxx_  
  W:   stg/var/log/install_upgrade.log"" failed. Cache types config flushed s  
  W:   uccessfully                                                             
  W:   Warning: unlink(/app/xxxxxxxx_stg/generated/staticcache/global  
  W:   _primary_adminhtml_compiled_plugins.php): Read-only file system in /app/xxxxxxxx  
  W:   _stg/vendor/creatuity/magento2-interceptors/Generator/FileCache.p  
  W:   hp on line 121                                                       
  W:                                                                                
  W: 
  W: deploy
  W: 
  E: Failed to run deploy hook
  E: Command '['sudo', '-u', 'xxxxxxxx_stg', 'bash', '-c', '/etc/platform/xxxxxxxx_stg/post_deploy.sh']' returned non-zero exit status 126
  Exiting maintenance mode

On cloud when running setup:upgrade, the generated directory is only available in read-only and there is no way to change this. The application when deploying performs compilation every time, and with older versions it worked without an issue.

[igorwulff]

Perhaps a soft deletion might be better then and/or it should check if --keep-generated is used?

[igorwulff]

The problem seems to originate from the --keep-generated param. During a setup upgrade the following code is called:

setup/src/Magento/Setup/Model/Installer.php

public function updateModulesSequence($keepGeneratedFiles = false)
    {
        $config = $this->deploymentConfig->get(ConfigOptionsListConstants::KEY_MODULES);
        if (!$config) {
            // phpcs:ignore Magento2.Exceptions.DirectThrow
            throw new Exception(
                "Can't run this operation: deployment configuration is absent."
                . " Run 'magento setup:config:set --help' for options."
            );
        }
        $this->flushCaches([ConfigCache::TYPE_IDENTIFIER]);
        $this->cleanCaches();
        if (!$keepGeneratedFiles) {
            $this->cleanupGeneratedFiles();
        }
        $this->log->log('Updating modules:');
        $this->createModulesConfig([]);
    }

In the current setup the staticcache folder is cleaned in the call for $this->cleanCaches(). What I would expect is that it would get cleaned ONLY in the $this->cleanupGeneratedFiles() call when $keepGeneratedFiles is false.

This cleanupGeneratedFiles is sourced in vendor/magento/framework/App/State/CleanupFiles.php, where the following method amongs others is called:

    public function clearCodeGeneratedClasses()
    {
        return array_merge(
            $this->emptyDir(DirectoryList::GENERATED_CODE),
            $this->emptyDir(DirectoryList::GENERATED_METADATA)
        );
    }

Returning false in the FileCache implementation and clean method and adding a plugin for this class and method as following could solve the issue:

    public function clearCodeGeneratedClasses()
    {
        return array_merge(
            $this->emptyDir(DirectoryList::GENERATED_CODE),
            $this->emptyDir(DirectoryList::GENERATED_METADATA),
            $this->emptyDir('staticcache')
        );
    }

But then an entry in /vendor/magento/framework/App/Filesystem/DirectoryList.php should also be added for the getDefaultConfig call...

In the end it probably would make more sense to make use of the generated/metadata folder that's already there?

[fsw]

Thanks a lot @igorwulff

If it will be easier than working around this issue, I think it will make sense to move this cache into metadata.
I don't have any cloud project ATM but you can check if this will solve it:

#13