[6.0] Variables from non natural environment

[Fedik]

Pull Request for Issue #36898 .

Alternative to PR #45070 .

Summary of Changes

This is complete (I hope) implementation of environment variables in Joomla.

Key points:

• By default the feature is disabled. Create .env in root folder to enable it (can be just an empty file).
• I tried to keep the variable names in sync with Joomla Docker. If you find something is missing or a typo, please let me know.
• The installation can be completed with environment variables (except language part). This work with CLI and WEB installer. User can provide all options via environment variables (including admin user info) or a few (like DB only, then installer will ask for missing options).
• The configuration options provided by environment variables cannot be edited in backend, also via CLI.

Testing Instructions

Test installation, test with Web and in CLI installer

1. Create .env with DB options:

JOOMLA_DB_TYPE="mysql"
JOOMLA_DB_HOST="Your DB host"
JOOMLA_DB_USER="Your DB user"
JOOMLA_DB_PASSWORD="Your DB password"
JOOMLA_DB_NAME="Your db name"
JOOMLA_DB_PREFIX="DB prefix"

And run installer. You should be asked for Site name, and User information.
Then installation should be completed, as usual.

2. Create .env with all installation parameters and user information:

JOOMLA_DB_TYPE="mysql"
JOOMLA_DB_HOST="Your DB host"
JOOMLA_DB_USER="Your DB user"
JOOMLA_DB_PASSWORD="Your DB password"
JOOMLA_DB_NAME="Your db name"
JOOMLA_DB_PREFIX="DB prefix"

JOOMLA_SITE_NAME="Test installation"
JOOMLA_PUBLIC_FOLDER=""

JOOMLA_ADMIN_USER="Your admin user"
JOOMLA_ADMIN_USERNAME="Your admin username"
JOOMLA_ADMIN_PASSWORD="Your admin user password"
JOOMLA_ADMIN_EMAIL="Your admin user email"

And run installer. You will not be asked for Site name, and User information.
The installation should be completed, as usual.

Test the existing site

Create .env with DB options, and copy options from configuration.php.
(skip this step if the site was installed with use of .env)

JOOMLA_DB_TYPE="mysql"
JOOMLA_DB_HOST="Your DB host"
JOOMLA_DB_USER="Your DB user"
JOOMLA_DB_PASSWORD="Your DB password"
JOOMLA_DB_NAME="Your db name"
JOOMLA_DB_PREFIX="DB prefix"

Then visit the site, all should work as before.

Link to documentations

Please select:

• Documentation link for docs.joomla.org:
• No documentation changes for docs.joomla.org needed
• Pull Request link for manual.joomla.org: TBD
• No documentation changes for manual.joomla.org needed

@Llewellynvdm please have a look if it will be good with Joomla Docker, thanks!

[richard67]

Correct, but that is incomplete implementation.
This PR is alternative to that.

Would be good to comment on that pr etc

I've allowed myself to add a reference to the issue and a hint to the other PR at the top of the description.

[voronkovich]

Some suggestions:

1. Use the $_SERVER instead of $_ENV

   Both can be disabled in php.ini, but the $_ENV is disabled by default: https://github.com/php/php-src/blob/201c691fab036b40f8b2ddcfd253fd21089ed799/php.ini-production#L652

   variables_order = "GPCS"

   That's why I use getenv() as a fallback:

   // getenv() is not thread-safe and it can cause segmentaion fault, so we should try $_SERVER first
   $envs = !empty($_SERVER) ? $_SERVER : getenv();

2. Use the symfony/dotenv

   it's more advanced than the vlucas/phpdotenv and supports some good features like creating environments (dev, prod, test, staging and etc.) and dumping envs in production (.env.local.php).

   Also Joomla already uses some Symfony's components, so why not to use another one?

[Fedik]

Both can be disabled in php.ini, but the $_ENV is disabled by default

I tested on PHP 8.1 and 8.4, $_ENV is always present but unpopulated, with variables_order = "GPCS" and variables_order = "EGPCS".

The idea, that the feature is disable by default.
And when User add .env then the Dotenv library will populate $_ENV, and we can use it.

Use the symfony/dotenv

To me Dotenv also looks good, and well supported library. And I would prefer something light.
But if people will insist it could be changed to anything else, easily at any point of time, because it used only in bootstrap.

[brianteeman]

$_ENV is disabled by default on most php installations. When disabled it will return an empty value

[voronkovich]

@Fedik

The idea, that the feature is disable by default.
And when User add .env then the Dotenv library will populate $_ENV, and we can use it.

Then the real environment variables won't work in cases where the $_ENV is disabled.

Sometimes it's very useful to run program with a changed environment. For example:

JOOMLA_PROXY_ENABLED=false php cli/joomla.php core:update

[voronkovich]

But if people will insist it could be changed to anything else, easily at any point of time, because it used only in bootstrap.

It would be a BC break, because the symfony/dotenv works slightly different. It merges .env files, but the vlucas/phpdotenv doesn't change already loaded values.

Example

# .env
HELLO='env'

# .env.dev
HELLO='env.dev'

vlucas/phpdotenv:

<?php

require 'vendor/autoload.php';

Dotenv\Dotenv::createImmutable(__DIR__, ['.env', '.env.dev'])->safeLoad();

// Outputs: 'env'
echo $_ENV['HELLO'];

symfony/dotenv:

<?php

require 'vendor/autoload.php';

(new Symfony\Component\Dotenv\Dotenv())->bootEnv(__DIR__ . '/.env', 'dev');

// Outputs: 'env.dev'
echo $_ENV['HELLO'];

[Fedik]

I added code to check for empty ENV.

It would be a BC break, because the symfony/dotenv works slightly different. It merges .env files, but the vlucas/phpdotenv doesn't change already loaded values.

I switched shortCircuit to false, should be the same now

[voronkovich]

I switched shortCircuit to false, should be the same now

Nothing is changed. It still works like in the example above. I've created a repository with an example: https://github.com/voronkovich/dotenv-example.

You can't make the vlucas/phpdotenv works the same way as the symfony/dotenv (believe me, I've already tried).

I suggest to simplify this PR and load only the .env file. Later, in other PRs, we can add support for either symfony/dotenv or env.dev.

[voronkovich]

Let me give you my use case why I want to use real environment variables.

I have a VPS with several dozen Joomla installations, and for some reason, I need to use a proxy to load updates. Currently, I have to manually edit every configuration.php file and set the proxy settings. With environment variables, I could set all these settings in one place: /etc/environment (or maybe in /etc/environment.d/00-joomla.conf):

JOOMLA_PROXY_ENABLE=true
JOOMLA_PROXY_HOST='proxy_host'
JOOMLA_PROXY_PORT='proxy_port'
JOOMLA_PROXY_USER='proxy_user'
JOOMLA_PROXY_PASS='proxy_pass'

Also it would be great to set common settings for SMTP, caching, redis, disable errors_reporting an so on using envs. All in one place.

[Fedik]

Can you move this code block upward outside the if statement? Without it real environment variables won't work if .env file is not present and $_ENV is disabled.

This is intentional. To enable envs on the site User should create an .env file (at least empty), or enable in php.ini (variables_order parameter).
If you have VPS, then last option should not be a problem for you.

For most Users it does not need, so we do not need all that (for now) to be always enabled.
However it can be discussed.

[voronkovich]

This is intentional. To enable envs on the site User should create an .env file (at least empty), or enable in php.ini (variables_order parameter).

I've never seen an application that requires enabling an option to make environment variables work. Because it doesn't make sense.

In Symfony everything works out of the box. In Laravel everything works out of the box. In WordPress everything works out of the box. Even curl doesn't require you to do anything to make envs work. :)

Joomla will be the first one.

[voronkovich]

@Fedik, what's the difference between .env and .env.dev? In what cases should the latter be used? Why is the .env.dev not added to .gitignore file?

[Fedik]

what's the difference between .env and .env.dev?

First one is for production, second one for development. Or whatever User decide.
It is kind of look up list for which files to look.

In the example:

# .env
JOOMLA_DB_NAME=potato

#.env.dev
JOOMLA_DB_NAME=potato_dev

Will be used potato_dev

Will add to gitignore, but not very important.

[voronkovich]

@Fedik, Your example won't work, because you use Dotenv::createImmutable(). The suffix "immutable" means that the existing environment variables are never changed. You should rearrange the files like this:

Dotenv\Dotenv::createImmutable(JPATH_ROOT, ['.env.dev', '.env'], false)->safeLoad();

Alternatively you can use Dotenv::createMutable(), but it rewrites real envs which is very bad idea.

[voronkovich]

@Fedik,

second one for development. Or whatever User decide.

The .dev has a precise meaning: "development". If you want to use it for overriding the .env then I would suggest you to rename it to .env.local (local overrides).

Some examples: Symfony, NextJS

[Fedik]

It is good as it is.
Can be updated any time later.