generated from amazon-archives/__template_Apache-2.0
-
Notifications
You must be signed in to change notification settings - Fork 36
Open
Description
When using apache-airflow-providers-amazon version 9.5.0 or higher with MWAA 2.10.3, task logs in the Airflow UI display "[Invalid date]" instead of proper datetime stamps. The same logs appear correctly in CloudWatch logs.
Environment
- MWAA Version: 2.10.3
- apache-airflow-providers-amazon: 9.9.0
- Python Version: 3.11
Sample Task Log output showing issue
+++++
[Invalid date] {[taskinstance.py:2866](http://taskinstance.py:2866/)} INFO - Starting attempt 1 of 2
[Invalid date] {[taskinstance.py:2889](http://taskinstance.py:2889/)} INFO - Executing <Task(BashOperator): templated> on 2025-08-27 00:00:00+00:00
[Invalid date] {standard_task_runner.py:72} INFO - Started process 198 to run task
[Invalid date] {standard_task_runner.py:104} INFO - Running: ['airflow', 'tasks', 'run', 'tutorial', 'templated', 'scheduled__2025-08-27T00:00:00+00:00', '--job-id', '9', '--raw', '--subdir', 'DAGS_FOLDER/airflow_test.py', '--cfg-path', '/tmp/tmp3k9lojtd']
[Invalid date] {standard_task_runner.py:105} INFO - Job 9: Subtask templated
+++++
Expected Behavior
Task logs should display proper datetime stamps in the Airflow UI, matching what is shown in CloudWatch logs.
Reproduction Steps
- Create MWAA 2.10.3 environment
- Install apache-airflow-providers-amazon>=9.5.0
- Execute any DAG
- View task logs in Airflow UI
Related Changes and Context
- Task execution functionality is not affected
- Issue begins with amazon-provider version 9.5.0 and continues through current version (9.9.0)
- In the Airflow Repo, the function
_event_to_stris responsible for converting the events into strings, including formatting of the timestamps of task logs to be displayed in the airflow ui.- Link to Provider 9.4.0 implementation
- Link to Provider 9.5.0 implementation
- In 9.5.0, the implementation switches over to use struct logging apache/airflow@5ca62b8 apache/airflow@c1088b6
- MWAA overrides that function here in 2.10.3
- This issue does not occur on base Airflow deployment.
- MWAA 3.0.6 switches over to StructLogging and follows the convention of airflow-provider version 9.5.0.
Workaround
Currently, users can work around this issue by:
- Using apache-airflow-providers-amazon==9.4.0
- Or accepting the "[Invalid date]" display while continuing to use newer provider versions
Additional Details
- Amazon provider 9.4 needs
boto3>=1.34.90which meets the version in the default constraints file
Amazon provider 9.5 needsboto3>=1.37.0. MWAA 2.10.3 ships withboto3==1.35.36. There may be changes betweenboto3==1.35.36andboto3==1.37.2that is causing this issue. - Boto3 relevant changelog
Metadata
Metadata
Assignees
Labels
No labels