Replies: 3 comments
-
Executor CeleryExecutor(parallelism=500) reported that the task instance <TaskInstance: align_GROUP_525730891524354049.align_152 scheduled__2025-08-29T10:51:40+00:00 [queued]> finished with state failed, but the task instance's state attribute is queued. Learn more: https://airflow.apache.org/docs/apache-airflow/stable/troubleshooting.html#task-state-changed-externally |
Beta Was this translation helpful? Give feedback.
-
But we don't do a task operation on the state, which is reported when airflow is about to execute, For our production problems, if there are workers who are free to help confirm the logic of this piece, see what situations will arise |
Beta Was this translation helpful? Give feedback.
-
Until we have a reproducible case with that, we can't do much about it. converting into a discussion. Ideally - move to latest Airflow (2.11.0 if you do not want Airflow 3, or - better - Airflow 3) and see if the issue appears there, and try to provide detailed logs and some observations of what happened around the time when it happened. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Apache Airflow version
Other Airflow 2 version (please specify below)
If "Other Airflow 2 version" selected, which one?
2.10.4
What happened?
We deployed two schedulers and three workers in a cluster environment, and the task was interrupted in one execution, and the scheduler log showed that the task had failed, but the task was still queued in the metadatabase
What you think should happen instead?
I think it is airflow that produces scenes with inconsistent state in some scenes with high concurrency ...
How to reproduce
This problem is not easy to reproduce, we happened occasionally during production runs, but it was fine when the re-cleark allowed it
Operating System
Ubuntu 20.04.6 LTS
Versions of Apache Airflow Providers
2.10.4
Deployment
Other
Deployment details
using Helm and K8s for installation
Helm:v3.16.2+g13654a5
Kubernates: Client Version: v1.28.14 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.28.14
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions