The new 'Airflow Pipeline Editor' feature is not working properly. DAGs created with this function will be recognized as errors by 'Airflow'. #2124
Description
Describe the issue
Airflow Pipeline Generator generates malformed DAG files.
To Reproduce
Steps to reproduce the behavior:
- Go to 'Airflow pipeline Editor on Launcher'
- Drag 'BashOperator on template'
- Fill bash_command on Node properties 'echo hello-world'
- Click Run pipeline on Apache Airflow
- Fill 'Pipeline Name' to 'hello-world'
Screenshots or log output
If applicable, add screenshots or log output to help explain your problem.
Broken DAG: [/opt/airflow/dags/repo/hello-world-0909015623.py] Traceback (most recent call last): File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/decorators.py", line 94, in wrapper result = func(*args, **kwargs) File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/baseoperator.py", line 414, in __init__ "arguments were:\n**kwargs: {k}".format(c=self.__class__.__name__, k=kwargs, t=task_id), airflow.exceptions.AirflowException: Invalid arguments were passed to BashOperator (task_id: BashOperator). Invalid arguments were: **kwargs: {'namespace': 'airflow', 'xcom_push': False, 'inputs': [], 'outputs': [], 'secrets': [Secret(env, AWS_ACCESS_KEY_ID, elyra, AWS_ACCESS_KEY_ID), Secret(env, AWS_SECRET_ACCESS_KEY, elyra, AWS_SECRET_ACCESS_KEY)], 'in_cluster': True, 'config_file': 'None'}
Expected behavior
The hello-world DAG performs the 'echo hello' behavior.
Deployment information
Describe what you've deployed and how:
- Elyra version: 3.0.1
- Operating system: windows 10
- Installation source: Pip install
- Deployment type: EKS(Kubernetes v 1.21)
Pipeline runtime environment
If the issue is related to pipeline execution, identify the environment where the pipeline is executed
- Apache Airflow Version 2.0.2
The same goes for EmailOperators.
I checked because the generated dags passing arguments are based on k8soperator.