Airflow: Log file isn't local, Unsupported remote log location

I am not able see the logs attached to the tasks from the Airflow UI: 在这里输入图像描述

Log related settings in airflow.cfg file are:

  • remote_base_log_folder =
  • base_log_folder = /home/my_projects/ksaprice_project/airflow/logs
  • worker_log_server_port = 8793
  • child_process_log_directory = /home/my_projects/ksaprice_project/airflow/logs/scheduler
  • Although I am setting remote_base_log_folter it is trying to fetch the log from http://:8793/log/tutorial/print_date/2017-08-02T00:00:00 - I don't understand this behavior. According to the settings the workers should store the logs at /home/my_projects/ksaprice_project/airflow/logs and they should be fetched from the same location instead of remote.

    Update task_instance table content: 在这里输入图像描述


    I also faced the same problem.

    Setting below variables in airflow.cfg worked for me. Use {hostname} as machine's FQDN {hostname} instead of localhost.

    endpoint_url = http://{hostname}:8080
    
    base_url = http://{hostname}:8080
    

    Best of luck!


    I ran into this as well, and had to unpause the tasks.

    dags_are_paused_at_creation = False
    

    I also set new dags to default to unpaused in my airflow.cfg

    dags_are_paused_at_creation = False
    

    As you can see in the image-1 there is a timestamp , make sure in your logs you have the folder/file with that timestamp as name ..

    You are looking at UI, so first make sure you have log files created in the directory, in my case my log folder looks like

    (AIRFLOW-ENV) [cloudera@quickstart dags]$ ll /home/cloudera/workspace/python/airflow_home/logs/my_test_dag/my_sensor_task 
    total 8
    -rw-rw-rw- 1 cloudera cloudera 3215 Nov 14 08:45 2017-11-12T12:00:00
    -rw-rw-rw- 1 cloudera cloudera 2694 Nov 14 08:45 2017-11-14T08:36:06.920727
    (AIRFLOW-ENV) [cloudera@quickstart dags]$ 
    

    So my log URL is

    http://localhost:8080/admin/airflow/log?task_id=my_sensor_task&dag_id=my_test_dag&execution_date=2017-11-14T08:36:06.920727
    

    When you go to your DAG, and select the GRAPH-VIEW, you can see a dropdown next to "RUN", select the appropriate run, and then in the graph-view below , select the appropriate task/operator and select view-log

    链接地址: http://www.djcxy.com/p/40142.html

    上一篇: 如何更改CUDA版本

    下一篇: 气流:日志文件不是本地的,不支持的远程日志位置