DEV Community

anilkulkarni87
anilkulkarni87

Posted on

Airflow Api tests

airflow-api-tests

This is a collection of Pytest for the 2.0 Stable Rest Apis for Apache Airflow. I have another repo where you could setup airflow locally and play around with these. I am used to RestAssured, but trying out pytest here.

Apache Airflow 2.0 Stable Rest Api calls - Python

I am used to RestAssured for api testing. Trying out python api testing with Airflow stable rest api

📝 Table of Contents

🧐 About

This repository will contain api calls to stable Airflow Rest apis. Having worked in airflow, I felt the lack of it. Come Airflow 2.0, it resolved the issues for me.
It is also an attempt to understand api testing with python as I usually prefer Rest Assured (Java). This is a WIP repo as I continue to build and add calls to all apis.
I have another repo which can help people setup airflow locally and then play around with these api.

Uses

Listing out some reasons why I created this for myself:

  • Learn about airflow apis.
  • Learn about Python api testing.
  • Understand use cases of api and document for future needs.

🏁 Getting Started

Sequence of steps to be followed to be able to use this successfully:

  • Clone the airflow-docker repo.
  • Follow the instructions there to start running airflow locally.
  • Execute some DAGS that are part of my repo or create your own DAGS.
  • Start playing with the apis.

🎈 Usage

🔧 Running the tests

Here is an example of how you could run the tests. I will continue to evolve this

pytest test_dag.py
Enter fullscreen mode Exit fullscreen mode

Github Workflow for running tests

I added this step for me to understand more about github workflows and how i can leverage it for this specific usecase. Essentially what I will have to do is within the workflow:

  • Clone airflow-docker repo
  • Start Airflow
  • Run these tests
  • Do something with the results

⛏️Airflow APIs

I know you could get your hands on the swagger doc for these. But I still wanted to list down here.

  • Get Config
    • This is usually forbidden from the administrator owing to security reasons.
  • Connection
    • Get connections : To list all connections currently available
    • Post : To create a new connection Id
    • Patch : To patch means to update an existing connection Id
    • Delete : To delete a connection
    • Get Connection Id : Get details of a specific connection based on the id
  • DAG
    • Get Dag Source code : We can get the dag source code by passing a file token, We can get file_token by calling the get dag by dag id
    • Get DAGS : Able to get all DAGS
    • Get DAG Info : Get basic info about a DAG
    • Patch : This is to update the dag. You can refer the example I have.
    • Post - clear tasks instances of a DAG : #TODO
    • Get Dag details : Get Basic info about dags
    • Get Dag tasks : Get all tasks for the dag.
    • Get Task details : Get a representation of the task
    • Post UpdateTaskInstanceState : #TODO

✍️ Authors

Top comments (0)