pre requisites: python 3.7 or greater
Execute the below ones one after another
pip install apache-airflow==1.10.6
pip install 'apache-airflow[celery]'
pip install 'apache-airflow[mysql]'
brew install rabbitmq
start the rabbitmq-server in the background
rabbitmq-server -detached
make the mysql config changes
sudo chown -R _mysql:mysql /usr/local/var/mysql
start the mysql server
sudo mysql.server start
mysql setup
mysql -uroot
set the password for the root user with the below
mysql>ALTER USER 'root'@'localhost' IDENTIFIED BY 'new password';
create the DB, the user and grant the required privileges
mysql>CREATE DATABASE airflow CHARACTER SET utf8 COLLATE utf8_unicode_ci;
mysql>create user ‘airflow’@’localhost’ identified by ‘airflow’;
mysql>grant all privileges on * . * to 'airflow'@'localhost';
mysql>flush privileges;
mysql>quit
airflow initialization
airflow initdb
airflow config file update
Update the airflow.cfg file (should be available in ~/airflow/ directory.
sql connection as:
sql_alchemy_conn = mysql://root:airflow@localhost/airflow
there should be a sql alchemy connection string and you can comment it and add the above
executor as:
CeleryExecutor
Save the file and exit.
run the below on their separate windows
airflow webserver
airflow scheduler
airflow worker
airflow webserver window will show you the url for airflow UI.
note: airflow initialization (airflow initdb) should load the example dags. You can turn them off in your airflow.cfg file, if you don't want to. command: load_examples = False
Top comments (0)