History

Pratik raj b113da5d19 chore: Use --no-cache-dir flag to pip in Dockerfiles, to save space Using "--no-cache-dir" flag in pip install ,make sure dowloaded packages by pip don't cached on system . This is a best practise which make sure to fetch ftom repo instead of using local cached one . Further , in case of Docker Containers , by restricing caching , we can reduce image size. In term of stats , it depends upon the number of python packages multiplied by their respective size . e.g for heavy packages with a lot of dependencies it reduce a lot by don't caching pip packages. Further , more detail information can be found at https://medium.com/sciforce/strategies-of-docker-images-optimization-2ca9cc5719b6 Signed-off-by: Pratik Raj <rajpratik71@gmail.com>		2021-07-02 01:02:49 +05:30
..
data	update airflow	2020-08-17 15:45:42 +08:00
deprecated	add airflow	2019-09-28 18:58:18 +08:00
systemd	add systemd files for airflow+superset	2019-10-17 23:59:58 +08:00
Dockerfile	chore: Use --no-cache-dir flag to pip in Dockerfiles, to save space	2021-07-02 01:02:49 +05:30
README.md	update airflow	2020-08-17 15:45:42 +08:00
docker-stack.yaml	update airflow	2020-08-17 15:45:42 +08:00

README.md

airflow

How It Works

                   +---------+
              +--> | Worker1 |
 redis        |    +---------+
 postgres     |
+--------+    |    +---------+
| Master | <--+--> | Worker2 |
+--------+    |    +---------+
 webserver    |
 scheduler    |    +---------+
 flower       +--> | Worker3 |
                   +---------+

Quick Start

$ docker stack deploy -c docker-stack.yaml airflow
$ docker stack services airflow
$ docker service ps airflow_webserver
$ docker exec -it airflow_webserver.1.xxxxxx sh
>>> airflow create_user -r Admin -u admin -e admin@borderxlab.com -f Super -l Admin -p secret
>>> airflow list_users
╒══════╤════════════╤══════════════════════╤══════════════╤═════════════╤═════════╕
│   Id │ Username   │ Email                │ First name   │ Last name   │ Roles   │
╞══════╪════════════╪══════════════════════╪══════════════╪═════════════╪═════════╡
│    1 │ admin      │ admin@borderxlab.com │ Super        │ Admin       │ [Admin] │
╘══════╧════════════╧══════════════════════╧══════════════╧═════════════╧═════════╛
>>> exit

$ curl http://localhost:8080/
$ curl http://localhost:5555/

⚠️ You need to prepare nfs server with airflow.cfg.

$ python -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())'
CD2wL7G0zt1SLuO4JQpLJuHtBaBEcXWKbQyvkvf2cZ8=

⚠️ You should set another value to fernet_key in airflow.cfg to improve security.