We have migrated some ELT jobs to Apache Airflow scripts.These scripts move Oracle DB data to Google Bigquery.Steps requiredBuild apache airflow in docker (used puckel/docker-airflow image)As root user update docker imageAs airflow user install Java 1.8 + (required for running Talend ETL tools)Convert the export bash script to a DAGEnjoy the scheduling, retries and logging […]
Continue Reading...
