Serverless streaming ETL in with glue job & querying with Athena
-
Updated
Apr 5, 2021 - Python
#
glue-jobs
Here are 3 public repositories matching this topic...
Daily Incremental load ETL pipeline for Ecommerce company using AWS Lambda and AWS EMR cluster, Deployed using Apache airflow in a docker container.
docker dockerfile aws postgres lambda aws-lambda docker-compose docker-container aws-s3 lambda-functions postgresql aws-emr aws-ec2 redshift oltp aws-emr-clusters aws-glue redshift-aws glue-jobs lambda-trigger
-
Updated
Mar 17, 2023 - Python
Automated serverless ETL pipeline built on Aws.In this project transformation of CSV data ingested into S3 is done ,using Aws Lamda to trigger Aws Glue Workflow containing Crawlers and Spark ETL jobs.The processed data is then Loaded into Redshift,with real time notification via Amazon SNS through EventBridge.
aws glue pyspark event-driven redshift etl-pipeline sns-notifications eventbridge lamda-functions awslamda glue-jobs
-
Updated
Jun 22, 2025
Improve this page
Add a description, image, and links to the glue-jobs topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the glue-jobs topic, visit your repo's landing page and select "manage topics."