Create Data Lake on AWS S3 to store dimensional tables after processing data using Spark on AWS EMR cluster
This is our final project for SFU's CMPT 353 taught by Greg Baker during Summer 2023
work with a flight dataset and use Spark SQL to analyze flight delays, airport traffic, and other key metrics
Advanced Topics in Databases course project - NTUA ECE - 2022-23
Some batch processing demos with various data warehouses like local, S3 and HDFS in AWS
This project is about exploring and analysing E-commerce data. This primarily includes leveraging Apache Spark Dataframe API, joins, functions and aggregations to generate summarized results.
Treat Spark like pandas.
A big data analysis of Los Angeles crime and demographic data using Apache Spark on Kubernetes.
To associate your repository with the spark-dataframes topic, visit your repo's landing page and select "manage topics."