Nexoryx_Spark_Terraform
Nexoryx_Spark_Terraform
Production-ready Terraform automation for deploying an Apache Spark cluster on AWS with:
- 1 Spark Master
- 20 Spark Workers
- Ubuntu 24.04
- Docker-based Spark deployment
- Security Groups
- Auto provisioning
- Scalable infrastructure
Architecture
AWS VPC | +– Spark Master EC2 | +– 20 Spark Worker EC2 Instances
Features
- Terraform-based infrastructure automation
- Spark Master auto configuration
- Spark Worker auto join
- Security groups
- SSH access
- Ubuntu 24.04
- Docker installation
- Apache Spark cluster deployment
Requirements
- AWS Account
- Terraform >= 1.5
- Existing AWS key pair
- AWS CLI configured
Usage
Initialize
terraform init
Plan
terraform plan
Deploy
terraform apply -auto-approve
Destroy
terraform destroy -auto-approve
Outputs
- Spark Master Public IP
- Spark Master URL
- Worker Instance IDs
Spark Web UI
http://SPARK_MASTER_IP:8080
License
MIT License
Project Roadmap
- Kubernetes Helm charts
- GitOps support
- CI/CD improvements
- Monitoring dashboards
- Multi-cloud support
- Security hardening
GitHub Actions
This repository includes:
- Shell validation
- Markdown linting
- Terraform validation (where applicable)
Example Deployments
See:
- examples/
- docs/
Related Nexoryx Projects
This repository is part of the Nexoryx infrastructure ecosystem.