I busted some grey matter to make this Intellij Idea Spark project to just work, so you don't have to.
This project can run a Spark job with SBT run and can generate a fat jar to be uploaded to a cluster.
Clone this repo and start developing RDDs and Datasets. But first, check the prereqs.
To run local you need the following software:
- Spark
- Hadoop (Optional, only if you want to read/write from HDFS)
TODO: Add a guide on how to configure Spark in Windows