Lakhasly

3.2.1 Resilient Distributed Dataset (RDD): it works as a collection of elements which are operated in parallel.In the distributed file system Spark runs on Hadoop cluster, and RDD is created from files in the format of text or sequence files.RDD is used for reading the objects in the collection and when some partition is lost, it can be rebuilt because RDDs are distributed across a set of machines.

3.2.1 Resilient Distributed Dataset (RDD): it works as a collection of elements which are operated in parallel. In the distributed file system Spark runs on Hadoop cluster, and RDD is created from files in the format of text or sequence files. RDD is used for reading the objects in the collection and when some partition is lost, it can be rebuilt because RDDs are distributed across a set of machines. The two operations supported by RDD: Transformations and Actions. 3.2.2 Directed Acyclic Graph The data flow is cyclic in Directed Acyclic Graph (DAG) engine. To perform on cluster, a DAG of task stages is created by each Spark job. In map and reduce stage DAG is created 4. It just takes one single stage to complete simple jobs and multiple stages to complete complex jobs in one single run. Thus, jobs completed faster than MapReduce [3].