Spark lazy evaluation

Lazy Evaluation in Sparks means Spark will not start the execution of the process until an ACTION is called.


Ex:  RDD1==>> SELECT ==>> RDD2==>>GROUP==>>RDD3==>>FILTER()

  • until defended action, transformations will not trigger.
  • without action, data will not transfer from one RDD to another RDD
  • because of this we are calling data transformations are lazy evaluated

RDD1==>> SELECT ==>> RDD2==>>GROUP==>>RDD3==>>FILTER().SHOW().




With Lazy:

scala> val sparklist=List(1,2,3,4,5);
scala> lazy val output = sparklist .map(List=>List*10)
scala> println(output)
List(10, 20, 30, 40, 50)