score:-1

You can't access df from otherFunction as it is declared at the scope of createDF. I would change it something like this.

def createDF() = {
  spark.read.avro("fileA")
}

def otherFunction(create: () => DataFrame) = {
  val df = create()
  ...
}

And call otherFunction as otherFunction(createDF). Another approach using implicits...

implicit val df = createDF()

def otherFunction(implicit df: DataFrame) = {
  ...
}

score:0

You can't directly, variables declared in a function have the function scope and are only visible inside the function.

If you want to reuse the variable, you need to set it as a class variable:

class MyClass {
  private var df: Option[DataFrame] = None

  private def createDF() = {
    this.df = Some(spark.read.avro("fileA"))
  }
  def otherFunction() = {
    this.df.map(df => ...)
  }
}

I use Option here to avoid NullPointerExceptions that may arrive when using null if the variable is not yet initialized


Related Query