[Solved]-New column receives the value Null-scala

That happens because you use lit. You should use col instead:

df
  .withColumn(
    "Rate" ,when(col("nodes_count") =!= 0,
    (col("edges_count") / col("nodes_count")).as[Double]).otherwise(0.0))

although both when and as Double are useless here, and simple division would be more than sufficient:

df.withColumn("Rate", col("edges_count") / col("nodes_count"))