score:-1

Think I found a solution, downgrading the Scala version from 2.12.8 to 2.11.12 solved the issue.

score:0

Try persisting the dataframe. When multiple actions are performed over the same dataframe, it is better to persist it rather than going through the cycle again

lines.persist(MEMORY_AND_DISK)

Related Query

More Query from same tag