score:0
i am answering this question for sbt configurations. i also got the same issues which i resolved recently and made some basic mistakes which i would like you to note :
1. configure your sbt file
go to build.sbt file and see that the scala version you are using is compatible with spark.as per version 2.4.0 of spark https://spark.apache.org/docs/latest/ ,scala version required is 2.11.x and not 2.12.x . so, even though your ide (eclipse/intellij) shows the latest version of scala or the version you downloaded, change it to compatible version. also, include this line of code
librarydependencies += "org.scala-lang" % "scala-library" % "2.11.6"
2.11.x is your scala version
2. file hierarchy make sure your scala file is under /src/main/scala package only
3. terminal if your ide allows you to launch terminal within it, launch it(intellij allows, not sure of eclipse or any other) or go to terminal and change directory to your project directory
then run :
sbt clean
this will clear any libraries loaded previously or folders created after compilation.
sbt package
this will pack your files into a single jar file under target/scala-/ package
then submit to spark :
spark-submit target/scala-<version>/<.jar file> --class "<classname>(in your case , com.jsonreader.json)" --jars target/scala-<version>/<.jar file> --master local[*]
note here that -- if specified in a program isnt required here
score:1
hard to say, but maybe you have many classes that qualify as main so the build tool does not know which one to choose. maybe try to clean the project first sbt clean
.
anyway in scala the preferred way to define a main class is to extend the app -trait.
object someapp extends app
then the whole object body will become your main method. you can also define in your build.sbt the main class. this is necessary if you have many objects that extend the app -trait.
mainclass in (compile, run) := some("io.example.someapp")
Source: stackoverflow.com
Related Query
- main class not found in spark scala program
- Spark Scala Error - Error: Main method not found in class
- Spark Scala getting class not found scala.Any
- Scala IDE Error: Main method not found in class 'hello'
- Scala Main Class not found in Eclipse ide
- Error in running Scala Program: Main method not found in class main, please define the main method
- Main class not found error while creating Maven Project in Scala IDE
- Could not find or load main class in scala in intellij IDE
- reduceByKey method not being found in Scala Spark
- Scala Error: Could not find or load main class in both Scala IDE and Eclipse
- java.lang.NoClassDefFoundError: Could not initialize class when launching spark job via spark-submit in scala code
- Scala deserialization: class not found
- Scala IDE Error -could not find/load main class
- Scala - IntelliJ IDEA Error: Could not find or load main class
- Spark RDD Class not Found
- Write an Arbitrary Value Not Found in a Case Class Using Play's (2.2) Scala JSON Combinators
- Playframework, scala case class and property not found
- Test Class Not Found when Unit Testing on Scala
- Command Strategy Class Not found in Spark 1.3
- Could not find or load main class (classname) in Scala Intellij
- Task not serializable while using custom dataframe class in Spark Scala
- Apache Spark Mongo-Hadoop Connector class not found
- Not able to complete the word count program in spark using scala
- EMR always gives me Class Not Found for Scala app
- getOrElse method not being found in Scala Spark
- Compiling a Scala program failing Due to Dependencies not found
- Error: Could not find or load main class with Spark in Eclipse
- Scala class not found on classpath when specifying jar in classpath
- Dockerize a scala app: Could not find or load main class
- Implicits in a Spark Scala program not working
More Query from same tag
- The signature of Resolver.url in Sbt
- java.lang.RuntimeException: Multiple sources found for jdbc
- Message passing using Actors
- Is there a convenient helper in Play 2.4.x to build a uri from play.api.mvc.Request.queryString
- accessing host ip and port in play framework scala template?
- Type aliasing to avoid name clash in type refinement
- Find Maximun area on Array
- Typesafe Joins with Spark Datasets Less Safe Than I'd Expect
- ScalaJS fastOptJS and start script with sbt-native-packager
- How to modify values of JsonObject
- Titan DB bad index
- Failed to submit local jar to spark cluster: java.nio.file.NoSuchFileException
- RDF4J method for splitting literal into value and datatype
- cannot filter arbitrary HLists
- Save DF with JSON string as JSON without escape characters with Apache Spark
- Hashcode doesn't change between reruns
- State management not serializable
- Play Framework 2.2.1 / How should controller handle a queryString in JSON format
- [scala]How to make the BigDecimal is exact to integer part?
- Passing a list as varargs Scala
- Left Join errors out: org.apache.spark.sql.AnalysisException: Detected implicit cartesian product
- Scala can't recognize which method to call
- sbt.Resolve Exception : unresolved dependency: com.typesafe.sbteclipse#sbteclipse-plugin 2.4.0 not found
- Why does creating sbt project in IntelliJ IDEA cause "Permission Denied"?
- Contravariant binary operations in Scala
- Inserting iterated value to scala list
- How to define a global function in scala?
- What date/time class should I be mapping too?
- sbt val evaluated more than once
- Convert scala json string to an object