score:1
Accepted answer
Please look at the option "es.nodes.wan.only". By default, the value for this key is set to "false", and when I set it to true, that exception went away. Here is the current documentation for the configuration values: https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html.
val conf = new org.apache.spark.SparkConf()
.setMaster("local[*]")
.setAppName("es-example")
.set("es.nodes", "search-2meoihmu.us-est-1.es.amazonaws.com")
.set("es.nodes.wan.only", "true")
Note that the doc specifies to flip this value to true for environments like those on AWS, but this exception happened for me when attempting to point to a VM with Elasticsearch running.
Source: stackoverflow.com
Related Query
- Getting connection error while reading data from ElasticSearch using apache Spark & Scala
- How to convert types when reading data from Elasticsearch using elasticsearch-spark in SPARK
- Reading data from mongodb to spark using scala from docker. Getting 0 records, why?
- Getting Error while connection with ElasticSearch in Spark Streamming
- reading data from URL using spark databricks platform
- How to read json data using scala from kafka topic in apache spark
- Parallelism in reading Oracle data from using Spark 1.6.2 JDBC
- Reading Nested data from ElasticSearch via Spark Scala
- Getting out of memory error while reading parquet file in spark submit job
- Apache spark - loading data from elasticsearch is too slow
- java.lang.NullPointerException while reading data from MSSQL server with spark
- getting Py4JJavaError while using apache spark big query connector
- getting error while importing data into snappy data from csv in java
- Getting error while defining schema for csv file in spark using scala
- Issue in spliting data in Txt file while converting from RDD to DataFrame in Spark using Scala
- how we can sort line by line data from txt file using Apache Spark scala?
- Error in spark submit java.lang.OutOfMemoryError while reading 5-6 GB of text file using wholetextfile method
- Getting error while converting DynamicFrame to a Spark DataFrame using toDF
- Error while calling udf from within withColumn in Spark using Scala
- Error while creating a table from another one in Apache Spark
- Getting 'Query exceeded the maximum allowed memory usage of 40 MB' while loading huge data from MongoDB to spark dataframe
- Getting Error While Creating JAR from the SPARK SCALA MAVEN Project in Eclipse IDE
- error while fetching data from column of type case class in spark scala
- Error while reading an Excel file using Spark Scala with filename as a parameter
- How can I skip ssl.truststore.password property while reading from Kafka using Spark Structured streaming?
- Error While reading Teradata from Spark. It loaded the Table and Shows Schema But fails to give data set result
- Getting the number of records read from Cassandra table while using Spark Cassandra Connector
- End of file exception while reading a file from remote hdfs cluster using spark
- Throwing NullPointerException while reading from MySQL in Apache Spark with Scala
- NoClassDefFoundError: org/apache/hadoop/fs/StreamCapabilities while reading s3 Data with spark
More Query from same tag
- Lift - Display notice after signup
- Finding lengths between characters in a string
- How to test ask pattern with fallback
- Type Constraints with 2 Parameters
- How aggregate by value in apache spark
- Idiomatic way to continuously poll a HTTP server and dispatch to an actor
- Hive data to Kafka topic using Spark
- How can I consume a Scala macro/quasiquote for code templates?
- Scala : Can there be covariance in a constructor for defined type?
- Using Spark 3.2 to ingest IoT data into delta lake continuously
- From Scala, access static inner class of Java class
- Scala String parser
- Read CSV file in Spark and insert it to HBase using created RDD
- Scala : how to expose regex variables bound within a case statement
- Variable rampUp with Gatling
- How to write to HDFS using spark programming API if I have authentication details?
- How can I externalize these play libraries
- Merging HList elements
- How is an imported name resolved in Scala? (Spark / Zeppelin)
- Hudi: Access to timeserver times out in embedded mode
- Scala source external to SBT project folder
- Run list of akka actors for list of messages
- Not able to send multiple message with same websocket connection using Akka-Http
- ZeroMQ Callbacks for Java/Scala
- Cannot set breakpoints in scala editor by using eclipse scala IDE
- how to find the tuple value using scala?
- Scala tailrec annotation error
- ActorSystem singleton injection
- AffinityKeyMapped not working with Ignite 2.4/2.5/2.6 and Scala
- Implicit class for overriding String.toBoolean