Add verify_certs = False possibilities

[dlubomski]

There should be an option to disable certificate verification during SSL connection. It will simplify developing and debugging process.

[priamai]

I think this is already working, let me check my code.

[priamai]

Okay my case is a bit different, I am using a self signing certificate and disabling check like so:

import pyspark
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, count

ss = (
    SparkSession.builder.master('local[24]').appName("ES")
    .config("spark.driver.memory", "8g")
    .getOrCreate()
)

es_reader = (ss.read
    .format("org.elasticsearch.spark.sql").option("inferSchema", "true")
    .option("es.read.field.as.array.include", "tags").option("es.nodes","elasticsearch:9200")
    .option("es.net.http.auth.user","xxxxx").option("es.net.http.auth.pass","xxxxx")
             .option("es.net.ssl","true").option("es.net.ssl.cert.allow.self.signed","true"))

events_df = es_reader.load("myindex/_doc")

[sss-ng]

Does .option("es.net.ssl.cert.allow.self.signed","true") not do what you are requesting. That works for me

[ebuildy]

Actually, this is not the same thing than "verify = None".

The code behind es.net.ssl.cert.allow.self.signed = true :

https://github.com/elastic/elasticsearch-hadoop/blob/master/mr/src/main/java/org/elasticsearch/hadoop/rest/commonshttp/SSLSocketFactory.java#L90

private static class SelfSignedStrategy implements TrustStrategy {
        public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException {
            return chain.length == 1;
        }
    }

What about if a self-signed cert contains 2 cert in the chain?

[j-adamczyk]

@ebuildy it is currently bugged and does not work: #1146