Ranger Policy Sync Fails in Spark Driver on Remote Nodes (Kyuubi/Kerberos/LDAP)

[hw-zhangyl]

We're encountering a Kerberos authentication issue in our Kyuubi (1.10.1) / Spark (3.4.2) / Ranger (2.4.0) cluster. Our setup uses LDAP for user authentication and Kerberos for service authentication.

The problem is: when a Spark Driver (launched by Kyuubi) starts on a remote machine within the cluster, its attempts to periodically fetch the latest authorization policies from the Ranger Admin server fail. The error observed is "simple user authentication failed".

This prevents the Spark application from receiving dynamic Ranger policy updates, leading to stale authorization decisions.

Steps to Reproduce
Cluster setup: Kyuubi 1.10.1, Ranger 2.4.0, Spark 3.4.2 with Kerberos + LDAP.

Submit a SQL query via Kyuubi.

Observe the Spark Driver launching on a non-Kyuubi Server host.

Check Spark Driver logs for "simple user authentication failed" errors during Ranger policy pull attempts.

Actual Behavior
Spark Driver on remote nodes fails to authenticate with Ranger Admin for policy synchronization, resulting in "simple user authentication failed" errors.

Environment
Kyuubi Version: 1.10.1

Ranger Version: 2.4.0

Spark Version: 3.4.2

Authentication: Kerberos + LDAP