I've some 3rd party jars in my spark code, bundled using maven and submit.
Want to know, when ever we submit our spark job, where it copies all the required 3rd party jars in worker node?
Can we configure that path so that spark could copies jars there only in worker nodes? if yes, then where it can be configured?
There is change in 3rd party jar which I used to use in my spark code and didn't change the jar version. So while running with new jar with same version name
(1.1.1), its still picking old jar(1.1.1). Any suggestion, what can be done?