Spark: Why the StructType merge method is private?

Question

There is a merge method in the spark.sql.types package:

private[sql] def merge(that: StructType): StructType

Is there a particular reason it is private? I would like to use it in my project, is there any other way than copying the code?

Answer 1

I know am very late but hope my answer might help someone else:

So, I found that StructType.merge() has not been included in spark version > 2.x

but Still you can use the StructType.merge() if you are using Spark with Java but in Scala it is not supported.

Work around for Spark with Scala:

val consolidatedSchema = test1Df.schema.++:(test2Df.schema).toSet
val uniqueConsolidatedSchemas = StructType(consolidatedSchema.toSeq)

Spark with Java:

StructType consolidatedSchema = test1Df.schema().merge(test2Df.schema());

If you want, you can check using Java and Scala with same version of Spark.