Am trying to export 250mb of data(75 chararray columns) from hdfs to sqlserver. It failed with the below error,
Caused by: java.io.IOException: com.microsoft.sqlserver.jdbc.SQLServerException: The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Too many parameters were provided in this RPC request. The maximum is 2100.
Then i passed "-D sqoop.export.records.per.statement=10" this statement along with sqoop export it worked but it is very slow. It took 15 minutes to load 250mb of data.
Is there anyway we can improve the performence.
Below is the actual sqoop cmd:
sqoop export -D sqoop.export.records.per.statement=10 --connect 'jdbc:sqlserver://199.198.165.191:1433;username=;password=;database=database' --table Facttable --columns DimDateID,DimQHourID,ETLMergedFileQHourlyNortelID,DimSWVersionID,DimFreqCellRelationID,OSSC_RC,SubNetwork1,SubNetwork2,MeContext,ENodeBFunction,EUtranCellFDD,EUtranFreqRelation,EUtranCellRelation,Time,GmtOffset,ffv,sn,st,vn,cbt,ts,neun,nedn,nesw,mts,gp,sf,pmHoExeAttLteInterF,pmHoExeAttLteIntraF,pmHoExeSuccLteInterF,pmHoExeSuccLteIntraF,pmHoPrepAttLteInterF,pmHoPrepAttLteIntraF,pmHoPrepSuccLteInterF,pmHoPrepSuccLteIntraF,Count_Null,Count_Negative,Count_Threshold,pmHoExeAttLteInterFLb,pmHoExeSuccLteInterFLb,pmHoOscInterF,pmHoOscIntraF,pmHoPrepAttLteInterFLb,pmHoPrepSuccLteInterFLb,pmHoPrepTNotAllowedLteInterF,pmHoPrepTNotAllowedLteIntraF,pmHoTooEarlyHoInterF,pmHoTooEarlyHoIntraF,pmHoTooLateHoInterF,pmHoTooLateHoIntraF,pmHoWrongCellInterF,pmHoWrongCellIntraF,pmHoWrongCellReestInterF,pmHoWrongCellReestIntraF,pmLbQualifiedUe,pmZtemporary36,pmHoExeAttLteIntraFTuneOut,pmHoExeSuccLteIntraFTuneOut --export-dir /Fact_Peg --direct -m 8 --input-fields-terminated-by "," --input-lines-terminated-by "\n";
"
