SSIS BIML Derived Column syntax for expressions

Question

I am defining a Derived Column transformation in BIML but I am having trouble referencing the output from the previous Excel Source in my Derived Column transformation.

I receive the error upon opening the package after successfully generating the SSIS package and it suggests that it the Derived Transformation cannot find the output from the Excel Source.

Error 2 Error loading AFR_ShareTableBIML.dtsx: The object "/DTS:Executable/DTS:Executables/DTS:Executable/DTS:ObjectData/pipeline/components/component/inputs/input/inputColumns/inputColumn/properties/property" references ID "#{Package\Data Flow {Import Share Table CSV}\Source {Flat File Share Table}.Outputs[Output].Columns[Div c per share]}", but no object in the package has this ID.

Here is a code snippet:

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<FileFormats>
    <FlatFileFormat Name="FFF_AFRShareTable" ColumnNamesInFirstDataRow="true"
            FlatFileType="Delimited" IsUnicode="false" TextQualifer="None" HeaderRowsToSkip="6">
        <Columns>
            <Column Name="Quote Buy" ColumnType="Delimited" DataType="AnsiString" Length ="50" Delimiter=","></Column>
            <Column Name="Quote Sell" ColumnType="Delimited" DataType="AnsiString" Length ="50" Delimiter=","></Column>
            <Column Name="Div c per share" ColumnType="Delimited" DataType="AnsiString" Length ="50" Delimiter=","></Column>
        </Columns>
    </FlatFileFormat>
</FileFormats>
<Connections>
    <FlatFileConnection Name="FF_AFRShareTable" FileFormat="FFF_AFRShareTable"
        FilePath="C:\Temp\Stocks.csv"></FlatFileConnection>
    <OleDbConnection Name="CMD DB"
        ConnectionString="Data Source=Localhost;Initial Catalog=DB;Provider=SQLNCLI11.1;Integrated Security=SSPI;" CreateInProject="true">
    </OleDbConnection>
</Connections>
<Packages>
    <Package Name="AFR_ShareTableBIML" ConstraintMode="Linear" ProtectionLevel="DontSaveSensitive">
        <Tasks>
            <ExecuteSQL Name="SQLTask {OLE_DB} Truncate Security Share Table" ConnectionName="CMD DB">
                <DirectInput>truncate table Staging.SecurityShareTable</DirectInput>
            </ExecuteSQL>
            <Dataflow Name="Data Flow {Import Share Table CSV}">
                <Transformations>
                    <FlatFileSource Name="Source {Flat File Share Table}" ConnectionName="FF_AFRShareTable"></FlatFileSource>
                    <DerivedColumns Name="DER_NullifyColumns">
                        <Columns>
                            <Column Name ="DER_DPS" DataType = "Decimal" Precision="4">
                                [Div c per share] == "-" ? NULL(DT_DECIMAL, 4)  : (DT_DECIMAL, 4)[Div c per share]
                            </Column>
                        </Columns>
                    </DerivedColumns>
                </Transformations>
            </Dataflow>
        </Tasks>
    </Package>
</Packages>

I have already defined the column name via the FlatFileFormat and I have confirmed that the expression in the DER_DPS column is is syntactically correct. I found that through replacing the square brackets "[" and "]" with double apostrophes, the SSIS package can be opened. For example:

"Div c per share" == "-" ? NULL(DT_DECIMAL, 4)  : (DT_DECIMAL, 4) "Div c per share"

However there are derived column transformation errors on incorrect syntax. Are square brackets special characters in BIML that I need to escape?

billinkc billinkc · Accepted Answer · 2016-06-15T15:34:35

That was ... interesting.

It appears that your use of curly braces in your component names causes the Biml expansion to go haywire.

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
    <FileFormats>
        <FlatFileFormat Name="FFF_AFRShareTable" ColumnNamesInFirstDataRow="true"
                FlatFileType="Delimited" IsUnicode="false" TextQualifer="None" HeaderRowsToSkip="6">
            <Columns>
                <Column Name="Quote Buy" ColumnType="Delimited" DataType="AnsiString" Length ="50" Delimiter=","></Column>
                <Column Name="Quote Sell" ColumnType="Delimited" DataType="AnsiString" Length ="50" Delimiter=","></Column>
                <!-- Change -->
                <Column Name="Div c per share" ColumnType="Delimited" DataType="AnsiString" Length ="50" Delimiter="CRLF"></Column>
            </Columns>
        </FlatFileFormat>
    </FileFormats>
    <Connections>
        <FlatFileConnection Name="FF_AFRShareTable" FileFormat="FFF_AFRShareTable"
            FilePath="C:\ssisdata\so\input\Stocks.csv"></FlatFileConnection>
        <OleDbConnection Name="CMD DB"
            ConnectionString="Data Source=Localhost\dev2014;Initial Catalog=tempdb;Provider=SQLNCLI11.1;Integrated Security=SSPI;" 
            CreateInProject="false">
        </OleDbConnection>
    </Connections>
    <Packages>
        <Package Name="so_37641290_AFR_ShareTableBIML" ConstraintMode="Linear" ProtectionLevel="DontSaveSensitive">
            <Tasks>
                <ExecuteSQL Name="SQLTask OLE_DB Truncate Security Share Table" ConnectionName="CMD DB">
                    <DirectInput>truncate table Staging.SecurityShareTable</DirectInput>
                </ExecuteSQL>
                <Dataflow Name="Data Flow Import Share Table CSV">
                    <Transformations>
                        <FlatFileSource Name="Source Flat File Share Table" ConnectionName="FF_AFRShareTable"></FlatFileSource>
                        <DerivedColumns Name="DER_NullifyColumns">
                            <Columns>
                                <Column Name="DER_DPS" DataType="Decimal" Precision="4"><![CDATA[[Div c per share] == "-" ? NULL(DT_DECIMAL, 4)  : (DT_DECIMAL, 4)[Div c per share]]]></Column>
                            </Columns>
                        </DerivedColumns>
                    </Transformations>
                </Dataflow>
            </Tasks>
        </Package>
    </Packages>
</Biml>

The above biml works for me. Changes I made:

removed { and } from the tasks and component names
updated the last Column definition within your FlatFileFormat Columns collection to have a delimiter of CRLF instead of ,
I used the CDATA tag for the expression. Not needed here but if you had a > or < in there, then you'd need to escape them as either < or the CDATA approach as I used.
I also cleaned up the Derived Column's entity assignments. There were spaces around the equals and I don't believe those are supposed to be there.
Path updates for flat file + OLE DB to work with my setup.

Source data

0
1
2
3
4
5
Quote Buy,Quote Sell,Div c per share
1,1,1
2,2,2
3,3,-

SSIS BIML Derived Column syntax for expressions

1 Answers

Source data

Results