Enumerating RecordSet in DataFlow Script Component as Data Source

Question

This is an SSIS related issue

I have a variable that is set to type object. One Dataflow imports some filtered rows into a recordset and this recordset is stored in the object variable.

In a completely seperate dataflow i need to use that recordset as the source. So i created a script component and told it it would be a data source.

I set it up to have the three output columns that i need. My problem is, how do i get every row in the recordset to create a new row in the script component?

I passed in the recordset variable as a readonly variable, when i try to foreach the variable to get to each row i cannot do it because the variable doesn't define a get enumerator method.

Therefore i cant print each row out into those columns and cannot use my script component as a data source.

Has anyone else faced a similar situation? Am i doing something stupid or did you do it in another alternative way?

As a note i am using C# in the script and Visual studio 2008

Answer 1

So i looked around a bit and found a VB solution to my issue, i translated it over into C# and this now compiles and behaves as expected. The code i used was this:

    DataTable datatable = new DataTable();
    System.Data.OleDb.OleDbDataAdapter oAdapter = new System.Data.OleDb.OleDbDataAdapter();

    oAdapter.Fill(datatable,ReadOnlyVariables["User::XXXXX"]);

    foreach (DataRow row in datatable.Rows)
    {
        Output0Buffer.AddRow();
        Output0Buffer.CoverAmount = Convert.ToInt32(row["XXXX"].ToString());
    } 

for any others who face a similar issue!

Thanks to all for the help

Answer 2

I do something similar based on an older versio of Andy Leonard's Incremental Load framework. Our child packages populate a recordset indicating how many new, changed, unchanged, etc row counts we had. In the parent package we test that object to ensure it's been populated before we use it. I need to skip to a meeting so pardon this code as it doesn't quite solve your specific need but hopefully provides a solid shove in the right direction until I can come back in and tailor for your case. I have pseudocode in there for where you'll want to do stuff.

    public void Main()
    {
        bool debug = Convert.ToBoolean(Dts.Variables["Debug"].Value);
        string taskName = string.Empty;
        string packageName = string.Empty;
        string sourceName = string.Empty;
        bool fireAgain = false;

        taskName = Convert.ToString(Dts.Variables["TaskName"].Value);
        packageName = Convert.ToString(Dts.Variables["PackageName"].Value);
        // Fix this by defining and passing in params
        sourceName = Convert.ToString(Dts.Variables["TaskName"].Value);

        System.Data.OleDb.OleDbDataAdapter adapater = null;
        System.Data.DataTable table = null;
        System.Data.DataColumn column = null;
        System.Data.DataRow row = null;
        string message = string.Empty;

        object rowCounts = null;
        rowCounts = Dts.Variables["RowCounts"].Value;
        table = new DataTable();

        try
        {
            // Get us out of this crazy thing - should only be an issue
            // first pass through
            if (rowCounts == null)
            {
                Dts.TaskResult = (int)ScriptResults.Success;
                return;
            }
        }
        catch (Exception ex)
        {
            throw new Exception("Failed here");
        }

        adapater = new System.Data.OleDb.OleDbDataAdapter();
        try
        {
            // This works if we pass in a dataset
            //adapater.Fill(table, Dts.Variables["RowCounts"].Value);
            adapater.Fill(table, rowCounts);
            // TODO: Enumerate through adapter
            // Call Output0Buffer.AddRow();
            // and Output0Buffer.MyColumn.Value = adapter[i].value // possibly casting to strong type
        }
        catch (Exception ex)
        {
            try
            {
                // This works if we use a datatable
                System.Data.DataSet ds = null;
                //ds = (DataSet)Dts.Variables["RowCounts"].Value;
                ds = (DataSet)rowCounts;
                table = ds.Tables[0];
                // TODO: Enumerate through datatable as we do with adapter

            }
            catch (Exception innerException)
            {
                // continue to swallow exceptions
            }
            Dts.Variables["ValidCounts"].Value = false;

            // trap "Object is not an ADODB.RecordSet or an ADODB.Record
            // parse ex.Message
            if (ex.Message.Contains("System.ArgumentException: "))
            {
                System.Text.StringBuilder exceptionMessage = null;
                exceptionMessage = new System.Text.StringBuilder();
                exceptionMessage.Append(ex.Message);
                exceptionMessage.Replace("\nParameter name: adodb", string.Empty);
                exceptionMessage.Replace("System.ArgumentException: ", string.Empty);

                if (exceptionMessage.ToString() != "Object is not an ADODB.RecordSet")
                {
                    Dts.Events.FireInformation(0, string.Format("{0}.{1}", packageName, taskName), exceptionMessage.ToString(), string.Empty, 0, ref fireAgain);
                }
            }
        }

        Dts.Variables["ValidCounts"].Value = false;
        if (table.Rows.Count > 0)
        {
            Dts.Variables["ValidCounts"].Value = true;
        }

        if (debug)
        {
            message = string.Format("SourceName:  {0}\nValidCounts:  {1}", sourceName, false);
            //System.Windows.Forms.MessageBox msgBox = null;
            //msgBox = new MessageBox();
            System.Windows.Forms.MessageBox.Show(message, string.Format("{0}.{1}", packageName, taskName));
            //MessageBox(message, string.Format("{0}.{1}", packageName, taskName));
        }

        Dts.TaskResult = (int)ScriptResults.Success;
    }

Answer 3

I'm working on the same problem as OP. In my case, I assumed that it was an object, and I spent a lot of time trying to convert it to a datatable. What I discovered was that the object was already a datatable, so from an SSIS Script Task Component, I was able to write this:

DataTable dt = (DataTable)ReadOnlyVariables["User::FTP_DataPath_File_Metadata"].Value;

foreach (DataRow row in dt.Rows)
{
    CustomOutputBuffer.AddRow();
    CustomOutputBuffer.FileName = row.ItemArray[0].ToString();
    CustomOutputBuffer.FileLastModified = Convert.ToDateTime(row.ItemArray[1]);
    CustomOutputBuffer.FileSize = Convert.ToInt32(row.ItemArray[2]);
}

This successfully converted my "object variable" to a dataflow, using the Script Component as a Source.

This example was made to accommodate the Task Factory Secure FTP Component, which saves the results of "Get file list with metadata" to an object variable.