Want some ideas how to implement the below logic please?
This question relates to SSIS package. I am reading flat file of fixed width, which will contain at least 500 rows. I have a lookup table in SQL with a single two columns ID and Description. If any of the rows of flat file contain the lookup table description column data, then those rows should not be processed.
What I have done so far:
I am using a script task and loading lookup column data in to data reader. And checking every column of the input file in the script task to see if it contains the data reader items, then that row is not processed. I know this giving me bad performance, so I just need an approach how to tackle this problem. I am not asking for solution. Please let me know, if you need more information about the problem.