Looping Over Flat Files

Kevin Feasel

2016-07-19

Biml

Ben Weissman uses Biml to loop over flat files with matching schemas:

To get that done in SSIS, we need:

– A variable to store the current file name and will be used as the connection string for our flat file connection
– A for each loop that will loop through our directory, pass each file name in there to our variable and then run a data flow task
– A derived column in our data flow task that will store our variable so we can import it into our SQL table
– Since we want this to be “clean”, we will also run a truncate before the for each loop

So let’s look at the code for that

It’s Biml, so there isn’t much code.  Check it out.

Related Posts

Remote Code Execution with CallBimlScriptContent

Kevin Feasel

2019-03-01

Biml

Bill Fellows describes the ability to execute code in Biml from a remote source with the CallBimlScriptContent function: CallBimlScriptContent was introduced with the migration from Mist to BimlStudio. Why is this cool? You do not have to use files sitting on your computer as the source for your Biml. As long as you can reconstitute the […]

Read More

Generating SQL With Biml

Cathrine Wilhelmsen shows us you can do a lot more with Biml than just generating SSIS packages: This actually happened to me in a previous job. We had a fairly complex ETL solution for the most critical part of our Data Warehouse. Many SSIS packages, views, and stored procedures queried the tables that were replicas […]

Read More

Categories

July 2016
MTWTFSS
« Jun Aug »
 123
45678910
11121314151617
18192021222324
25262728293031