Press "Enter" to skip to content

Category: Integration Services

Using a File Cache in SSIS

Published 2025-07-07 by Kevin Feasel

Andy Brownsword makes those SSIS jobs run faster:

Last week we looked at using a cache to improve lookup performance. We saw how a cache improves performance by being able to reuse reference data repeatedly. That used a regular cache but it’s not the only option available to us.

In this post we’re going to look at the File Cache option which can achieve the same results – plus a little more.

My experience with file caches is that they’re a bit finicky but when you get them running, they can provide a significant speed-up to data enrichment tasks.

Leave a Comment

Optimizing Multiple Lookup Transformations in SSIS

Published 2025-06-30 by Kevin Feasel

Andy Brownsword doesn’t want to keep hitting the database:

Lookup transformations provide us a way to access related values from another source, such as retrieving surrogate keys in data warehousing. When we need multiple lookups to the same reference data we can improve performance through the use of a Cache.

If we consider data warehousing, a prime example of this would be an order table which has values for Order Date, Dispatch Date, Delivery Date, etc. All of these would require a lookup to a calendar dimension.

This is a perfect use case for a cache.

Read on to see how the cache connector works.

Leave a Comment

Tuning SSIS Data Flow Buffers

Published 2025-06-09 by Kevin Feasel

Andy Brownsword speeds things up:

When using data flows in SSIS packages we want the flow to be fast and fluid. Improving performance of the flows will vary in different packages, but one element can consistently help – tuning memory utilisation.

In this post we’ll look at tuning the memory utilisation by altering the number of rows passing through the flow concurrently. Specifically we’re looking at the following properties:

  • DefaultBufferSize
  • DefaultBufferMaxRows

This is a pretty big deal, finding the right size that allows you to keep data flowing smoothly without having to wait for buffers to fill. The defaults are aggressively low in most cases. And good on Andy for providing a series of tests to give practical numbers.

Comments closed

Handling a Sort Operation in SQL Server Integration Services

Published 2025-04-07 by Kevin Feasel

Andy Brownsword knows that sometimes, the only winning move is not to play:

Last time out we discussed blocking transformations, what they are, the impact of them, and touched on how to deal with them. In this post we’re going a step further to tackle one of them head on.

Here we’ll demonstrate the impact of blocking caused by the Sort transformation, and look at two options for solving this and slashing execution time.

Sorts aren’t the only blocking transformation that you should push back down to your source (if possible), but it is the most common example.

Comments closed

Blocking Transformations in SSIS Data Flows

Published 2025-03-26 by Kevin Feasel

Andy Brownsword doesn’t want any blockers:

We have a platter of transformations to use when constructing data flows in SSIS packages. Not all transformations are equal though, and some can catch us out with performance impact as data volumes scale.

In this post we’ll look at blocking transformations which can trip us up if we’re not careful with them (or avoid them).

Click through for some examples of blocking transformations, as well as some tips on how to manage them.

Comments closed

Dealing with Optional Carriage Returns in SSIS

Published 2025-03-04 by Kevin Feasel

Andy Brownsword has fun with file formats:

When ingesting files in SSIS via Flat File Connections, a consistent format is key. Sometimes that isn’t the case. Here we’ll look at an example where the carriage return (CR, \r) may or may not be included in the file.

Pepperidge Farms remembers back in the day when Windows, MacOS, and Linux (or any flavor of UNIX for that matter) each had a different way of ending a line: line feed, carriage return, or both. And of course most tools weren’t smart enough to figure out which your particular text file followed and display it correctly.

Comments closed

Debugging SSIS Packages

Published 2025-02-28 by Kevin Feasel

Andy Brownsword wrote code with an error in it:

When constructing or investigating a SSIS package we can run into a variety of issues. To help resolve there are various techniques which can be used to troubleshoot the package.

Whilst we have the Progress tab for the package to tell us what’s happened during execution, it’s usually more effective to debug packages in flight. Below we’ll look at a few ways we can achieve this:

Never having written code that doesn’t work perfectly the first time, this post is, of course, merely academic for all of us perfect developers. But for those people who may have let a little something sneak in or have to deal with the errors that others have seeded into our beautiful programs, Andy provides three separate techniques for troubleshooting packages in flight.

Comments closed

REPLACENULL in SSIS versus DT_DBTIMESTAMP2 Columns

Published 2025-02-11 by Kevin Feasel

Slava Murygin notes an error:

 Using the “REPLACENULL” functionality frequently in the “Derived Column” component, the “Conditional Split” component, and other places in SSIS where formulas can be applied is common.

However, I recently encountered an issue with the “DT_DBTIMESTAMP2” data type.

The following formula produced an error:

REPLACENULL(TestDt, (DT_DBTIMESTAMP2,7)”1900-01-01 00:00:00.0000000″)

Error: 0xC020902A at Test Transformation, Derived Column [2]: The “Derived Column” failed because truncation occurred, and the truncation row disposition on “Derived Column.Outputs[Derived Column Output].Columns[TestDt]” specifies failure on truncation. A truncation error occurred on the specified object of the specified component.

Read on for an explanation and two alternatives.

Comments closed

SSIS: Operating Completed with Problems Logged

Published 2025-02-03 by Kevin Feasel

Slava Murygin uses Integration Services to process an Analysis Services cube:

If you process SSAS cubes via SSIS packages you might notice a weird message like “Full Processing:Warning: Server: Operation completed with XXX problems logged.“

How you can get that message (if you have that problem):

Slava notes that the simple solution doesn’t actually work, as both services decide not to store that information. But Slava does provide a solution to the problem.

Comments closed