Press "Enter" to skip to content

Category: Integration Services

Thoughts on the State of SSIS

Published 2026-01-05 by Kevin Feasel

Andy Leonard shares some thoughts:

At the outset of 2026, I live in two worlds.

I continue to support clients who use SSIS for enterprise data engineering. Some of the enterprises are small-ish by comparison. Others are huge. Some friends also continue to support these clients and clients like them. That’s World 1.

World 2 is social media. I have some friends on social media. Many I know IRL (in real life). Most, though? Most are acquaintances. Most of my interactions on social media are with people I don’t really know or don’t know that well. Conversely, many of them – of you – don’t know me that well, either. I touched on this in a recent newsletter / post titled 2025: A Number Containing Two 2’s, One 5, and a 0.

My biggest problem with SSIS is that Microsoft has almost zero real effort into it in the past decade, which makes me hesitant to use it anymore. Yeah, I have gripes about CI/CD—BIML resolved a lot of those for me, but I haven’t seen anybody talk about that topic in the past 8 or so years, either. Andy has done a lot of good stuff with his DILM suite as well, and there are some good third-party components that implement functionality Microsoft never got around to doing. But at the end of the day, how long does Microsoft continue to “support” SSIS, and at what point does this become a corporate risk?

Leave a Comment

What’s New in SSIS for SQL Server 2025

Published 2025-12-15 by Kevin Feasel

Chunhua Gu says, not much:

Security is a top priority for SSIS 2025, reflecting the broader enterprise’s focus on data protection and compliance. Microsoft.Data.SqlClient provides a modern, secure data access layer. This new provider supports advanced security protocols, including TLS 1.3 for encrypted connections, and integrates seamlessly with Microsoft Entra ID (formerly Azure Active Directory) for robust authentication.

In short, support the new-ish library (that has been around for several years), tie in with Microsoft Fabric, remove functionality that used to be in the product while spinning this as a grand new opportunity for developers to spend money on Fabric, and that’s it. Granted, SSIS hasn’t been a proper focus for the product since 2012 (sorry, Hadoop components in 2016—you’re out of the product now, so you don’t count), so all of this should come at no surprise.

Comments closed

Using Python Code in SSIS

Published 2025-08-12 by Kevin Feasel

Tim Mitchell shoe-horns a language in:

SQL Server Integration Services (SSIS) is a mature, proven tool for ETL orchestration and data movement. In recent years, Python has exploded in popularity as a data movement and analysis tool. Surprisingly, though, there are no native hooks for Python in SSIS. In my experience using each of these tools independently, I’d love to see an extension of SSIS to naturally host Python integrations.

Fortunately, with a bit of creativity, it is possible to invoke Python logic in SSIS packages. In this post, I’ll walk you through the tasks to merge Python and SSIS together. If you want to follow along on your own, you can clone the repo I created for this project.

Honestly, it’s not that surprising. The last time there was significant development on Integration Services was roughly 2012 (unless you include the well-intentioned but barely-functional Hadoop support they added in around 2016). At that point, in the Windows world, Python was not at all a dominant programming language.

Comments closed

Tips for Solving SSIS Package Bottlenecks

Published 2025-08-12 by Kevin Feasel

Andy Brownsword has some advice:

Last time out we started to look at optimising SSIS packages by showing how to identify bottlenecks with a handy script. This time we’re turning insights into action to solve those pain points.

The solutions are grouped into 3 areas: Data Flows, as they do a lot of heavy lifting; the Execute SQL task, which can also be used for transformation and calculations; and finally everything else (because the first two are usually the issue).

Andy has some good advice and plenty of links to prior content around optimizing SSIS performance. One small thing I’d add is architectural: think about whether you can solve the slow part inside SQL Server. If you’re grabbing a huge amount of data from a SQL Server instance and then narrowing it down with filters, it might be a lot faster to transform that into a SQL query with a stronger WHERE clause. But let’s say there’s some small file you’re using to filter, so you need to pull all of the data out of SQL Server to compare against the small file so that you know what you need. Instead of pulling all of the data out of SQL Server or setting up a Lookup component to hit the SQL Server instance for each row in the file, how about loading that file into SQL Server and then writing a query to do the work?

In short, the database engine is typically going to be a much better at performance than an integration layer would be.

Comments closed

Choosing the Right Logging Level in SSIS

Published 2025-08-07 by Kevin Feasel

Andy Brownsword does a bit of logging:

When creating SQL Agent jobs to execute SSIS packages we can choose the level of logging to be captured. Different settings are more beneficial under the right circumstances so it’s important to understand the differences to make the right decision.

These settings control the internal logging done by SSIS. This is out of the box and freely available, so why not use it effectively.

The real trick is that if you swallow all of the exceptions and errors, everybody will just assume your code is working perfectly and boom, problem solved. Or you could read Andy’s post and get actual information. Whatever works for you, I suppose.

Comments closed

SSIS Slowdowns in Paging to Disk

Published 2025-07-23 by Kevin Feasel

Andy Brownsword notes a major performance risk in Integration Services:

One particular performance issue with SSIS data flows can fly under the radar – spilling to disk. This isn’t clearly visible through regular debugging or execution so can go unnoticed. And it hurts.

Paging to disk is bad for performance. Disks are much slower to access than memory, so we want to keep our data away when possible.

Andy calls out two reasons why we might find spilling to disk, as well as how to track if this is happening.

Comments closed

Selective Caching in SSIS

Published 2025-07-10 by Kevin Feasel

Andy Brownsword takes us through a pattern:

We’ve recently looked at how caching can improve performance and I wanted to show how we can eek even more performance out of caches by using a custom approach I’ll term Selective Caching.

I’ll note here that there’s a potential gotcha with this approach which we’ll get to before the end of the post!

Click through for a description of the pattern and when it starts to break down.

Comments closed

Using a File Cache in SSIS

Published 2025-07-07 by Kevin Feasel

Andy Brownsword makes those SSIS jobs run faster:

Last week we looked at using a cache to improve lookup performance. We saw how a cache improves performance by being able to reuse reference data repeatedly. That used a regular cache but it’s not the only option available to us.

In this post we’re going to look at the File Cache option which can achieve the same results – plus a little more.

My experience with file caches is that they’re a bit finicky but when you get them running, they can provide a significant speed-up to data enrichment tasks.

Comments closed

Optimizing Multiple Lookup Transformations in SSIS

Published 2025-06-30 by Kevin Feasel

Andy Brownsword doesn’t want to keep hitting the database:

Lookup transformations provide us a way to access related values from another source, such as retrieving surrogate keys in data warehousing. When we need multiple lookups to the same reference data we can improve performance through the use of a Cache.

If we consider data warehousing, a prime example of this would be an order table which has values for Order Date, Dispatch Date, Delivery Date, etc. All of these would require a lookup to a calendar dimension.

This is a perfect use case for a cache.

Read on to see how the cache connector works.

Comments closed