Unit Testing ETL Processes

Ed Elliott has a post showing how to perform unit testing on ETL processes:

In tSQLt, we can call tSQLt.FakeTable and then do an insert, if we don’t use tSQLt what do we do? Well, we need to setup the data we want, this could be by using a tool or by writing a load of insert statements. I have seen this done in various ways such as:

  1. Writing manual insert scripts
  2. Using a tool to setup the data
  3. Making use of API’s in the application to setup the data we need
  4. Some wierd and wonderful things that we shouldn’t recommend

Ultimately, for each test that you do you need to know what data you need for it. There isn’t really any way around this, and the sooner you get yourself in the position where you can setup the data you need for a test, the better.

Read the whole thing.

Related Posts

An Overview of Regression Testing

Ust Oldfield gives us a primer on regression testing: There are a variety of methods and techniques that can be used in the design and execution of regression tests. These are:– Retest All– Test Selection– Test Case Prioritisation Regression testing is really nice to have in place because it keeps you from looking like a […]

Read More

Generating Fake Data

Rich Benner shows us how to use the Faker library in Python to generate test data: There are far more options when using Faker. Looking at the official documentation you’ll see the list of different data types you can generate as well as options such as region specific data. Go have fun trying this, it’s a small […]

Read More

Categories

May 2018
MTWTFSS
« Apr Jun »
 123456
78910111213
14151617181920
21222324252627
28293031