A Fine Slice Of SQL Server
Arthur Breant runs some tests:
You’ve created a fantastic mockup and your client is delighted. You’re ready to move to production with your application. But one question haunts you: how can you ensure that your application will remain stable and functional through modifications and evolutions?
The answer comes down to one word: testing.
Read on to learn how you can perform unit testing, integration testing, and end-to-end testing of Shiny applications in R. H/T R-Bloggers.
Leave a CommentStephen Planck does some testing:
Regularly restoring test copies of your databases is the gold-standard proof that your backups work. Between those tests, however, RESTORE VERIFYONLY offers a fast way to confirm that a backup file is readable, that its page checksums are valid, and that the media set is complete. In this post you will see how to run that command from PowerShell by invoking SQL Server Management Objects (SMO), turning a one-off verification into a repeatable step you can schedule across all your servers.
Click through for the script and explanation. I also like dbatools’ Test-DbaLastBackup command, as that can also run RESTORE VERIFYONLY but goes further and allows you to restore the backup and then run DBCC CHECKDB against its contents.
Brandur Leach lays out the argument:
The textbook example of this is the database mock. Here’s a rough articulation of the bull case for this idea: CPUs are fast. Memory is fast. Disks are slow. Why should tests have to store data to a full relational database with all its associated bookkeeping when that could be swapped out for an ultra-fast, in-memory key/value store? Think of all the time that could be saved by skipping that pesky
fsync, not having to update that plethora of indexes, and foregoing all that expensive WAL accounting. Database operations measured in hundreds of microseconds or even *gasp*, milliseconds, could plausibly be knocked down to 10s of microseconds instead.
Prior to reading the article, my stance was as follows: use database mocks for unit test libraries, in which you aren’t testing the actual data processing or retrieval. Those should be able to run on an isolated build server with no access to a database. But you also need proper integration tests that cover how you interact with the database, and those tests should be a majority of your test suite. You should have a known state database before each test run (which is where Docker containers or database snapshots become extremely helpful), and passing the database tests should be a gate early on in the CI/CD process.
After reading the article, my priors remain the same. I think there’s still scope for database mocks, but not as a replacement for proper integration testing with the database.
Leave a CommentAida Gjoka and Russ Hyde show off some capabilities in the pytest library:
Writing tests is one of the best ways to keep your code reliable and reproducible. This post builds on our previous blog about Python testing with pytest Part 1, and explores some of the more advanced features it offers. From parametrised fixtures to mocking and other useful pytest plugins, we will show how to make your tests more reproducible, easier to manage and demonstrate how writing simple tests can save you time in the long run.
Click through to learn more. I’m a huge fan of parameterization in pytest—it’s really easy to do. Mocks are a bit harder to pull off in practice, though quite useful.
Comments closedSoftware testing plays a crucial role in ensuring the delivery of high-quality products. However, even experienced testers can fall into common traps that compromise the effectiveness of testing processes, allowing defects to slip into production. Avoiding these pitfalls is essential for maintaining the reliability and functionality of software. In this blog, we’ll explore some of the most common software testing mistakes and provide strategies to overcome them.
This is specifically for software testing, but most of these principles apply the same for database work.
Comments closedSairamakrishna BuchiReddy Karri and Srinivasarao Rayankula show off Great Expectations:
Data quality checks are critical for any production pipeline. While there are many ways to implement them, the Great Expectations library is a popular one.
Great Expectations is a powerful tool for maintaining data quality by defining, managing, and validating expectations for your data. In this article, we will discuss how you can use it to ensure data quality in your data pipelines.
Click through to see how it all works.
Comments closedAdron Hall has a new tool and a new blog series. The first post is a product overview:
DataDiluvium is a web-based tool available at datadiluvium.com that helps developers, database administrators, and data engineers generate realistic test data from SQL schema definitions. Whether you’re setting up a development environment, creating test scenarios, or preparing data for demonstrations, DataDiluvium streamlines the process of data generation.
The second covers some of the development precepts Adron used:
DataDiluvium is a web-based tool I’ve built designed to help developers, database administrators, and data engineers generate realistic test data based on SQL schema definitions. The tool takes SQL table definitions as input and produces sample data in various formats, making it easier to populate development and testing environments with meaningful data.
The tool is free, so if you’re looking for a sample data generator, check it out.
Comments closedToday, my colleague wanted to quickly test out some dbatools commands to install the Ola Hallengren maintenance solution. They had a local instance of SQL installed, but it already had the maintenance jobs running, so it wasn’t a fresh, out of the box instance.
So let’s spin a SQL Server instance in seconds to test against! (Ok it’s seconds if you have the pre-requisites installed, but I’ll get you setup in a few minutes if not)!
Click through for a primer on using SQL Server in a container.
Comments closedTien Nguyen Anh shares some advice:
A flaky test is a test that has inconsistent results in multiple test run, even when the automation script has not been changed. Sometimes it passes, and sometimes it fails, which makes it unreliable and difficult to trust for identifying issues.
Read on to understand a variety of causes and what you can do about them.
Comments closedJasleen Kaur Wahi generates some data:
In a recent project, I faced the need to generate randomized data for transmission to the Azure Event Hub. This hub is a key component of Microsoft Azure, used for real-time data ingestion and processing.
First, let’s take look at how I created this random data. I wanted to come up with a way to make data that looks like what we see in the real world, but without using any real information from users. This made-up data was really important for a bunch of things, like checking if our software works well.
Read on to see how Mockaroo works and the end result. Creating tests for streaming services like Event Hubs is a challenge, so this is an interesting approach to the task.
Comments closed