A Fine Slice Of SQL Server
Gilbert Quevauvilliers encounters an error:
I got the following error DM_GWPipeline_Gateway_SpooledOperationMissing”,”parameters”:{},”details”:[],”exceptionCulprit”:1
This error was caused because the current Virtual machine where the On-Premise Data Gateway was running was cloned and then started up.
Read on to learn about the implications and how Gilbert was able to solve this issue.
Leave a CommentAndy Leonard is planning a jailbreak:
“What if something fails inside the ForEach activity’s inner activities, Andy?”
That is an excellent question! I’m glad you asked. The answer is: The ForEach activity continues iterating.
I can now hear some of you asking…
“What if I want the ForEach activity to fail when an inner activity fails, Andy?”
Another excellent question, and you’ve found a post discussing one way to “break out” of a ForEach activity’s iteration.
Read on for the process.
Leave a CommentRobin Moffatt explains common errors people run into when trying to connect to remote Kafka clusters:
In this example, my client is running on my laptop, connecting to Kafka running on another machine on my LAN called
asgard03:The initial connection succeeds. But note that the
BrokerMetadatawe get back shows that there is one broker, with a hostname oflocalhost. That means that our client is going to be usinglocalhostto try to connect to a broker when producing and consuming messages. That’s bad news, because on our client machine, there is no Kafka broker atlocalhost(or if there happened to be, some really weird things would probably happen).
As usual, things boil down to “Configure it correctly and it works.”
Comments closedWilliam Assaf diagnoses an issue:
If you’ve tried doing Backup to URL with SQL Server using a Shared Access Signature (SAS) certificate and received this error:
Error: 18204, Severity: 16, State: 1.BackupDiskFile::CreateMedia: Backup device ‘https://account.blob.core.windows.net/container/folder/msdb_log_202005170801.trn’ failed to create. Operating system error 50(The request is not supported.).Cannot open backup device ‘https://account.blob.core.windows.net/container/folder/msdb_log_202005170801.trn’. Operating system error 50(The request is not supported.). [SQLSTATE 42000] (Error 3201) BACKUP LOG is terminating abnormally. [SQLSTATE 42000] (Error 3013). NOTE: The step was retried the requested number of times (1) without succeeding. The step failed.
You may have received the same error I encountered.
This error popped up only after startup of SQL Server. To resolve the problem, we’d recreate the SAS key, using the same cert in the same script, and the backups would start working again. This affected all types of SQL database backups.
William did some troubleshooting and explains the core problem. It’s a weird one.
Comments closedSolomon Rutzky discovered something kind of funny about SQL Server Management Studio:
While researching some odd behavior in SQL Server Management Studio (SSMS) I ran across something even stranger. I was testing the effect of character 0 (sometimes referred to as “NUL”) in the “Messages” tab. For those who have not worked with the C programming language (or similar languages), character 0 (usually indicated by the
\0escape sequence) is the “null terminator” for strings. Meaning, this character marks the end of this string and anything after it should not be displayed.
In Solomon’s usual fashion, you get a thorough testing of the scenario.
Comments closedBob Rudis looks at one of the R 4.0 changes hidden in the changelog:
R 4.0.0 has been out for a while, now, and — apart from a case where
merge()was slower than dirt — it’s been really stable for at least me (I use it daily on macOS, Linux, and Windows). Sure, it came with some headline-grabbing features/upgrades, but I’ve started looking at what other useful nuggets might be in the changelog and decided to blog them as I find them.Today’s nugget is the venerable
stopifnot()function which was significantly enhanced by this PR by Neil Fultz.
Read on for a quality of life improvement with error handling in R.
Comments closedA couple of weeks ago, at my work place, I wrote a metadata-driven data validation framework for Spark. After the initial euphoria of having created the framework in Scala/Spark and Python/Spark, I started reviewing the framework. During the review, I noted that the User Defined Functions (UDF) I had written were prone to throw an error in certain situations.
I then explored various options to make the UDFs fail-safe.
These are like any other code: you want it to be as robust to failure as you can get it (or at least robust enough at the margin).
Comments closedMark Lelijveld has a workaround for a really annoying problem:
I do not think that it is a surprise for anybody that working with Microsoft Teams and different organizations is a big hassle. The multi-tenant setup is not very user friendly yet. On the right top in Teams you can switch organizations, but that only works if you are signing in with the same username and password to another organization. In other words, your account needs to be added as a guest account to the Active Directory in the other organization.
For a lot of us, working as a consultant for different clients, this is not our day-to-day practice. We have a separate Microsoft account for every client, which requires signing out and signing in again. If case you have single sign-on enabled, it can become even a bigger struggle to sign in with a different account than your organization one. As a solution, you might be running Teams in a browser window, which is a first step, but it is still hard to keep up with everything that is happening in Teams.
Well… not for long. With the few little tricks I describe below, you can run multiple Teams environments at the same time by using the new Edge Chromium browser and also customize it so you directly see which Teams environment you opened.
I appreciate that Marc has a very useful workaround. I hate that Marc has to spend the time coming up with a workaround in the first place. I definitely prefer the Slack model, where it doesn’t matter which account I’m using—I see all of my workspaces in one go, regardless of the e-mail address I’ve used.
Comments closedDivyansh Jain shows three techniques for handling invalid input data with Apache Spark:
Most of the time writing ETL jobs becomes very expensive when it comes to handling corrupt records. And in such cases, ETL pipelines need a good solution to handle corrupted records. Because, larger the ETL pipeline is, the more complex it becomes to handle such bad records in between. Corrupt data includes:
– Missing information
– Incomplete information
– Schema mismatch
– Differing formats or data typesSince ETL pipelines are built to be automated, production-oriented solutions must ensure pipelines behave as expected. This means that data engineers must both expect and systematically handle corrupt records.
This is the seedy underbelly of semi-structured data: you don’t have control over the data as it comes in, so you have to control the data coming out.
Comments closedAaron Bertrand takes us through a problem with seemingly indeterminate query errors:
I’ve talked about illogical errors before. In several answers on Database Administrators (one, two, three), I show how you can use a
CASEexpression orTRY_CONVERTto work around an error where a non-numeric value, that should have been filtered out by a join or other clause, still leads to a conversion error. Erland Sommarskog raised a Connect item over a decade ago, still unaddressed, called “SQL Server should not raise illogical errors.”Recently we had a scenario where a query was failing on one server but not another. But this was slightly different; there were no numerics involved. Imagine this scenario: a source table has a column that is
varchar(20). A query creates a table variable with a column that isvarchar(10), and inserts rows from the source table, with a filter in place that only exposes values that are 10 characters or less.In a lot of cases, of course, this scenario is perfectly fine, and everything works as expected.
Read the whole thing. There is a method to the madness, and Aaron explains how it can come up in some cases but not others.
Comments closed