A Fine Slice Of SQL Server
When I added the “*”, this is where I received unexpected results. All I did was add the “*”. Looking at the code below, you can see SQL Server is having issues with the Name column in the ORDER BY.
I do wish SQL had a symbol representing “everything else,” where the engine of choice would include all columns except those explicitly named. I know there’d be trickiness around things like “LTRIM(ColumnA) AS TrimmedColumnA” but that’d be for the language designers to figure out…
Comments closedGilbert Quevauvilliers encounters an error:
I got the following error DM_GWPipeline_Gateway_SpooledOperationMissing”,”parameters”:{},”details”:[],”exceptionCulprit”:1
This error was caused because the current Virtual machine where the On-Premise Data Gateway was running was cloned and then started up.
Read on to learn about the implications and how Gilbert was able to solve this issue.
Comments closedAndy Leonard is planning a jailbreak:
“What if something fails inside the ForEach activity’s inner activities, Andy?”
That is an excellent question! I’m glad you asked. The answer is: The ForEach activity continues iterating.
I can now hear some of you asking…
“What if I want the ForEach activity to fail when an inner activity fails, Andy?”
Another excellent question, and you’ve found a post discussing one way to “break out” of a ForEach activity’s iteration.
Read on for the process.
Comments closedRobin Moffatt explains common errors people run into when trying to connect to remote Kafka clusters:
In this example, my client is running on my laptop, connecting to Kafka running on another machine on my LAN called
asgard03:The initial connection succeeds. But note that the
BrokerMetadatawe get back shows that there is one broker, with a hostname oflocalhost. That means that our client is going to be usinglocalhostto try to connect to a broker when producing and consuming messages. That’s bad news, because on our client machine, there is no Kafka broker atlocalhost(or if there happened to be, some really weird things would probably happen).
As usual, things boil down to “Configure it correctly and it works.”
Comments closedWilliam Assaf diagnoses an issue:
If you’ve tried doing Backup to URL with SQL Server using a Shared Access Signature (SAS) certificate and received this error:
Error: 18204, Severity: 16, State: 1.BackupDiskFile::CreateMedia: Backup device ‘https://account.blob.core.windows.net/container/folder/msdb_log_202005170801.trn’ failed to create. Operating system error 50(The request is not supported.).Cannot open backup device ‘https://account.blob.core.windows.net/container/folder/msdb_log_202005170801.trn’. Operating system error 50(The request is not supported.). [SQLSTATE 42000] (Error 3201) BACKUP LOG is terminating abnormally. [SQLSTATE 42000] (Error 3013). NOTE: The step was retried the requested number of times (1) without succeeding. The step failed.
You may have received the same error I encountered.
This error popped up only after startup of SQL Server. To resolve the problem, we’d recreate the SAS key, using the same cert in the same script, and the backups would start working again. This affected all types of SQL database backups.
William did some troubleshooting and explains the core problem. It’s a weird one.
Comments closedSolomon Rutzky discovered something kind of funny about SQL Server Management Studio:
While researching some odd behavior in SQL Server Management Studio (SSMS) I ran across something even stranger. I was testing the effect of character 0 (sometimes referred to as “NUL”) in the “Messages” tab. For those who have not worked with the C programming language (or similar languages), character 0 (usually indicated by the
\0escape sequence) is the “null terminator” for strings. Meaning, this character marks the end of this string and anything after it should not be displayed.
In Solomon’s usual fashion, you get a thorough testing of the scenario.
Comments closedBob Rudis looks at one of the R 4.0 changes hidden in the changelog:
R 4.0.0 has been out for a while, now, and — apart from a case where
merge()was slower than dirt — it’s been really stable for at least me (I use it daily on macOS, Linux, and Windows). Sure, it came with some headline-grabbing features/upgrades, but I’ve started looking at what other useful nuggets might be in the changelog and decided to blog them as I find them.Today’s nugget is the venerable
stopifnot()function which was significantly enhanced by this PR by Neil Fultz.
Read on for a quality of life improvement with error handling in R.
Comments closedA couple of weeks ago, at my work place, I wrote a metadata-driven data validation framework for Spark. After the initial euphoria of having created the framework in Scala/Spark and Python/Spark, I started reviewing the framework. During the review, I noted that the User Defined Functions (UDF) I had written were prone to throw an error in certain situations.
I then explored various options to make the UDFs fail-safe.
These are like any other code: you want it to be as robust to failure as you can get it (or at least robust enough at the margin).
Comments closedMark Lelijveld has a workaround for a really annoying problem:
I do not think that it is a surprise for anybody that working with Microsoft Teams and different organizations is a big hassle. The multi-tenant setup is not very user friendly yet. On the right top in Teams you can switch organizations, but that only works if you are signing in with the same username and password to another organization. In other words, your account needs to be added as a guest account to the Active Directory in the other organization.
For a lot of us, working as a consultant for different clients, this is not our day-to-day practice. We have a separate Microsoft account for every client, which requires signing out and signing in again. If case you have single sign-on enabled, it can become even a bigger struggle to sign in with a different account than your organization one. As a solution, you might be running Teams in a browser window, which is a first step, but it is still hard to keep up with everything that is happening in Teams.
Well… not for long. With the few little tricks I describe below, you can run multiple Teams environments at the same time by using the new Edge Chromium browser and also customize it so you directly see which Teams environment you opened.
I appreciate that Marc has a very useful workaround. I hate that Marc has to spend the time coming up with a workaround in the first place. I definitely prefer the Slack model, where it doesn’t matter which account I’m using—I see all of my workspaces in one go, regardless of the e-mail address I’ve used.
Comments closedDivyansh Jain shows three techniques for handling invalid input data with Apache Spark:
Most of the time writing ETL jobs becomes very expensive when it comes to handling corrupt records. And in such cases, ETL pipelines need a good solution to handle corrupted records. Because, larger the ETL pipeline is, the more complex it becomes to handle such bad records in between. Corrupt data includes:
– Missing information
– Incomplete information
– Schema mismatch
– Differing formats or data typesSince ETL pipelines are built to be automated, production-oriented solutions must ensure pipelines behave as expected. This means that data engineers must both expect and systematically handle corrupt records.
This is the seedy underbelly of semi-structured data: you don’t have control over the data as it comes in, so you have to control the data coming out.
Comments closed