A Fine Slice Of SQL Server
Chris Webb draws a connection:
Since I wrote about the Snowflake query tags generated by Power BI earlier this year, one important piece of functionality has been added: an OperationID that allows you to link a SQL query in Snowflake to events in Workspace Monitoring and the Capacity Metrics app. Let’s see some examples.
Click through for those examples.
Leave a CommentAbhishek Narain has a list of updates:
Workspace Private Link Support for Data Factory (Preview): Microsoft Fabric enables secure data integration through Private Link support in Dataflows Gen2, Pipelines, and Copy jobs. This ensures that inbound data access remains isolated and compliant within protected workspaces. By leveraging VNet data gateways, organizations can securely connect to data sources across Private Link-enabled environments—eliminating exposure to public networks and reinforcing enterprise-grade security for sensitive data operations.
Most of these are security-related updates, with a mixture of things now GA, things currently in preview, and a pair of items coming soon.
Leave a CommentRam Ghadiyaram adds some error handling logic:
In PySpark, processing massive datasets across distributed clusters is powerful but comes with challenges. A single bad record, missing file, or network glitch can crash an entire job, wasting compute resources and leaving you with stack traces that have many lines.
Spark’s lazy evaluation, where transformations don’t execute until an action is triggered, makes errors harder to catch early, and debugging them can feel like very, very difficult.
Read on for five patterns that can help with error handling in PySpark.
Leave a CommentJared Westover shares some findings:
In Part One, we saw that simple scalar user-defined functions (UDFs) perform as well as inline code in a Fabric warehouse. But with a more complex UDF, does performance change? If it drops, is the code-reuse convenience worth the price?
I’m surprised that the performance profile was so good. I had assumed it would perform like T-SQL user-defined functions—namely, worse in general.
Leave a CommentLaurenz Albe doesn’t have MySQL envy:
If you are reading this hoping that PostgreSQL finally got
UPDATE ... LIMITlike MySQL, I have to disappoint you. TheLIMITclause is not yet supported for DML statements in PostgreSQL. If you want toUPDATEonly a limited number of rows, you have to use workarounds. This article will describe how to do this and how to avoid the pitfalls and race condition you may encounter. Note that most of the following also applies toDELETE ... LIMIT!
Click through for what you can do in PostgreSQL instead. In T-SQL, we can use UPDATE TOP(n).
Leave a CommentHaripriya Naidu digs into a few scenarios:
When RCSI is enabled, I’d like to discuss three scenarios:
- UPDATE is in progress, and SELECT starts to run.
Where does SELECT read from?- SELECT runs, and there are no concurrent operations or uncommitted transactions. Where does SELECT read from?
- SELECT is running, and now an UPDATE starts to run concurrently. What happens to SELECT that is in progress? What about the UPDATE that started? Does it wait for SELECT to finish?
Click through to see what happens during each of these scenarios.
Leave a CommentDavid Fowler announces a tool update:
Presenting you with an updated version of our sp_snapshot procedure, allowing you to easily create database snapshots.
This new version fixes a bug that we’ve found in version 2 where snapshots will fail for databases with multiple data files.
We’ve also added the @STMTOnly parameter, allowing you to generate the scripts for creating the required snapshots without actually doing so.
Click through for more information, as well as where you can go to download the script.
Leave a CommentHarris Amjad explains the basics of principal component analysis:
In this series of tips, we will delve into the unsupervised learning branch of Machine Learning. Principal Component Analysis (PCA) is a powerful technique for dimensionality reduction, but its mathematical foundation involving eigenvalues and eigenvectors can be intimidating. This tip aims to demystify PCA, explaining its purpose, how it works, and its use in visualizing high-dimensional data.
Click through to learn how it works. This is a solid primer.
Leave a CommentBala Pirya C performs a comparison:
In this article, you will learn how MinMaxScaler, StandardScaler, and RobustScaler transform skewed, outlier-heavy data, and how to pick the right one for your modeling pipeline.
Topics we will cover include:
- How each scaler works and where it breaks on skewed or outlier-rich data
- A realistic synthetic dataset to stress-test the scalers
- A practical, code-ready heuristic for choosing a scaler
Read on to learn more about each of these three scaler types, the use cases that best fit each of them, and even a flow chart at the end.
Leave a CommentYou’re building a new table or adding a column, and you wanna know which datatype to use: VARCHAR or NVARCHAR?
If you need to store Unicode data, the choice is made for you: NVARCHAR says it’s gonna be me.
But if you’re not sure, maybe you think, “I should use VARCHAR because it takes half the storage space.” I know I certainly felt that way, but a ton of commenters called me out on it when I posted an Office Hours answer about how I default to VARCHAR. One developer after another told me I was wrong, and that in 2025, it’s time to default to NVARCHAR instead. Let’s run an experiment!
This is going back a long way (June of 2020) but one of my earliest YouTube videos is entitled NVARCHAR Everywhere. I’ve gotten a lot better at presentation skill since then (and have a much nicer camera), but I still stand by the arguments.
Leave a Comment