Press "Enter" to skip to content

Curated SQL Posts

Querying a Microsoft Fabric SQL Endpoint from a Notebook

Published 2025-03-27 by Kevin Feasel

Dennes Torres wants to hit a SQL endpoint:

Let’s analyse why we would like to query an SQL Endpoint. Once we understand why, we can dig into how to make a query to a SQL Endpoint.

We use notebooks to connect directly to lakehouse. Except by the T-SQL notebook, the notebooks have a default lakehouse and work directly with it from spark. However, accessing other data object may be more complex

Specifically, this is a Spark notebook in Microsoft Fabric running Scala rather than a pure Python notebook, and is hitting the data warehouse SQL endpoint.

Leave a Comment

Reporting Query Errors via Extended Events

Published 2025-03-27 by Kevin Feasel

Tom Collins wants to know what’s going wrong:

We have an ETL job failing every night . The ETL job is using SQL Server Stored Procedures ,  so we’re looking to be trace the errors reported in SQL Server, to give some clues on how to fix the problem?

Do you have an approach to monitor and save SQL Server  error messages ?

This is an extremely useful way of catching problems in code, as well as people goofing up ad hoc queries.

Leave a Comment

SQL Server 2019 Mainstream Support Ended

Published 2025-03-27 by Kevin Feasel

Steve Jones points out an important milestone:

Well, not really the end. I doubt anyone running SQL Server 2019 is going to stop (or upgrade) just because mainstream support ended. Actually, I wonder how many of you know that SQL Server 2019 passed out of mainstream support on Feb 28, 2025. I do think the 6 or 7 of you running Big Data Clusters likely knew this was the end of any support.

I saw a report in the Register on this, which includes a survey of which versions are still running. This is from an IT asset firm and matches Brent Ozar’s Population report. 44% of you are running SQL Server 2019, which is the largest percentage. Since there’s an additional 32% of you running versions older than 2019, I’m sure that upgrading isn’t a priority.

Given how there were some own-goals with SQL Server 2022, especially the earlier CUs of it, it’s not very surprising at all that the plurality of customers are still on 2019. Companies tend to follow an “every other release” strategy, so I’d expect some gradual migration to SQL Server 2025 after it comes out and a few CUs are in place.

Leave a Comment

GiST Indexes in PostgreSQL

Published 2025-03-26 by Kevin Feasel

Lee Asher shows off a neat form of indexing:

In Parts I (Solving the Overlap Query Problem in PostgreSQL) and II (Overlapping Ranges in Subsets in PostgreSQL) of this series, we used the GiST index type – and its lesser known cousin: SP-GiST – to turbocharge the performance of overlap queries. But GiST indexes are extremely versatile, with uses far beyond the examples I have used so far.

In this final article, we’ll explore some of the many other ways they can be used.

Click through for use cases, such as multi-dimensional BETWEEN operations (e.g., bid price between X and Y, and ask price between X1 and Y1), date intervals, and more.

Leave a Comment

CIS Benchmarks for PostgreSQL

Published 2025-03-26 by Kevin Feasel

Elizabeth Christensen reviews some security guidance:

The Center for Internet Security (CIS) releases security benchmarks to cover a wide variety of infrastructure used in modern applications, including databases, operating systems, cloud services, containerized services, and even networking. Since 2016 Crunchy Data has collaborated with CIS to provide this security resource for those deploying Postgres. The output of this collaboration is a checklist for folks to follow and improve the security posture of Postgres deployments.

The PostgreSQL CIS Benchmark™ for PostgreSQL 17 was just recently released.

Click through for more information on the CIS, as well as a link to the benchmark PDF and some tips around it.

Leave a Comment

Fabric Espresso Episodes on Data Warehousing and Storage

Published 2025-03-26 by Kevin Feasel

Estera Kot shares some links:

For the past 1.5 years, the Microsoft Fabric Product Group Product Managers have been publishing a YouTube series featuring deep dives into Microsoft Fabric’s features. These episodes cover both technical functionalities and real-world scenarios, providing insights into the product roadmap and the people driving innovation. With over 80+ episodes, the series serves as a valuable resource for anyone looking to understand and optimize their use of Microsoft Fabric.

Click through for a link to the full series, as well as 22 separate episodes covering warehousing and data storage topics.

Leave a Comment

Blocking Transformations in SSIS Data Flows

Published 2025-03-26 by Kevin Feasel

Andy Brownsword doesn’t want any blockers:

We have a platter of transformations to use when constructing data flows in SSIS packages. Not all transformations are equal though, and some can catch us out with performance impact as data volumes scale.

In this post we’ll look at blocking transformations which can trip us up if we’re not careful with them (or avoid them).

Click through for some examples of blocking transformations, as well as some tips on how to manage them.

Leave a Comment

Apache Kafka 4.0 Now Available

Published 2025-03-25 by Kevin Feasel

David Jacot announces a milestone release for Apache Kafka:

Apache Kafka 4.0 is a significant milestone, marking the first major release to operate entirely without Apache ZooKeeper™️. By running in KRaft mode by default, Kafka simplifies deployment and management, eliminating the complexity of maintaining a separate ZooKeeper ensemble. This change significantly reduces operational overhead, enhances scalability, and streamlines administrative tasks. We want to take this as an opportunity to express our gratitude to the ZooKeeper community and say thank you! ZooKeeper was the backbone of Kafka for more than 10 years, and it did serve Kafka very well. Kafka would most likely not be what it is today without it. We don’t take this for granted, and highly appreciate all of the hard work the community invested to build ZooKeeper. Thank you!

There are some other big items in Kafka 4.0 and you can see more in the post’s changelog.

Leave a Comment

Building a Package in R

Published 2025-03-25 by Kevin Feasel

Ken Koon Wong builds a package:

Made a hilariously redundant R package for a simple OpenAI calls, but the real win was finally learning how to build an R package! Is it efficient? Absolutely not! Was it worth the time and experience? Yes! Will I do it again? Yes! Will it break? Yes! 

Package management in R is a bit of a challenge, but well worth it, even if the only consumers of your code will be in your own company. H/T R-Bloggers.

Leave a Comment