Press "Enter" to skip to content

Curated SQL Posts

Delta Tables in Microsoft Fabric with Polars

Published 2024-12-04 by Kevin Feasel

Sandeep Pawar tries out the Polars library:

The much-anticipated Python notebook in Fabric is finally available and the Fabric users have already developed cool libraries and blogged about the usefulness of these notebooks. Duckdb is everyone’s favorite, but I am a Python guy so here is quick overview of how you can use Polars in the Python notebook.

Polars is an open-source library that uses a Rust engine and supports multi-threaded execution. This means it’s significantly faster than pandas and, in some cases, even faster than Spark. It can efficiently use the limited resources available in Python notebooks (2 cores, 16GB RAM). Polars v1.6 is installed in the default Python notebook environment. So, let’s see how to perform some common operations.

Read on to see how you can load and write out files via Polars.

Comments closed

The Cost of Free Testing Tools

Published 2024-12-04 by Kevin Feasel

Lauren Harold reminds us that time has value and opportunity cost exists:

Open source frameworks like Selenium, Cypress, and Playwright are the first stop for most startups when they automate their end-to-end tests. This process generally happens early, according to our research, with 69% of software teams with just 6-10 developers automating their testing.

But make no mistake. We recently surveyed developers and engineering leaders from around the globe to understand how current practices impact software velocity. One thing that immediately became apparent was that open source testing frameworks are “free,” but their costs snowball once you start adding testing infrastructure, plugins, and test management software.

The proposed solution in the article is no-code end-to-end testing, though that often ends up being brittle like other end-to-end testing solutions, and a huge challenge for complicated flows.

Comments closed

Importing PDF Contents into SQL Server

Published 2024-12-04 by Kevin Feasel

Sebastiao Pereira loads a PDF:

PDF forms are widely used for data collection, document processing, and digital workflows due to their versatility and consistency across different platforms and devices. They are essential in various industries, including healthcare, education, finance, government, and business. How do you retrieve data from PDF forms and insert into a SQL Server database table?

Read on for an answer using Visual Basic, which is a name I haven’t heard in quite some time.

Comments closed

Interpolating Missing Values in R

Published 2024-12-03 by Kevin Feasel

Steven Sanderson fills in the blanks:

Interpolation is a method of estimating missing values based on the surrounding known values. It’s particularly useful when dealing with time series data or any dataset where the missing values are not randomly distributed.

There are various interpolation methods, but we’ll focus on linear interpolation in this article. Linear interpolation assumes a straight line between two known points and estimates the missing values along that line.

Read on to see how you can perform linear interpolation in R.

Comments closed

Features in Azure AI Foundry

Published 2024-12-03 by Kevin Feasel

Tomaz Kastrun continues a series:

Azure AI Foundry is all purpose tool that provides all of the capital ingredients that data scientists would need in order to create, develop and deploy the generative AI applications. The platfrom supports and gets you the following services and abilitiies:

Click through for those features and how you can access the Azure AI Foundry.

Comments closed

Blank Dates and DAX

Published 2024-12-03 by Kevin Feasel

Marco Russo and Alberto Ferrari are blanking on us:

Handling missing dates in a semantic model can be challenging, especially when working with DAX time intelligence functions. Dates might be missing for various reasons: incomplete data entry, system errors, special placeholder values like 0000, or dates set far in the future. We will see that using a blank is the best way to manage missing dates, even though you should pay attention to DAX conditional expressions operating on those dates. We will also consider how to hide these blanks in a Power BI report if their presence is not desired in charts and slicers.

Read on to learn more.

Comments closed

Optimizing AWS Costs

Published 2024-12-03 by Kevin Feasel

Albert McQuiston speaks my language (that is, saving money):

Every organization looks to save on its cloud expenses to align with business objectives. With the following tips, you can optimize your Amazon Web Services (AWS) cloud expenditure and review the key aspects where you can save more effectively.

Read on for some high-level tips. It doesn’t cover things like spot instances, but does a pretty decent job of laying out the problem and showing some of the cost and budgeting tools available to figure out where your company’s money is going.

Comments closed

GiST Indexes and Range Queries in PostgreSQL

Published 2024-12-03 by Kevin Feasel

Lee Asher can’t be limited to a single point:

Our Part I query used the following WHERE clause:

WHERE tsrange(o.start_time, o.end_time) && tsrange(p.enter, p.leave)

The “tsrange()” functions return timestamp ranges. But overlap queries aren’t limited to timestamps; they can be constructed from integers and floating-point values too. Imagine an arbitrage database that tracks the minimum and maximum price paid for a commodity.

Read on for examples of other types of ranges, preventing range intersection, and more.

Comments closed

External Data Sharing in OneLake

Published 2024-12-03 by Kevin Feasel

Jens Vestergaard shares some info about sharing some info:

At #MSIgnite Microsoft announced a new feature in Fabric that allows people from one organization to share data with people from another organization. You might ask yourself why is this even news, and rightly so. Up until last week, professionals have had to use tools like (S)FTP clients like FileZilla, Azure Storage Explorer, WeTransfer or similar products in order to share data. Some of these tools are in fact hard to use and/or understand for a great number of business users – they are familiar with Windows and the Office suite and not much more. This is all to be expected, as business users in general should focus on business stuff rather than IT stuff.

Read on to see how this has changed, and an update to what I consider one of the coolest products to come out of Microsoft Fabric.

Comments closed

A Review of the Azure AI Foundry

Published 2024-12-02 by Kevin Feasel

Tomaz Kastrun starts a new series:

Microsoft Azure offers multiple services that enable developers to build amazing AI-powered solutions. Azure AI Foundry brings these services together in a single unified experience for AI development on the Azure cloud platform.

Until now, developers needed to work with multiple tools and web portals in a single project. With Azure AI Foundry, these tasks are now simplified and offers same environment for better collaboration.

Read on to see more about the Azure AI Foundry.

Comments closed