Press "Enter" to skip to content

Category: Warehousing

Microsoft Fabric Data Warehouse July 2025 Recap

Published 2025-08-06 by Kevin Feasel

Charles Webb lays out some updates:

Welcome to What’s New in Fabric Warehouse, where we’ll spotlight our work improving quality, delivering major performance enhancements, boosting developer productivity, and our continuous investments in security. Whether you’re migrating from Synapse, optimizing your workloads, writing SQL in VS Code, or exploring new APIs, this roundup has something for every data professional. With quality and experience at the forefront, we’ve summarized and highlighted key improvements we think you’ll love, organized into three sections:

  1. What’s New
  2. Docs Updates
  3. Roadmap Updates

Read on for that update.

Comments closed

Building a Snowflake Dashboard that Uses Filters

Published 2025-07-30 by Kevin Feasel

Kevin Wilkie does a bit of filtering:

Snowflake Dashboards can do a lot more than just show pretty numbers. Today, let’s focus on something that every data pro eventually has to deal with—filters that make navigating your dashboards less painful, especially when it comes to everyone’s favorite task: AUDITING.

Ah yes, auditing—because nothing says “data dream job” like tracing permissions. Whether it’s quarterly compliance checks or a sudden request from an overly curious auditor, somebody, at some point, will ask, “Who has access to what in Snowflake?” So let’s make that answer easy to deliver.

Click through for the process, using the development of a permissions auditing dashboard as the example.

Comments closed

Result Set Chaining in Snowflake

Published 2025-07-17 by Kevin Feasel

Kevin Wilkie tries out a new operator:

In a recent Snowflake release, a slick new operator quietly entered the scene: ->>. This little guy can make certain query workflows both more readable and more efficient—especially when you’re dealing with multi-step commands like SHOW, LIST, or DESCRIBE.

Click through to see how it works. Seems that this operator has some pretty strict limitations, but for certain use cases, it’s quite nice.

Comments closed

Writing Back to a Fabric Data Warehouse via UDF

Published 2025-07-08 by Kevin Feasel

Jon Vöge continues a series on write-back options into Microsoft Fabric:

In that article, we took advantage of some of the built-in sample code from the User Data Function editor, as well as some great code examples from Sujata: Example User data functions for Translytical task flows · GitHub

The problem? All of these samples use SQL Databases in Fabric as the backend item.

Jon switches this from a SQL database into a Fabric Data Warehouse, and notes some of the challenges along the way.

Comments closed

Loading Data into Snowflake via Python

Published 2025-06-13 by Kevin Feasel

Anil Kumar Moka does a bit of data loading:

In our ongoing exploration of Snowflake data loading strategies, we’ve previously examined how to use pandas with SQLAlchemy to efficiently move data into Snowflake tables. That approach leverages pandas’ intuitive DataFrame handling and works well for many common scenarios where you’re already manipulating data in Python before loading it to Snowflake.

In this article, we’re diving deeper into the Snowflake toolbox by exploring the native Snowflake Connector for Python. While pandas offers simplicity and familiarity, the native connector provides a different set of capabilities focused on precision control and Snowflake-specific optimizations. This article explains you when and how to use this more direct approach for everything from small CSV files to massive datasets that would overwhelm pandas.

Click through for the full article.

Comments closed

Optimizing a Snowflake Data Warehouse

Published 2025-06-12 by Kevin Feasel

Harshavardhan Yedla gives us some guidance:

Optimizing a Snowflake data warehouse (DWH) is crucial for ensuring high performance, cost-efficiency, and long-term effectiveness in data processing and analytics. The following outlines the key reasons optimization is essential:

Read on for some tips around optimizing Snowflake warehouses. A lot of this stays at a pretty high level and doesn’t provide detailed guidance, but it’s a good checklist for thinking about your own situation.

Comments closed

Analyzing Snowflake Costs

Published 2025-05-21 by Kevin Feasel

Kevin Wilkie watches a moth fly out of his wallet and wonders where all of the money went:

Last time, in Dashboard Dreams and Snowflake Schemes, we talked a little about showing how much Snowflake really costs in a dashboard internal to Snowflake itself instead of having to push it to PowerBi, Tableau, Looker, or a myriad of other tools.

This time, let’s take it a step further: instead of sticking with the basic bar charts or exploding pie charts, we’ll explore how to better highlight usage trends by adding a Rolling 7-Day Average to our visualizations. This helps us more easily spot patterns and anomalies within our warehouses.

Read on for a pair of queries and a neat chart.

Comments closed

Real-Time Data Streaming in Snowflake

Published 2025-05-09 by Kevin Feasel

Anil Kumar Moka streams some data:

Real-time data ingestion has become essential for modern analytics and operational intelligence. Organizations across industries need to process data streams from IoT sensors, financial transactions, and application events with minimal latency. Snowflake offers two robust approaches to meet these real-time data needs: Snowpipe for near-real-time file-based streaming and Direct Streaming via Snowpark API for true real-time data integration.

This guide explores both options in depth, providing detailed implementations with explanation of code parameters, performance comparisons, and practical recommendations to help you choose the right approach for your specific use case.

Click through to see how it works. I’ll only make one semi-snarky comment that ‘real-time’ doesn’t mean “takes several seconds” but I realize I’m the one tilting at windmills here.

Comments closed