Press "Enter" to skip to content

Month: August 2025

Tips for Working with Pandas

Published 2025-08-22 by Kevin Feasel

Matthew Mayo has a few tips when working with Pandas for data preparation:

If you’re reading this, it’s likely that you are already aware that the performance of a machine learning model is not just a function of the chosen algorithm. It is also highly influenced by the quality and representation of the data that said model has been trained on.

Data preprocessing and feature engineering are some of the most important steps in your machine learning workflow. In the Python ecosystem, Pandas is the go-to library for these types of data manipulation tasks, something you also likely know. Mastering a few select Pandas data transformation techniques can significantly streamline your workflow, make your code cleaner and more efficient, and ultimately lead to better performing models.

This tutorial will walk you through seven practical Pandas scenarios and the tricks that can enhance your data preparation and feature engineering process, setting you up for success in your next machine learning project.

Click through for those tips and tricks.

Leave a Comment

Handling Missing Data in R

Published 2025-08-22 by Kevin Feasel

M. Fatih Tüzen fills in the gaps:

Data preprocessing is a cornerstone of any data analysis or machine learning pipeline. Raw data rarely comes in a form ready for direct analysis — it often requires cleaning, transformation, normalization, and careful handling of anomalies. Among these preprocessing tasks, dealing with missing data stands out as one of the most critical and unavoidable challenges.

Missing values appear in virtually every domain: surveys may have skipped questions, administrative registers might contain incomplete records, and clinical trials can suffer from dropout patients. Ignoring these gaps or handling them naively does not just reduce the amount of usable information; it can also introduce bias, decrease statistical power, and ultimately compromise the validity of conclusions. In other words, missing data is not just an inconvenience — it is a methodological problem that demands rigorous attention.

Quite often, we gloss over what to do with missing data when explaining or working through the data science process, in part because it’s a hard problem. This post digs into the specifics of the matter, taking us through eight separate methods. H/T R-Bloggers.

Leave a Comment

Lessons Learned on Migrating to Apache Kafka

Published 2025-08-22 by Kevin Feasel

Ravi Teja Thutari shares some advice:

The legacy e-commerce platform was a PHP-based monolith handling catalog, orders, inventory, and customer data. With business growth, the monolith could not scale further.  Maintaining feature velocity was hard because every change risked the entire system. We needed scalability, resilience, and faster releases.  Shifting to event-driven microservices promised to address these issues.  In practice we adopted Kafka on Kubernetes, similar to other online retailers .

Our priorities were (1) decoupling services so each team could deploy independently, (2) modeling business events consistently across domains, and (3) ensuring reliable delivery at scale (with retries and DLQs for failures).  As a starting point, we documented key domain events (e.g. OrderCreated, PaymentProcessed, InventoryAllocated) and sketched a target architecture.  Like other high-traffic systems, we planned horizontal scaling: adding Kafka brokers and topic partitions to match consumer parallelism. We also planned for observability from Day 1 (metrics, logs, traces) to monitor performance and troubleshoot issues.

Read on for more information about how that migration went.

Leave a Comment

Configuring Alerts in Azure SQL Managed Instance

Published 2025-08-22 by Kevin Feasel

Aleksey Vitsko wants an alert:

You have an Azure SQL Managed Instance and you want to set up SQL Server alerts for errors with severity 17-25, similar what you would do for an on-prem SQL Server. You go to the SQL Server Agent folder in Object Explorer, expand it, and whoops – there is no Alerts folder.

As of time of writing this article (June 2025), Azure SQL Managed Instance doesn’t have this functionality, and we don’t have any ETA on when it will be implemented. So, how can we setup alerts in Azure SQL MI to notify us when there are issues?

Read on for a workaround and a warning.

Leave a Comment

Pattern Matching with REGEXP_LIKE() in SQL Server 2025

Published 2025-08-22 by Kevin Feasel

Koen Verbeeck writes a regular expression:

I need to do some data validation in our SQL Server database. However, the validation rules are too complex for the T-SQL LIKE function, and I can’t seem to get it done either with PATINDEX or something similar. I’d like to use regular expressions as they’re more powerful. SQL Server 2025 now has a regex function regexep_like to use regular expressions.

Read on for some examples, advice on validating e-mail addresses, and more.

Leave a Comment

Item History in the Microsoft Fabric Capacity Metrics App

Published 2025-08-22 by Kevin Feasel

Ope Aladekomo announces a new feature:

We’re thrilled to announce the Preview of the Item History page in the latest version of the Microsoft Fabric Capacity Metrics App. The Item History page provides a 30-day compute usage analysis through dynamic visuals and slicers, enabling users to explore both high-level consumption trends and granular item-level metrics. This page helps you understand how individual items and operations contribute to overall capacity usage.

Click through to see a picture of the page, as well as some of the information you can glean from it.

Leave a Comment

Sorting a Visual by a Field Not on the Visual

Published 2025-08-22 by Kevin Feasel

Nikola Ilic does a bit of sorting:

Recently, I was dealing with a Power BI report where the client had a very specific requirement – to sort the data in the visual based on a particular field from the semantic model. The only “issue” was that this particular field wasn’t part of the visual. So, while figuring out how this can be accomplished (because, yes, everything can be accomplished when the client needs it, hehe), I decided to write it down and share it with everyone who might find it useful.

Nikola successfully uses machine trickery to solve the problem.

Leave a Comment

Diagnosing Classification Model Failures

Published 2025-08-21 by Kevin Feasel

Ivan Palomares Carrascosa looks into a poorly-fitting model:

In classification models, failure occurs when the model assigns the wrong class to a new data observation; that is, when its classification accuracy is not high enough over a certain number of predictions. It also manifests when a trained classifier fails to generalize well to new data that differs from the examples it was trained on. While model failure typically presents itself in several forms, including the aforementioned ones, the root causes can sometimes be more diverse and subtle.

This article explores some common reasons why classification models may underperform and outlines how to detect, diagnose, and mitigate these issues.

The explanations are fairly high-level and focus mostly on two-class rather than multi-class classification, but there is some good guidance in here.

Leave a Comment

Auditing Specific Data Access in SQL Server

Published 2025-08-21 by Kevin Feasel

Andreas Wolter wants to focus in on specific database objects:

In this article I want to share a targeted approach to audit access to specific objects within a database in Microsoft SQL Server.

  • In my last article, Evading Data Access Auditing in Microsoft SQL Server – and how to close the gaps, I showed multiple approaches to gain access to a chunk of sensitive data using the statistics object in SQL Server. The hardest one to capture is access to data that is exposed via the dynamic management function (DMF) dm_db_stats_histogram. This requires an additional Audit Specification in the master database for this system object. In the end we required 3 different Audit Action Groups to cover all the methods used to read data from our example table.

Read on to see what you can do as of SQL Server 2022.

Leave a Comment

Multi- and Single-Line Regular Expression Processing in SQL Server

Published 2025-08-21 by Kevin Feasel

Louis Davidson continues a series on regular expressions in SQL Server:

There are currently only 4 flags that SQL Server supports and they are used to change some of the fundamental ways that the expressions are applied. These flags are:

i – insensitive
c – case sensitive
m – ^ and $ match end of line, not entire string
s – single line, dot matches newline

In Part 6, I covered i and c, no[w] let’s do m and s. These flags are not ones I expect to use all that often, but they are definitely useful to know.

Read on to see how they work, as well as some of the issues Louis ran into along the way.

Leave a Comment