Press "Enter" to skip to content

Category: KQL

DCountIf in KQL

Robert Cain continues a series on KQL:

In the previous post of this series, Fun With KQL – DCount, we saw how to use the dcount function to get an estimated count of rows for an incoming dataset.

It’s common though to want to filter out certain rows from the count. While you could do the filtering before getting to the dcount, there’s an alternative function that allows you to do the filtering right within it: dcountif.

Read on to learn more about how this function works, as well as several useful examples.

Comments closed

Distinct Counts in KQL

Robert Cain continues a series on KQL:

In an earlier post in this series, Fun With KQL – Count, you saw how to use the count operator to count the number of rows in a dataset.

Then we learned about another operator, distinct, in the post Fun With KQL – Distinct. This showed how to get a list of distinct values from a table.

While we could combine these, it would be logical to have a single command that returns a distinct count in one operation. As you may have guessed by the title of this post, such an operator exists: dcount.

Read on to see how you can use dcount in queries, including how you can perform speed versus accuracy trade-offs.

Comments closed

Multivariate Anomaly Detection with ADX

Adi Eldar shows off multivariate anomaly detection in Azure Data Explorer:

Azure Data Explorer (ADX) is commonly used for monitoring cloud resources and IoT devices performance and health. This is done by continuous collection of multiple metrics emitted by these sources, and on-going analysis of the collected data to detect anomalies. The analysis is applied over time series of the relevant metrics in order to locate significant deviations of the metrics values relative to their typical normal baseline pattern.

Click through for a nice overview of the topic, including two different scenarios: one which emphasizes time series data and the other, which does not.

Comments closed

Using the Kusto Time Pivot Chart

Chango Valtchev reminds us of Gantt charts:

This is the scenario: We have a job scheduler and a related job deployment manager, both implemented based on a state machines framework. One of the scheduler features is preemptable jobs: Jobs of that class can be suspended when a high-priority job needs to be scheduled and there is no available capacity. Effecting preemption requires some involved orchestration between the scheduler and the deployment manager, and we’ve had reliability issues in some cases – both due to incorrectly handled races and latency spikes in the cleanup of the suspended jobs from the cluster. Debugging such issues based on the raw logs has been very tedious – a typical log is 10-30K lines. This gets much worse with the number of dependencies. Given the concurrent processing of the suspensions, tracking the interactions with the new job’s deployment can be mentally taxing. The timeline visualization brought a breakthrough to our debugging ability and productivity. The following sample is a purposefully simplified case. In this scenario, things worked well. It shows the ‘Main’ job, at high priority, waiting on its dependencies to be suspended (while waiting, “Skipped schedule processing” is logged). Shortly after all the suspensions complete, the main job gets to Running state.

Read on to see the scenario in action.

Comments closed

Percentiles in KQL

Robert Cain continues a series on KQL:

Often we want to get data that is relative to other data. For example, we want a list of computers that have free space that is greater than the free space of other computers. We need to set a threshold, for example we want to return results where the free space is greater than 95% of the free space on other computers.

To do this, Kusto provides the percentile operator, along with its variants percentiles and percentiles_array.

Read on to see how it works. I do like the way that KQL handles percentile operations.

Comments closed

Making Sets and Lists with KQL

Robert Cain is making a list and checking it twice:

In previous posts, I’ve mentioned using certain functions and operators to investigate conditions in your system. Naturally you’ll need to create lists of those items, based on certain conditions.

For example, you may want to get a list of the counters associated with an object. Or, you may want to get a list of computer where a certain condition is met.

In this article we’ll see how to get those lists using the Kusto make_set and make_list functions.

Read on to see how these two functions work, as well as their conditional brethren.

Comments closed

Arg_Max and Arg_Min in KQL

Robert Cain continues a series on KQL:

A very common need in query languages is the ability to extract the maximum and minimum values in a column of data. The Kusto Query Language provides this capability through two functions, arg_max and arg_min. In this post we’ll take a look at these functions.

Click through to learn more about how these work.

Comments closed

Working with strcat in KQL

Robert Cain has a post dedicated to the strcat() function in KQL:

The strcat function has been shown in previous articles, but it’s so useful it deserves a post all of its own.

As usual, the samples in this post will be run inside the LogAnalytics demo site found at https://aka.ms/LADemo. This demo site has been provided by Microsoft and can be used to learn the Kusto Query Language at no cost to you.

Read on to (re-)learn the power of string concatenation, in Kusto form.

Comments closed

Splitting Strings with KQL

Robert Cain splits the baby:

In databases, we often find columns that are stored in a hierarchy structure, not unlike a file path on your drive. For example, in the Microsoft Logs sample database the Perf table stores its counter path this way: \\computername\Memory\Available MBytes.

It would be helpful to have a way to easily break this path out into its individual parts. KQL provides us a way of doing this using the split function.

Check out examples of how you can perform splitting.

Comments closed