Press "Enter" to skip to content

Category: Misc Languages

An Overview of Spark in Microsoft Fabric

Published 2024-06-20 by Kevin Feasel

Reza Rad gives people a primer on Apache Spark:

Microsoft Fabric runs some workloads under the Spark engine, but what is it really? In this article, I’ll take you through the question of what Spark is, What benefits it has, how it is associated with Fabric, what configurations you have, and other things you need to know about it.

Reza talks a bit about history, interaction with languages, etc. As a quick addition to the languages list, you can use .NET languages like F# and C# with Spark, though it does involve setting up dotnet/spark and there are some open questions about its future. And I’m not even sure you could get it to work with Microsoft Fabric.

Comments closed

Converting Numbers into Roman Numerals with C#

Published 2024-06-11 by Kevin Feasel

Adron Hall changes numbering schemes:

Today, I’ll dive into a fascinating challenge: converting modern numbers into their ancient Roman numeral counterparts. The task is straightforward but intricate, involving a programming challenge that takes any positive integer from 1 to 3999 and converts it into the corresponding Roman numeral.

To convert regular decimal numbers into Roman numerals, one must follow a set of rules based on the values and combinations of specific Roman numeral characters. Here’s a brief summary of the conversion process:

Click through for the translation rules and how to do this in C#. Now I’m thinking about how to do this in F# and thinking tail-call recursion. I might give that a try on my own and blog about it if I come up with something neat.

Comments closed

Cross-Correlation of Time Series to Identify Time Lags in SAS

Published 2024-05-28 by Kevin Feasel

Kevin Scott and David Frede notice the pattern:

Batch manufacturing involves producing goods in batches rather than in a continuous stream. This approach is common in industries such as pharmaceuticals, chemicals, and materials processing, where precise control over the production process is essential to ensure product quality and consistency. One critical aspect of batch manufacturing is the need to manage and understand inherent time delays that occur at various stages of the process.

In the glass manufacturing industry, which operates under the principles of batch manufacturing, precisely controlling the furnace temperature is essential for producing high-quality glass. The process involves melting raw materials like silica sand, soda ash, and limestone at high temperatures, where maintaining the correct temperature is crucial.

Read on to see an example of how you can automate the identification of a time lag using cross-correlation techniques.

Comments closed

Discriminated Unions in F# and C#

Published 2024-05-08 by Kevin Feasel

The NDepend blog talks discriminated unions:

In recent years, there has been a notable surge in inquiries from the C# community regarding language-level support for C# Discriminated Unions. What individuals truly desire is the ability to express complex data structures more elegantly and effectively.

This blog post offers a thorough introduction to simulating Discriminated Unions (DUs) in C# programs, highlighting their usefulness.

Discriminated Unions, also known as “or” types (because you select one from the set of options), are really powerful concepts in functional programming. And this article gives you a way to simulate the approach in C#, at least until C# catches up to F# in this aspect.

Comments closed

Listen and Notify in Postgres

Published 2024-05-06 by Kevin Feasel

Brandur Leach shows how to use PostgreSQL’s listen/notify capabilities:

Listen/notify in Postgres is an incredible feature that makes itself useful in all kinds of situations. I’ve been using it a long time, started taking it for granted long ago, and was somewhat shocked recently looking into MySQL and SQLite to learn that even in 2024, no equivalent exists.

In a basic sense, listen/notify is such a simple concept that it needs little explanation. Clients subscribe on topics and other clients can send on topics, passing a message to each subscribed client. The idea takes only three seconds to demonstrate using nothing more than a psql shell:

Read on to learn more about the notifier pattern. What’s interesting is that the notifier patter, which adds a fair bit of structure to this very simple process, makes it work a good bit like SQL Server’s Service Broker.

Comments closed

IAsyncEnumerable in C# 8

Published 2024-04-25 by Kevin Feasel

Camilo Reyes shows off an interface:

The IAsyncEnumerable interface was introduced to address the limitations of the IEnumerable interface and the Task class. This way, you can stream asynchronous data and process it efficiently as soon as it becomes available.

In this take, you will learn how to work with IAsyncEnumerable to asynchronously stream a big table and extract the data in a ETL process. You will also learn the difference between IAsyncEnumerable and IEnumerable and how to use IAsyncEnumerable in your everyday work. Then, you will look at comparisons between the two different approaches and why one is better than the other in certain scenarios.

Read on for a demonstration and dive into how IAsyncEnumerable implementations compare to IEnumerable in terms of memory utilization.

Comments closed

pl/dotnet Version 0.99

Published 2024-04-18 by Kevin Feasel

Brick Abode announces F# and C# support within Postgres:

pl/dotnet adds full support for C# and F# to PostgreSQL. 0.99 is our public beta release; we wish to share its amazingness with the world.

  • We support all PL operations: functions, procedures, DO, SPI, triggers, records, SRF, OUT/INOUT, table functions, etc
  • We natively support 40 out of 46 standard user types, the most of any external PL
  • Fully NPGSQL-compatible, and SPI is exposed through the NPGSQL API for maximum compatibility
  • In our benchmarks, C# and F# are the fastest Procedural Languages in PostrgreSQL
  • All features are fully tested for both C# and F#, with 1013 unit tests
  • 100% free software under the PostgreSQL license

This is a beta release; we invite usage and welcome feedback.

Look at me, side-eyeing SQL Server and how SQLCLR still doesn’t have F# support. I still maintain that the single biggest mistake Microsoft made around SQLCLR was adopting the “safe” and “unsafe” mode language. C# developers understood that “unsafe” meant you could get access to pointers and other internals that .NET languages typically hide from us. But try explaining that to a DBA, who doesn’t understand the language or the concepts.

On the bright side, .NET languages are the fastest procedural languages for Postgres, so that’s pretty neat. H/T Sergey Tihon.

Comments closed

Converting Cursors to PL/pgSQL

Published 2024-04-12 by Kevin Feasel

Deepak Mahto explains a difference in cursors from Oracle:

In the blog, we will cover scenarios with cursors that differ from how Oracle handles them. During conversion, our initial approach is to match all codebases as closely as possible with the target compatibility. However, in some cases, although the code appears identical, the functionality might vary. Let’s explore one such case here.

Click through for the scenario.

Comments closed

Querying a Database from Rust

Published 2024-04-11 by Kevin Feasel

Ukeje Goodness writes a query:

You’ll need to download and install Rust and your preferred SQL database management system to interact with SQL databases through Rust. Diesel. In this tutorial, we will use SQLite as the DBMS, but you can use your preferred relational database

After you’ve installed Rust and a preferred SQL DBMS that Diesel supports, you can proceed to create a new Rust project with Cargo’s init command:

Doing a separate search, it does look like you can execute stored procedures as well, using either the sql_query() function (when there is a result set you expect back) or execute() (when there isn’t).

Comments closed

Infrastructure as Code in GitHub

Published 2024-04-04 by Kevin Feasel

I have a new video:

In this video, we look at how to perform Infrastructure as Code in GitHub. We take a Bicep script and generate new Azure resources using it and GitHub Actions.

The video includes a very brief primer on Azure Resource Manager (ARM) and Bicep, and then gets into how you can use GitHub Actions to keep your Azure resources configured the way you expect.

Comments closed