A Fine Slice Of SQL Server
Jyoti Sachdeva explains some of the power of Either:
We use Options in scala but why do we want to go for Either?
Either is a better approach in the respect that if something fails we can track down the reason, which in Option None case is not possible.
We simply pass None but what is the reason we got None instead of Some. We will see how to tackle this scenario using Either.
This is a classic functional programming pattern and one of the easier monads to understand.
Comments closedPatrick Smacchia has 12 tips for debugging in Visual Studio:
4) Data breakpoint: Break when value changes
If you set a breakpoint to a non-static property setter it will be hit when changing the property value for all objects. The same behavior can be obtained for a single object thanks to the Locals (or Watch) window right click : Break When Value Changes menu.
This facility is illustrated with the animation above. The hit occurs only when obj2.Prop is changed, not when obj1.Prop is changed.
These go a step beyond the basics, so check them out.
Comments closedThe match expression consist of multiple parts:
1. The value we’ll use to match the patterns is called a candidate
2. The keyword match
3. At least one case clause consisting of the case keyword, the pattern, an arrow symbol, and the code to be executed when the pattern matches
4. A default clause when no other pattern has matched. The default clause is recognizable because it consists of the underscore character (_) and is the last of the case clauses
This is similar to the switch statement in C-like languages but offers up a few more things like partial matching of complex objects. Mansi covers some of the ways in which these two things differ.
Rahul Agarwal explains how higher-order functions make your life easier:
As a part of the functional programming paradigm, whatever logic we need to write is to be implemented in terms of pure and immutable functions. Here, functions take arguments from other functions as input and return values/functions which used by other functions for further processing. Here, pure means that the function does not produce any side-effects like printing to the console and immutable means that the function takes in and produces immutable data(val) only.
Higher-order functions comply with the above idea. As compared to for loops, we can iterate a data structure using higher-order functions with much less code.
The term “higher-order function” can sound a bit overwhelming if you’re completely unfamiliar, but it’s a pretty simple concept: a function which takes another function as (at least) one of its inputs. As Rahul points out, this is quite the useful concept.
Comments closedUnni Mana shows how to create a Kafka consumer and producer in the Kotlin language:
We are using
KafkaTemplateto send the message to a topic calledtest_topic. This will return aListenableFutureobject from which we can get the result of this action. This approach is the easiest one if you just want to send a message to a topic.
Generally, when we talk about the Hadoop ecosystem and functional programming languages on the Java Virtual Machine, we think Scala. But this is an example showing that Kotlin is in that discussion too.
Comments closedMikey Bronowski walks through an introduction to the Graphviz diagramming language:
I came across Graphviz which is an open-source graph visualization software initiated by AT&T Labs Research. It can process the graphs that are written in the DOT language.
What is the DOT language?
In short, it is a graph description language that has few keywords like graph, digraph, node, edge. You cannot miss it has something to do with graphs.
I’ve used the R implementation of this as well. It doesn’t create beautiful diagrams, but it is fast, easy, and the output makes sense.
Comments closedAnna Wykes continues a series on Scala for data engineers:
This is the second of my blogs in the Scala Parlour Series, in which we explore Scala, and why it is great for Data Engineering. If you haven’t already, please check out the first in the series here, in which you can read all about the core concepts of Scala, including who uses it and why
In this article we will explore monads within the Functional Programming (FP) paradigm, and how they can be used in Scala to aid Data Engineering.
Anna explains monads quite well here. This is a topic which is notoriously in how people perceive its difficulty, but conceptually it’s not as difficult as people take it to mean…if you understand a few concepts coming in.
Comments closedDan Guzman provides a public service:
Scripts are parsed by invoking the Parse method of T-SQL script DOM library TSqlParser class. The parser understands the complex T-SQL abstract syntax tree and splits T-SQL source into atomic TSqlParserTokens of TSqlTokenTypes that represent keywords, identifiers, punctuation, literals, whitespace, etc. These low-level tokens are grouped into more meaningful TSqlFragment objects that represent language elements of the script DOM, such as batches, statements, clauses, etc. Fragments, rather than the low-level parser tokens, are most often used in practice, although the underlying tokens are available for specialized requirements
The Parse method returns a TSqlFragment object of type TSqlScript containing all fragments within the script. This top-level fragment of the DOM hierarchy provides programmatic access to all language element fragments in the script. Nearly 1,000 different fragment types exist today due to the many granular T-SQL language elements.
Dan provides several examples of how to use the script DOM, making this a must-read if you’re interested in writing code around SQL Server.
Comments closedAdrian Hills walks us through the SqlBulkCopy class:
Ever been in a situation where rumblings of “process X is too slow” suddenly build into a super-high priority ball of urgency when that next step up in data volume hits? Yeah, that can be fun. No, really, it can be fun because we have strategies to sort this stuff out, right?
In this blog post, I’m going to talk about one particular piece of functionality—SqlBulkCopy—that can help you with bulk data loading. If I had to single out my favorite .NET class, SqlBulkCopy would be at the top of the list. My goal is to introduce you to this class so that maybe it can become a part of your tool belt, too.
Click through to see how it works. If you’re familiar with SSIS, you’re already familiar with the concept if not the specifics.
Comments closedSophia Rowland shows off the sastopypackage:
I started by declaring my parameters and sets, including my risk threshold, my stock portfolio, the expected return of my stock portfolio, and covariance matrix estimated using the shrinkage estimator of Ledoit and Wolf(2003). I will use these pieces of information in my objective function and constraints. Now I will need SWAT, sasoptpy, and my optimization model object.
Read on for a demo.
Comments closed