A Fine Slice Of SQL Server
Reitse Eskens checks the logs:
I reported an issue yesterday with Microsoft Support and during the following call today (they’re really quick to set up an initial meeting), the support engineer showed me where I can find a lot of logging information.
Suppose you’ve got a notebook that has been run a few times. The front-end will only retain the information from the last run. If you see an error, for example this one
Click through to learn where you can find these execution logs.
Comments closedJohn Mount shows off a feature:
I’d like to share a great new feature in the wvpy package (available at PyPi).
This package is useful in converting Jupiter notebooks to/from python, and also in rendering many parameterized notebooks. The idea is to make Jupyter notebook easier to use in production.
The latest feature is an extension of notebook parameterization. In addition to the
init_codeandoutput_suffixfeatures, which allow adding arbitrary code to notebooks and saving multiple renders of the same notebook under different (non-coliding!) names. The newsheet_varsfeature allows the insertion of arbitrary data into notebook renders (in addition to the earlier code insertion facility).
Click through for an example on how to use this. Several years ago, I would have considered this to be outstanding. Today, I think it’s cool, but I’ve also gravitated toward using notebooks as an intermediary step rather than a final product, so it’s less critical for me these days.
Comments closedGilbert Quevauvilliers tries out a tool:
I was going through my twitter feed and I came across this tweet where they spoke about the Data Wrangler Calling all #Python users! Have you tried Data Wrangler in #MicrosoftFabric?
I thought I would give this a try and that was the idea for my blog post. I honestly had no idea that firstly was this possible, but second that it is so easy for data wrangler to do all the hard work for me
I am going to demonstrate 2 transformations in this blog post, the first will be changing the d_date from date to datetime and then using the columns from examples I am going to create a new column where it concatenates 2 columns delimited with a double pipe command.
Read on for Gilbert’s thoughts.
Comments closedLeila Etaati provides an explanation:
In Fabric, there are tools for different personas of the users to work with. For example, for a citizen data analyst, Dataflows and Power BI Datasets are the tools with which the analyst can build the data model. For Data Engineers and Scientists, one of the tools is Notebook.
The Notebook is a place to write and run codes in languages such as; PySpark (Python), Spark (Scala), Spark SQL, and SparkR (R). These languages are usually familiar languages for data engineers and data scientists. The Notebook provides an editor to write code in these languages, run it in the same place, and see the results. Consider this as the coding tool for the data engineer and scientist.
Click through for a video, as well as a regular blog post.
Comments closedMatt Eland brings me back to college:
Polyglot Notebooks is a great way of running interactive code experiments mixed together with rich markdown documentation.
In this short article I want to introduce you to the
#!timemagic command and show you how you can easily measure the execution time of a block of code.This can be helpful for understanding the rough performance characteristics of a block of code inside of your Polyglot Notebook.
In fact, we’ll use this to explore the programming concepts behind Big O notation and how code performance changes based on the number of items.
I like this for two reasons. First, because a visual indicator of Big-O notation is helpful for students learning about the topic. Second, because that’s not the only thing you can do with the #time magic.
Matt Eland brings some code to the party:
We’ve seen that Polyglot Notebooks allow you to mix together markdown and code (including C# code) in an interactive notebook and these notebooks allow you to share data between cells and between languages. However, frequently in programming you want to reference code that others have written without having to redefine everything yourself.
In this article we’ll explore how Polyglot Notebooks allows you to import dotnet code from stand-alone files, DLLs, and NuGet packages so your notebooks can take advantage of external code files and the same libraries that you can work with from your code in Visual Studio.
The syntax, by the way, is very similar to the F# Interactive (and the short-lived C# Interactive) tool, particularly #i and #r.
Matt Eland continues a series on dotnet Polyglot Notebooks:
As I’ve been doing more and more dotnet development in notebooks with Polyglot Notebooks I’ve found myself wanting more options to create rich data visualizations inside of VS Code the same way I could use Plotly for data visualization using Python and Jupyter Notebooks.
Thankfully, Microsoft SandDance exists and fills some of that gap in terms of doing rich data visualization from dotnet code.
In this article I’ll talk more about what SandDance is, show you how you can use it inside of a Polyglot Notebook in VS Code, and show you a simple way you can use it without needing a Polyglot Notebook.
Most of my experience with SandDance was with Azure Data Studio, but it’s nice to see this capability in notebooks as well.
Comments closedMatt Eland performs a few swaps:
While Polyglot Notebooks certainly brings the dream of notebook development to dotnet, Polyglot is at its finest when you work with one language and then hand off data to the next language for additional processing.
In this article we’ll talk about sharing variables between kernels using Polyglot Notebooks and VS Code. We’ll explore the syntax and tooling that exists around these functionalities as well as the current limitation of sharing variables between kernels.
For simplicity, I’m going to avoid getting into SQL and KQL kernels in this article, but I plan on delving further into each of these specialized kernels in future articles.
Click through for an example using the best .NET language, as well as C#. Do read the whole thing, especially if you think about passing around discriminated unions or method-reach objects.
Comments closedMatt Eland chooses a file extension:
I’ve been talking more and more about Polyglot Notebooks and as people try it out, they tend to ask me one common question: should I create a
.dibfile or an.ipynbfile? What’s the difference anyways?In this short article I’m going to explore the
.diband.ipynbfile formats and explain the difference between the two while answering the question of which one you should choose when creating your own notebooks in Polyglot Notebooks.
Read on for Matt’s thoughts. My tendency is to create them in .ipynb format for additional tooling support and potential cross-product flexibility (assuming you have the right kernels installed on your Jupyter server), though Matt explains his preference for .dib.
Comments closedMatt Eland walks us through a sample:
Polyglot Notebooks allows you to create notebooks composed of multiple cells. These cells can be either markdown cells for documentation or code cells containing code in either C#, F#, PowerShell, SQL, KQL, HTML, JavaScript, or mermaid markdown for diagramming.
This allows you to mix together rich documentation supported by little pieces of code that progressively expand upon an idea, tell a story, or otherwise provide insight or information to you as a developer.
Click through for the example. The thing I hadn’t realized—because I don’t really do this in Jupyter—is that you can share variables between languages. That’s a fairly useful feature when you want to do most of your work in one language but just happen to need a library using a different language.
Comments closed