A Fine Slice Of SQL Server
Databricks natively stores it’s notebook files by default as DBC files, a closed, binary format. A .dbc file has a nice benefit of being self-contained. One dbc file can consist of an entire folder of notebooks and supporting files. But other than that, dbc files are frankly obnoxious.
Read on to see how to convert between these two formats.
Comments closedHasan Savran builds an Azure Data Studio notebook to query SQL Server from Python:
SQL Kernel is the default language, to query database with Python change SQL to Python 3. Probably, you will see the following message if this is the first time you are trying this. You need to install Python packages to be able to run python scripts. I have Visual Studio installed on my machine and I already have Python, I taught I could to use it by clicking “Use existing Python installation”. I was wrong, I couldn’t. This option looks for local installation files and when I point to Visual Studio Python files, it throws error in the middle of the installation. So, I will ignore this option for now.
In ADS, I haven’t gotten “Use existing Python location” to work either, so Hasan’s not alone in that regard.
Comments closedBernhard Walter announces an integration between JupyterLab and Databricks:
This blog post starts with a quick overview how using a remote Databricks cluster from your local JupyterLab would look like. It then provides an end to end example of working with JupyterLab Integration followed by explaining the differences to Databricks Connect. If you want to try it yourself, the last section explains the installation.
I like this a lot, as it fights back a bit against the balkanization of data science: it means I don’t need to keep one set of notebooks here and another set of notebooks there and a third set of notebooks somewhere else.
Comments closedFor me, one of the hardest parts of developing anything is when you need to pick up and rework code that has been created by someone else. That said, my preferred Notebook structure shown below is not about technical performance or anything complicated. This is simply for ease of sharing and understanding, as well as some initial documentation for work done.
In my example I created a Scala Notebook, but this could of course apply to any flavour.
This makes good use of markdown capabilities without being too heavy. I like it. The same general principles apply if you’re putting together Jupyter notebooks outside of Databricks.
Comments closedKevin Chant points out a change to Azure Data Studio:
I talked about the using the wizard in the Insiders build to deploy Big Data Cluster in that post. You can read that post in detail here.
Now, as well as deploying Big Data Clusters you can also use the wizard to deploy SQL Server container images locally using Docker.
Read on for notes about the process.
Comments closedRob Sewell inverts the “Use Azure Data Studio to create Powershell notebooks” mantra:
This module contains only 3 commands at present
* Convert-ADSPowerShellForMarkdown
This will create the markdown link for embedding PowerShell code in a Text Cell for a SQL Notebook as described in this blog post
* New-ADSWorkBookCell
This command will create a workbook text cell or a code cell for adding to the New-ADSWorkBook command
* New-ADSWorkBook
This will create a new SQL Notebook using the cell objects created by New-ADSWorkBookCell
Click through for an example.
Comments closedAaron Nelson announces a new feature in Azure Data Studio:
In order to get all the nice intellisense and tab completion features of the PowerShell language inside your PowerShell Notebooks, be sure to install the PowerShell extension from the Azure Data Studio marketplace.
At this point, the biggest remaining language is R, though I’d love to see F# support as well (hey, Azure Notebooks offers F# support).
Comments closedDave Bland takes us through some of the formatting options available in Azure Data Studio notebooks:
When working in a Notebook you have two types of cells, text and code. The focus of this post is how to format the text cell. Of course text goes into this cell so that part is easy and of course the text can say anything you would like to say. When we work with text in Word, there is a format tool bar that we can use to make it look like we want it. The text cells do not have this toolbar.
You might be asking, without the format toolbar, does that mean we can’t format the text? That answer is no….we can still format the text, we just need to do it slightly different. Rather than use a toolbar, we need to use characters.
There’s a lot of power in Markdown.
Comments closedAlan Yu announces the newest release of Azure Data Studio:
With the November release of Azure Data Studio, we have added three key features:
– PowerShell kernel support
– Collapsible code cells
– Output toolbar for save output actions
There are plenty of other additions as well. Adding Powershell kernel support is interesting…but I still want F#.
Comments closedJeremy Smith, et al, announce a new product:
We are pleased to announce the open-source launch of Polynote: a new, polyglot notebook with first-class Scala support, Apache Spark integration, multi-language interoperability including Scala, Python, and SQL, as-you-type autocomplete, and more.
Polynote provides data scientists and machine learning researchers with a notebook environment that allows them the freedom to seamlessly integrate our JVM-based ML platform — which makes heavy use of Scala — with the Python ecosystem’s popular machine learning and visualization libraries. It has seen substantial adoption among Netflix’s personalization and recommendation teams, and it is now being integrated with the rest of our research platform.
There are some nice pieces to it, especially around language interop.
Comments closed