A Fine Slice Of SQL Server
Josephine Bush automates a deployment:
A well-structured Terraform module for Azure SQL DB typically consists of the following elements:
- Main Configuration Files:
main.tf,variables.tf,outputs.tf- Helper Files: (if necessary)
locals.tf,providers.tf, etc.If you want to learn more about the basics of Terraform, you can visit my previous blog post.
Click through to see how Josephine has put together the Azure SQL Database deployment module.
Comments closedKevin Chant checks in a Power BI project:
I decided to test the guide with the Power BI report that I showed in my post about work with Microsoft Fabric Git integration and multiple workspaces.
So, I went through the guide and was pleasantly surprised that it showed how to do it with a YAML pipeline in Azure Pipelines. Which I must admit I prefer for reasons that I covered why in a previous post about disabling classic pipelines in Azure DevOps.
Read on for a review of the issues Kevin had to sort out, as well as two mechanisms to improve the performance of your Azure DevOps CI process.
Comments closedAmit Khandelwal packs a lot into a post:
It’s time for a new blog on how Ansible can simplify SQL Server deployment, configuration, and availability. If you’ve read my previous blogs on Ansible for SQL Server installation and configuration, and the pacemaker-based Always On availability group, you know how powerful Ansible can be. Now, let’s talk about HPE Serviceguard for Linux (SGLX), a high availability/disaster recovery solution that provides business continuity for critical applications like SQL Server.
Deploying SQL Server Always On availability groups on HPE SGLX is a fully supported solution for production workloads.
Today, let’s look at how you can configure Always On availability group based on HPE SGLX via Ansible. We have collaborated with our friends in HPE to enable the Ansible based deployment for HPE SGLX with SQL Server. This feature is now available for HPE SGLX 15.10. For this demonstration, you can download the evaluation bits from the ‘My HPE Software Center‘. The Ansible bits with the scripts are available on GitHub:
Read on for instructions and what you need to make it all work.
Comments closedGeorgia Atkinson wraps things up with a bow:
Python is a general purpose, high level language which, thanks to its simplicity and versatility, has become very popular, especially within the data science community. The extensive Python community has developed and contributed thousands of libraries and packages over the years in a plethora of different disciplines to aid developers with their applications. Managing these packages can be a challenging task without the correct tools. That’s where Python package managers come in. In this blog post we will explore what a package manager is and why they are important. We will then cover some popular examples, including how to use them, how to install them and the pros and cons of each.
Whilst we will briefly touch on virtual environments in places, we will explore these in more depth in an upcoming post.
Read on for a primer on three options, including how they compare to one another for CI/CD purposes.
Comments closedDustin Vannoy covers one technique for CI/CD in Databricks:
Databricks Asset Bundles provides a way to version and deploy Databricks assets – notebooks, workflows, Delta Live Tables pipelines, etc. This is a great option to let data teams setup CI/CD (Continuous Integration / Continuous Deployment). Some of the common approaches in the past have been Terraform, REST API, Databricks command line interface (CLI), or dbx. You can watch this video to hear why I think Databricks Asset Bundles is a good choice for many teams and see a demo of using it from your local environment or in your CI/CD pipeline.
Click through for a video and some sample scripts.
Comments closedIt’s a bit of a challenge to keep up with all the changes, updates and all the new stuff coming out for Fabric. As I’m not really invested in the PowerBI part of the data platform (yay pie charts ;)), some things that are very common for the PowerBI community are very new to me. I have it on good authority that this blog covers a feature that is well know within PowerBI but quite new in the data engineering part. When I say that, I need to add that at the time of writing, only the PowerBI side of things are fully supported but I have very good hopes that pipelines and notebooks will be supported as well.
Supporting pie charts are fightin’ words here. Nonetheless, read on to see how deployment pipelines work in Microsoft Fabric.
2 CommentsKevin Chant creates a database project:
In this post I want to cover how you can share a Microsoft Fabric Data Warehouse Database Project with the new target platform.
Which is now possible thanks to the latest Azure Data Studio Insiders update. You can view the ‘Add projects support for Fabric DW‘ pull request in the public azuredatastudio GitHub repository.
Kevin takes us through creating the database project in Azure Data Studio and then using Azure DevOps or Azure Data Studio to deploy it back out.
Comments closedRui Cunha has a tutorial for us:
Azure Synapse Analytics Serverless SQL is a query service mostly used over the data in your data lake, for data discovery, transformation, and exploration purposes. It is, therefore, normal to find in a Synapse Serverless SQL pool many objects referencing external locations, using disparate external data sources, authentication mechanisms, file formats, etc. In the context of CICD, where automated processes are responsible for propagating the database code across environments, one can take advantage of database oriented tools like SSDT and SqlPackage CLI , ensuring that this code is conformed with the targeted resources.
In this article I will demonstrate how you can take advantage of thee tools when implementing the CICD for the Azure Synapse Serverless SQL engine. We will leverage SQL projects in SSDT to define our objects and implement deploy-time variables (SQLCMD variables). Through CICD pipelines, we will build the SQL project to a dacpac artifact, which enables us to deploy the database objects one or many times with automation.
Click through for the demonstration.
Comments closedReitse Eskens sets up some new resources:
When you start out with Terraform, you’ll most likely run the code locally with terraform on your own machine. Terraform works with a so-called state-file, it saves the state of the Azure deployment it left behind and compares the (new) code with the state it encounters when it runs again. Changes are resolved by changing, deleting or adding resources that don’t match the state-file.
This works fine when you’re flying solo and don’t have co-workers who can change resources as well. Whenever you need to share code, the industry standard is to use a git solution, whether GitHub, GitLab, Azure DevOps or some other solution, as long as it has version control you should be fine (providing people adhere to the correct usage of branches).
Click through for a step-by-step walkthrough, as well as explanation of the major actors in that play.
Comments closedMarc Lelijveld notes a chemical reaction:
Together with the introduction of Microsoft Fabric a long-awaited feature for Power BI also became available. Git integration now allows us to connect your workspace to a git repository to sync the meta data from your Power BI dataset and report between the workspace and the repository.
At the same time, Power BI deployment pipelines are around. The pipelines help you to move content between workspaces from Development to Test and finally to Production. The question that arises, when to use what? And how do they complement each other, or maybe even clash with each other?
Read on for Marc’s thoughts across these two topics.
Comments closed