A Fine Slice Of SQL Server
In this post I want to cover how you can keep your Azure Synapse secrets secret in Azure DevOps. Because you need to do this if you are working with production deployments.
With this in mind, I want to raise more awareness about it and make sure others avoid putting secrets directly in their pipelines like in the below example.
Read on to understand what options are available to you. My preference involves Key Vault references but there are alternatives available.
Comments closedKoen Verbeeck makes use of system databases in a database project:
I have a database project in Visual Studio. Inside the database, I use a couple of system views to fetch some metadata about tables. To make the project build successfully, you need to add a reference to the master database in the project.
That all works fine but there’s a bit more you need to do before Azure DevOps can work with the file. Read on to learn what that thing is.
Comments closedMeagan Longoria solves a problem:
We chose release pipelines over the YAML pipelines because it was easier to manage for the client and pretty quick to set up. While I had done this before, I had a couple of new challenges:
– I was deploying to an Azure SQL managed instance that had no public endpoint.
– There were multiple databases for which there may or may not be a change script to execute in each release.
This took a bit longer than I expected, and I enlisted my coworker Bill to help me work through the final details.
Read on to see how Meagan and Bill solved the problem.
Comments closedI continue a series on Azure ML:
We ended the prior series with model deployment via the Azure ML Studio UI. This is entirely manual and UI-driven. Then, we looked at model deployment via manually-run notebooks. This is still manual but at least offers the possibility of automation as we control the code to run.
From there, we moved to model deployment via the Azure CLI and Python SDK. Now we have the capability to run, train, register, and deploy models via scripts. This leads to the next phase in the process, in which we can perform continuous integration and continuous deployment of models using a tool like Azure DevOps or GitHub Actions. This is where MLOps starts to shine.
Read on for a few thoughts about MLOps and software maturity.
Comments closedMartin Schoombee keeps customer software separated:
As a consultant I work on at least a few projects at a time, and prefer to isolate my development environments by creating an Azure VM for each customer. Isolating the environments are great because I can focus on the software and setup I need for that customer, and will never be in a situation where VPN clients or different software versions clash with each other.
With my development environments in Azure I am truly mobile, can work from anywhere and can lose my working machine at any point without much impact beyond getting another one.
Click through to see how Martin can do this with hardly a problem.
Comments closedJonathan D’Aloia looks at Azure Bicep:
This is going to be the first a few blogs in a series related to Azure BICEP. I will start the journey from the very beginning by showing you how to configure a local environment all the way to automating bicep deployments through multi-stage YAML Pipelines, covering how you can scale your infrastructure quickly and effectively.
In this blog, I will give a brief introduction to Azure BICEP and will also cover the easiest way to configure an environment locally ready to build and deploy your bicep templates.
Read on for the setup portion of the series.
Comments closedRayis Imayev notes a problem with Azure DevOps deployments:
Here is one thing that still doesn’t work well when you try to alter an existing temporal table and run this change through the [SqlAzureDacpacDeployment@1] DevOps task, whether this change is to add a new column or modify existing attributes within the table. Your deployment will fail with the “This deployment may encounter errors during execution because changes to … are blocked by …’s dependency in the target database” error message.
Read on to see what causes this problem and what we can do to work around it.
Comments closedKevin Chant deploys some Synapse pipelines:
In this post I want to cover how you can automate a pipeline migration to a Synapse workspace using Azure DevOps. As a follow up to a previous post I did about one way to copy an Azure Data Factory pipeline to Synapse Studio.
Because even though the post is good it deserves a follow up showing an automated way of doing it. I wanted to show that it can be done more gracefully.
And we all want to be graceful, right?
Comments closedPiotr Majer and Michael Shtelma complete a series on MLOps on Databricks:
This is the second part of a two-part series of blog posts that show an end-to-end MLOps framework on Databricks, which is based on Notebooks. In the first post, we presented a complete CI/CD framework on Databricks with notebooks. The approach is based on the Azure DevOps ecosystem for the Continuous Integration (CI) part and Repos API for the Continuous Delivery (CD). This post extends the presented CI/CD framework with machine learning providing a complete ML Ops solution.
Check it out.
Comments closedAnna Wykes continues a series on DevOps for Databricks:
An Environment Variable is a variable stored outside of the Python script; in our instance it will be stored on the DevOps Agent running the DevOps Pipelines. Consequently, it is accessible to other scripts/programs running on the DevOps Agent. We will not cover DevOps Agents in this blog specifically, the simplest description is that they are the compute that runs your pipeline, normally a VM (Virtual Machine) or Docker Container
Read the whole thing.
Comments closed