Press "Enter" to skip to content

Category: Deployment

Source Control and Change Management for Postgres

Published 2023-06-05 by Kevin Feasel

Ryan Booz relives an older story:

For those of you that don’t know, those ER tools were really expensive (probably still are for the ones that exist) and only a few developers had access to the tool. They didn’t have a great DX either.

Aside from the lack of automation and ability of our developers to be more integrated into the process, there was always the one looming issue that we just couldn’t reconcile.

If Joe left and joined the circus (see, I got you there), we were stuck.

We knew this was a bottleneck for some time and we had tried multiple times to change the process. Our ability to iterate on new feature development went through one person and a set of 15-year-old scripts. It didn’t match our otherwise Agile process of front-end code and data analysis projects.

Read on for Ryan’s thoughts on database change management. Some of the tools mentioned work with multiple database platforms, whereas others are specific to Postgres.

Comments closed

Reusable Power BI Deployment Pipelines

Published 2023-05-23 by Kevin Feasel

Richard Swinbank re-uses a pipeline:

Implementation of one pipeline per report makes additional demands of a developer when creating a new report. To make this easier to manage, in this post I look how to make pipeline creation as simple as possible, by building each pipeline from a set of reusable components.

Click through to see how this works in Azure DevOps. I’d expect the process to be reasonably similar for GitHub Actions as well.

Comments closed

Feature Branching and Hotfixes for Azure DevOps

Published 2023-05-12 by Kevin Feasel

Vytas Suopys covers a bit of source control strategy:

Have you ever deployed a release to production only to find out a bug has escaped your testing process and now users are being severely impacted? In this post, I’ll discuss how to deploy a fix from your development Synapse Workspace into a production Synapse Workspace without adversely affecting ongoing development projects.

This example uses Azure DevOps for CICD along with a Synapse extension for Azure DevOps: Synapse Workspace Deployment. In this example, I assume Synapse is already configured for source control with Azure DevOps Git and Build and Release pipelines are already defined in Azure DevOps. Instructions on how to apply this this can be found in the Azure Synapse documentation for continuous integration and delivery.

The specific example covers Synapse, though the general principle applies no matter what you’re deploying.

Comments closed

Publish to Power BI Environments via ADO

Published 2023-05-10 by Kevin Feasel

Richard Swinbank deploys a report:

In the first post in this series, I built an Azure DevOps pipeline to automate steps in a Power BI development workflow. The pipeline implemented a very basic workflow – as soon as a developer committed a new report version to Git, the pipeline deployed it immediately into a Power BI workspace.

In this post I’ll be building a pipeline to support a more sophisticated workflow that enables peer review and stakeholder testing.

Click through for the step-by-step process.

Comments closed

Deploying Azure SQL Edge

Published 2023-05-04 by Kevin Feasel

Kevin Chant takes us to the edge:

Azure SQL Edge is a version of the SQL database engine that is designed to be deployed on IoT (Internet of Things) devices.

It is based on SQL Server 2019. Which means that by default all new databases are created using the SQL Server 2019 compatibility level. You can lower the compatibility level all the way down to SQL Server 2008 if required.

There was some nice functionality in Azure SQL Edge, some of which (like DATE_BUCKET() and DATETRUNC() made it into SQL Server 2022).

Comments closed

Installing SqlPackage for a Deployment Pipeline

Published 2023-05-01 by Kevin Feasel

Kevin Chant uses a deployment tool to install a deployment tool for his deployment tools:

I decided to do this post after some feedback I received about SqlPackage after a series of posts about deploying dacpacs to serverless SQL Pools. For example, my post about deploying a dacpac to a serverless SQL pool.

Because in order to deploy dacpacs to serverless SQL Pools you must update SqlPackage.

With this in mind, I thought I better go through various ways to update SqlPackage if intending to use it to deploy dacpacs to serverless SQL Pools.

Read on to see how you can do this.

Comments closed

Building an Azure DevOps YAML Pipeline

Published 2023-04-18 by Kevin Feasel

Olivier Van Steenlandt busts out the YAML:

In previous blog posts, I explained how to automate the Database Project Build & Deployment process using Azure DevOps (Release) Pipelines. These blog posts focused on setting up as easily as possible using the Classic Editor.

In this blog post, I’m going through the steps of setting up a build pipeline using YAML.

Read on to learn why the YAML-based approach is the best option for ADO and how to build a pipeline.

Comments closed

Using Flyway for MySQL Database Deployments

Published 2023-04-05 by Kevin Feasel

Ting Chou builds a proof of concept:

Few days ago, I asked myself a question about how can you better manage all the database changes defined in SQL scripts with DDL (Data Definition Language)? Maybe try to use version control on the SQL code and then use DevOps pipelines to achieve continuous deployment. Then, I came up with a poc solution, choux130/DevOps_In_DE/jenkins_mysql_flyway.

Click through for the repo, as well as a few TODOs.

Comments closed

Improving the Robustness of ML Model Deployment

Published 2023-04-04 by Kevin Feasel

Alexander Billington shares a few tools and tips:

Machine learning (ML) model deployment is a critical part of the MLOps lifecycle, and it can be a challenging process. In the previous blog, we explored how Azure Functions can simplify the deployment process. However, there are many other factors to consider when deploying ML models to production environments. In this blog, we’ll delve deeper into some of the essential hints and tips for more robust model deployments. We’ll look at topics such as proper model versioning and packaging, data validation, and performative code optimisations. By implementing these practices, data scientists and ML engineers can ensure their models are deployed efficiently, accurately, and with minimal downtime.

MLflow is definitely a good recommendation, as is Pydantic (which is on my to-learn list…one of these days).

Comments closed