A Fine Slice Of SQL Server
Olivier Van Steenlandt busts out the YAML:
In previous blog posts, I explained how to automate the Database Project Build & Deployment process using Azure DevOps (Release) Pipelines. These blog posts focused on setting up as easily as possible using the Classic Editor.
In this blog post, I’m going through the steps of setting up a build pipeline using YAML.
Read on to learn why the YAML-based approach is the best option for ADO and how to build a pipeline.
Comments closedTing Chou builds a proof of concept:
Few days ago, I asked myself a question about how can you better manage all the database changes defined in SQL scripts with DDL (Data Definition Language)? Maybe try to use version control on the SQL code and then use DevOps pipelines to achieve continuous deployment. Then, I came up with a poc solution, choux130/DevOps_In_DE/jenkins_mysql_flyway.
Click through for the repo, as well as a few TODOs.
Comments closedAlexander Billington shares a few tools and tips:
Machine learning (ML) model deployment is a critical part of the MLOps lifecycle, and it can be a challenging process. In the previous blog, we explored how Azure Functions can simplify the deployment process. However, there are many other factors to consider when deploying ML models to production environments. In this blog, we’ll delve deeper into some of the essential hints and tips for more robust model deployments. We’ll look at topics such as proper model versioning and packaging, data validation, and performative code optimisations. By implementing these practices, data scientists and ML engineers can ensure their models are deployed efficiently, accurately, and with minimal downtime.
MLflow is definitely a good recommendation, as is Pydantic (which is on my to-learn list…one of these days).
Comments closedStephanie Bruno does a bit of continuous integration:
There’s a lot of information on how to get around the lack of an out-of-the box CI/CD solution for Power BI datasets, but for me it’s often complicated and I have to read too many pages before making much progress on my own. This post is here to strip it down and provide you with the easiest way we know to enable a bonafide CI/CD process for Power BI datasets with Azure DevOps. The post is still longer than we’d like, but it includes detailed step-by-step instructions to walk you through every part of the process. To save space, we used slideshows for the screenshots, but you can pause them as you follow along.
There are a lot of steps but the goal is a worthwhile one.
Comments closedOlivier Van Steenlandt runs into a versioning issue:
The error message I received during the build process in my Azure DevOps YAML Pipeline was :
##[error]C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(1229,5): Error MSB3644: The reference assemblies for .NETFramework,Version=v4.5 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application. You can download .NET Framework Developer Packs at https://aka.ms/msbuild/developerpacksI wasn’t sure how to solve this issue, and when I was using my on-premise Agent Pool, the Database Project was able to build successfully.
Click through for the solution
Comments closedThe last year, however, has had more people looking to implement database DevOps and speed up their development, but not a lot of questions or demands for zero downtime during these deployments. I find that interesting as the world depends more and more on computer systems, and the customer base for many organizations may demand access to the systems at any hour of the day or night.
However, it doesn’t seem that as many people are concerned about small moments of downtime.
My ego would like to tell people that it’s probably because everybody read my post about how zero downtime isn’t really a thing.
Ego aside, it is pretty interesting that Steve’s not finding as much push for minimal downtime. Steve does include a few conjectures as to why it may be so.
Comments closedAlexander Billington needs to get that new model out:
Deploying machine learning (ML) models into production can be challenging, as it requires careful consideration of various factors such as scalability, reliability, and maintainability. While developing an ML model is an exciting process, deploying it into production can be a daunting task. The challenges faced in productionising data science projects can range from infrastructure to version control, model monitoring to integration with other systems. This blog will take a look at how Azure Functions can simplify the deployment process, getting models into production quickly and robustly to maximise their value.
I like this approach and find it interesting, as most of the time, the MLOps model Microsoft recommends has you scheduling Azure DevOps pipelines / GitHub Actions periodically or when new training data hits a specific folder. If you have some non-standard trigger for an action, this is a good way to get you going.
Comments closedOlivier Van Steenlandt pushes out the database:
In my previous blog post, Automate your Database Builds – Using Azure DevOps Pipelines, we successfully automated our Build process. As a next step, we will be automating our deployment process to Azure SQL.
In this blog post, we will go through the process step-by-step to set up our Release pipeline in Azure DevOps and execute our first release just by clicking a button.
Olivier has plenty of screenshots to take us through the process.
Comments closedHiram Fleitas rebuilds the QA environment:
In this post I am going to cover how to automatically refresh a lower environment commonly used for testing as part of your release (CD) pipeline.
Well, why? – you may be asking.
- In some cases, developers and testers need to test their application code-changes against a fresh copy of production-like data. This helps them do validations prior to publishing their changes to production where their apps are bombarded by end-user live workloads.
- Also, the lower environment may be used for testing, and we can’t overwrite the test data constantly. It needs to be a hot-standby refresh, made available when necessary.
Click through for notes on the process.
Comments closedOlivier Van Steenlandt deploys a database:
After we successfully introduced a database development strategy in my previous blog post series, Getting Started With Database Projects & Azure DevOps, we can look at how to introduce a database deployment automation strategy using Database Projects and Azure DevOps Pipelines.
As a starter, we will first be implementing a build automation process and in future blog posts, we will go through the different ways of deployment to different environments. On top of that, we will also discuss the differences between SQL Server and AzureSQL database deployments.
Read on for the full story.
Comments closed