A Fine Slice Of SQL Server
Tomaz Kastrun is winding down an advent of Azure ML. Day 22 covers batch scoring:
Batch endpoints are a great and simple way to run inference over large volumes of data. They simplify the process of hosting your models for batch scoring.
Click through to see how it all works.
Comments closedTomaz Kastrun nears the end of the Azure ML advent. Day 20 covers package requirements and other niceties:
When creating notebooks, it is always a good way to have the dependencies included. Whether it is a particular version of a package, a separate script file or an installation requirement.
Selecting an environment or kernel can be an issue if it is not correctly initiated with the code. And you can also check the kernels with a simple python code:
Day 21 looks at the Azure CLI and running code from within a compute instance terminal:
Comments closedUsing Azure CLI can help you progress faster, make repetitve tasks automated and even use the GIT integration, for faster and better collaboration.
So we have created a YAML file on Day20 and we can use it also with Azure CLI to create an environment.
Tomaz Kastrun continues an advent of Azure ML. Day 18 takes us through feature exploration:
Azure Machine Learning is also a great tool to do ordinary statistical analysis, graph plotting and everything that goes along.
Let’s get an open dataset, that is available on UCI Machine Learning repository and import it in the pandas dataframe.
Day 19 picks up with feature engineering:
Comments closedYesterday we have shown, that statistical analysis and all bolts and whistles can be done super simple in Azure machine learning. Today we will continue with feature engineering and modelling.
So, what is feature engineering? Is a general process and can involve both feature construction: adding new features from the existing data, and feature selection: choosing only the most important features for improving model performance, reducing data dimensionality, doing log-transformation, removing outliers, to do scaling (normalisation, standardisation), imputations, general transformation (and others, as polynomial), variable creation, variable extraction and so on.
Tomaz Kastrun continues an advent of Azure ML. Day 16 shows off MLflow:
Yesterday we have looked into how to start the MLflow configurations and today, let’s put this to the test.
We will create a new notebook and use Heart dataset (link to dataset) to toy around. We will also import xgboost classifier to asses the accuracy of the presence of heart disease in the patient. We will be using a categorical (integer) variable with values from 0 (no presence) to 4 (strong presence) and attempt to classify based on 15+ attributes (out of more than 70 attributes).
Day 17 pivots to using the responsible AI dashboard:
Comments closedAzure ML has provided users with collection of model and data exploration with the Studio user interface. But it also provides compatible solutions with Azure ML and Python package responsibleai. With the help of widgets, we will create an sample of dashboard to explore the solution with assessing the responsible decisions and actions.
Tomaz Kastrun continues an advent series on Azure ML:
MLFlow is an open-source framework for registering, managing and tracking machine learning models. It is multiplatform, bringing consistent model training and model consumption across different platforms. This means, that training a model locally and uploading it to Azure or training a model on remote compute instances and downloading it, is a great feature for MLflow.
You can use MLflow with Azure CLI, Azure Python SDK or in the studio and it will deliver a consistent experience (note, some functionalities are limited to the language).
Click through for a quick overview of MLflow.
Comments closedTomaz Kastrun continues an advent of Azure Machine Learning. Day 13 covers the topic of Automated ML:
Automated ML is a no-code automated machine learning task. It iterates over many combinations of algorithms and hyperparameters in order to find the best model for your dataset and your prediction variable(s). The final solution is a model, that can be downloaded and later reused. So Automated ML is not just giving you the best model out of a family of algorithms, but lets you use the model, generate the scripts and create the artefacts.
Day 14 concerns model registration:
Comments closedImportant asset is the “Models” in navigation bar. This feature allows you to work with different model types -> custom, MLflow, and Triton. What you do here is, you register a model from different locations (e.g.: local file, AzureML Datastore, AzureML Job, MLflow Job, Model asset in AzureML workspace, and Model asset in AzureML Registry).
Once you open the Models asset, you will see, that you can do many things here. I have already model register from the running the notebook on day4.
Tomaz Kastrun continues an advent on Azure ML. Day 11 covers pipelines:
A pipeline is set of instructions (or a workflow) for executing particular work of a machine learning task. The idea behind pipelines is that will help the team of data scientists and machine learning engineers standardize workflow and incorporate best practices of preparing data, producing training models, executing the models and deploying them. Pipelines will help improve and build workflow efficiently and in such a way that it can be reusable.
And the idea behind it, is to split a machine learning process into smaller tasks, a multistep workflow, where each step is a separate component than can be developed, upgraded, optimised, configured, automated, and deleted separately. And these steps, connected through interfaces, form a workflow.
Comments closedAn Azure ML job executes a task against a specified compute target. This is also how the job is created. By configuring a new job, you can also scale out model training, since there are single node and distributed training available.
A simple job command would be to execute a command in a Docker container. And further parameter sweeping can be executed, by specifying it in the job itself.
Matt Eland tries out ML.NET 2.0:
Recently ML.NET 2.0 was released, giving us a bevy of new features for the open source machine learning library for dotnet applications.
The release improved ML.NET’s text processing capabilities and improved some aspects of the already fantastic automated machine learning capabilities it had. Moreover, the release seemed to reaffirm ML.NET’s determination to be relevant for advanced machine learning tasks, including deep learning and transformer-based architectures.
In this article we’ll explore ML.NET 2.0’s new text classification capabilities and see how you can use C# to analyze sentiment, match utterances to intents, or otherwise classify textual data without having to write a lot of custom code.
Read on to learn more about ML.NET and plenty of turtles.
Comments closedTomaz Kastrun continues a series on Azure Machine Learning. Day 9 takes us through a piece of the Python SDK:
Python SDK namespace is
azureml.core.environment. Environments specify the set of Python packages, environment variables, and software settings around your training and scoring scripts. In addition to Python, you can also configure PySpark, Docker and R for environments.You can use namespace
Environment(or created object/asset) to make deployment and code reusable for training purposes at given docker images, configurations and compute type.
Day 10 shows us how to work with the Python SDK via VS Code or a local Jupyter notebook:
Comments closedLet’s continue to explore the power of SDK and the namespaces. And we will look into namespace that will help you connect to Azure ML resources with Python SDK.
Tomaz Kastrun continues an advent of Azure ML. First up is environments:
We have explored how to create a compute instance and compute target and learned that ML frameworks and scripting packages always come preinstalled.
Choosing the right set of components (CPU, GPU, RAM, Core) and corresponding software (OS, ML Framework, packages) can be time-consuming.
Under Curated environments, you will find predefined environments, with settings for running particular frameworks, like PyTorch or TensorFlow.
Then an overview of the Azure CLI and Python SDK for AML:
What is Azure CLI? It is an Azure Command Line, a great tool for running commands out of CMD. It is a multi-platform and can be run from Azure or from the client’s machine. It is great for scripting and automating repetitive tasks or making the complex task look like lines of code, especially when it comes to infrastructure, managing, provisioning and monitoring. It can also be run from Azure Cloud Shell. It is native to Azure and can be used across all the services and offerings. Usually, the Azure CLI commands start with “az ..”. On top of that, you can also install Azure Machine Learning CLI, as an extension to Azure CLI. The AML CLI will give you additional commands to manage resources for machine learning.
The same functionality (to some extent) in Azure Machine Learning can be achieved with Python SDK. In addition to that, it offers also great ways to create and manage resources you use for training and deployment of models.
And, so that we can catch up a bit to Tomaz, one more post covering the Python SDK:
Comments closedLooking briefly into Azure CLI and Python SDK, let’s explore the power of SDK and the most important namespaces.