Press "Enter" to skip to content

Category: Generative AI

Thoughts on LLM Ethics

Published 2025-01-02 by Kevin Feasel

Eugene Meidinger has some thoughts:

The more I tried to research practical ways to make use of ChatGPT and Power BI, the more pissed I became. Like bitcoin and NFTs before it, this is a world inextricably filled with liars, frauds, and scam artists. Honestly many of those people just frantically erased blockchain from their business cards and scribbled on “AI”.

There are many valid and practical uses of AI, I use it daily. But there are just as many people who want to take advantage of you. It is essential to educate yourself on how LLMs work and what their limitations are.

I am saddened that my rants on the topic didn’t merit Eugene explicitly mentioning me. My natural response will be to rant harder until I receive the attention I desire. In the meantime, read the whole thing.

Leave a Comment

Azure AI Foundry Notes

Published 2024-12-27 by Kevin Feasel

Tomaz Kastrun wraps up a series on Azure AI. First up is tracing in Azure AI Foundry:

Tracing is a powerful tool that offers developers an in-depth understanding of the execution process of their generative AI applications. Though still in preview (in the time of writing this post), It provides a detailed view of the execution flow of the application and the essential information for debugging or optimisations.

After that, we can see how to evaluate model results:

With evaluation you performing iterative, systematic evaluations with the right evaluators and measure and address potential response quality, safety, or security concerns throughout the AI development lifecycle, from initial model selection through post-production monitoring.

With the Evaluation in Azure AI Foundry, you can evaluation the GenAI Ops Lifecycle production. In addition, it also gives you the ability to  assess the frequency and severity of content risks or undesirable behavior in AI responses.

Finally, Tomaz wraps up the series with some notes on documentation:

Documentation and material for Azure AI Foundry are plentiful and growing on a daily basis, since the topic on AI and GenAI is evermore so popular.

I appreciate the challenge that Tomaz has of putting together 25 blog posts in a month, especially when they’re all tied to a single theme.

Comments closed

Prompt Flow in Azure AI

Published 2024-12-24 by Kevin Feasel

Tomaz Kastrun continues a series on Azure AI. First up is an introduction to Prompt Flow:

Prompt flow in Azure AI Foundry is development tool for designing the flows (streamlines) for the complete end-to-end development cycle of LLM’s AI application. You can create, iterate, test, orchestrate, debug, and monitor your flows.

After that, we get a demonstration a Prompt Flow in Python:

Prompty gives you the ability to create an end-to-end solution, like RAG where you can chat with LLM over an article or document, where you can ask to classify the input data (list of URLs,…)

Prompty is a markdown file, structured in YAML and encapsulates a series of metadata fields pivotal for defining the model’s configuration and the inputs. After this front matter is the prompt template, articulated in the Jinja format.

Comments closed

Models and Endpoints in Azure AI Foundry

Published 2024-12-23 by Kevin Feasel

Tomaz Kastrun continues a series on Azure AI:

Models from the model catalog can be deployed using programming languages or using the Foundry studio.

Model deployment has two types: Deploy from the base model or deploy from the fine-tuned model. The difference is that fine-tuned model is model taken from the model catalog and later tuned to an additional dataset, as the base model is the model as it is available in Azure AI Foundry.

Click through for a bit more information on the process.

Comments closed

An Overview of Azure OpenAI and the Azure AI Hub

Published 2024-12-19 by Kevin Feasel

Tomaz Kastrun has a pair of posts. First up, an overview of Azure OpenAI:

Let’s first address the elephant in the room. We have explored the Azure AI Foundry and the we have also Azure OpenAI. So what is the core difference? Let’s take a look:

The services in the back:

  • Azure AI Services has much broader AI capabilities and simpler integration into applications and usage of the real world. With mostly pre-build API for all services (face recognition, document recognition, speech recognition, computer vision, image recognition, and more) that will allow better interoperabilty and and connection to machine learning services (Azure Machine Learning Service).
  • Azure OpenAI is focusing primarly on OpenAI LLM models (Azure AI services supports many others) and provides great agents for conversations, content tools, RAG and natural language services.

After that comes an overview of the Azure AI Hub and AI projects:

In AI Foundry portal, hubs provide the environment for a team to collaborate and organize work, and help you as a team lead or IT admin centrally set up security settings and govern usage and spend. You can create and manage a hub from the Azure portal or from the AI Foundry portal, and then your developers can create projects from the hub.

In essence, Hubs are the primary top-level Azure resource for AI Foundry. Their purpose is to to govern security, connectivity, and computing resources across playgrounds and projects.

Comments closed

Fine-Tuning an Azure AI Model

Published 2024-12-18 by Kevin Feasel

Tomaz Kastrun updates a generative AI model:

Fine-tuning is the process of optimizing a pretrained model by training it on your specific dataset, which often contains more examples than you can typically fit in a prompt. Fine-tuning helps you achieve higher quality results for specific tasks, save on token costs with shorter prompts, and improve request latency.

Read on to see how you can do this. Note that you’ll need to set up the fine-tuning data in a particular format for whatever model you’re using.

Comments closed

Azure AI and Content Safety

Published 2024-12-17 by Kevin Feasel

Tomaz Kastrun continues a series on Azure AI, this time focusing on content safety functionality. First up is an overview of the product:

Content safety Azure AI service detects harmful user-generated and AI-generated content in applications and services. It includes text and image APIs that allow you to detect harmful or inappropriate material. This service is , as all other services, easy to integrate to your app.

After that is how to access items via the SDK:

The Python SDK contains several functions to analyze text, images, and manage blocklists in text moderation. With the SDK you can cover the following scenarios:

  • Text moderation: Detect hate speech, sexual, selfharm, violence content in text.
  • Image moderation: Detect hate speech, sexual, selfharm, violence content in images.

Coming back to example we covered yesterday – moderating the text content – we can alternate the filtering to suit your needs. 

Click through to see how it works.

Comments closed

Deployment Parameters in Azure AI Foundry

Published 2024-12-06 by Kevin Feasel

Tomaz Kastrun continues a series on Azure AI:

Give the model instructions about how it should behave and any context it should reference when generating a response. You can describe the assistant’s personality, tell it what it should and shouldn’t answer, and tell it how to format responses. There’s no token limit for this section, but it will be included with every API call, so it counts against the overall token limit

Click through for a description of each part of the deployment parameters section.

Comments closed

Deployment in Azure AI Foundry

Published 2024-12-05 by Kevin Feasel

Tomaz Kastrun continues a series on Azure AI:

When you are in Azure AI Foundry, on the left navigation bar, select “Model Catalog”.

For this demo, I will be selecting multimodal model “gpt-4” that can work with images and text.

Click “> Deploy” and select the deployment type and also customize the deployment details.

Tomaz has some step-by-step instructions, a bit of detail on deployment types, and a bit of info on how to consume the results.

Comments closed