Press "Enter" to skip to content

Category: Containers

Kafka Docker on Kubernetes

Published 2019-07-11 by Kevin Feasel

Bill Ward gives us a step-by-step set of instructions for installing Kafka Docker on Kubernetes:

In this ultimate guide I will give you a simple step-by-step tutorial on installing Kafka Docker on Kubernetes. This post includes a complete video walk-through.

There has been a lot of interest lately about deploying Kafka to a Kubernetes cluster. If you are wanting to take the deep dive yourself then you found the right article. Now that we have Kafka Docker, deploying a Kafka cluster to Kubernetes is a snap.

This makes it even easier to get started with Kafka in a development environment.

Comments closed

Spark and dotnet in a Single Container

Published 2019-07-11 by Kevin Feasel

Ed Elliott shows how you can combine Spark and .NET Core in a single Docker container:

This is quite new syntax in docker and you need at least docker 17.05 (client and daemon), after the images “FROM blah” you can specify a name “core” in this case, then later you can copy from the first image to the second using “–from=” on the “COPY” command.

In this dockerfile I have added Spark 2.4.3 and the default environment variables we need to get spark running, if you grab this dockerfile and run “docker build -t dotnet-spark .” you should get an images you can then run which includes the dependencies for dotnet as well as spark.

Ed includes all of the scripts needed to test this out, too.

Comments closed

Converting Docker Compose Files with Kompose

Published 2019-07-11 by Kevin Feasel

Andrew Pruski shows how you can convert a Docker compose file to something which Kubernetes can read using Kompose:

This will spin up one container running SQL Server 2019 CTP 3.1, accept the EULA, set the SA password, and set the default location for the database data/log/backup files using named volumes created on the fly.

Let’s convert this using Kompose and deploy to a Kubernetes cluster.

To get started with Kompose first install by following the instructions here. I installed on my Windows 10 laptop so I downloaded the binary and added to my PATH environment variable.

It looks pretty straightforward to use; check it out.

Comments closed

Chaos Engineering and KubeInvaders

Published 2019-07-05 by Kevin Feasel

Andrew Pruski wants to play a game:

KubeInvaders allows you to play Space Invaders in order to kill pods in Kubernetes and watch new pods be created (this actually might be my favourite github repo of all time).

I demo SQL Server running in Kubernetes a lot so really wanted to get this working in my Azure Kubernetes Service cluster. Here’s how you get this up and running.

I got to see Andrew show it off at SQL Saturday Cork and it was as fun as you’d expect.

Comments closed

Installing SQL Server on Windows Containers

Published 2019-07-03 by Kevin Feasel

Mohammad Darab shows how you can install SQL Server on a Windows Container:

Today, Microsoft released Windows Containers, currently in the Early Adoption Program here. You can use the same Docker Desktop for Windows but now you can use Windows Containers to deploy SQL Server (as you will see below).

This is primarily for dev/test environments. This is in no way production ready. Think about a developer asking for a SQL instance so that they can run their code against, etc. You can now easily, and quickly, spin up a container running SQL Server. Once they are done, you can remove it or stop it.

The downside is that your Docker Desktop for Windows is now stuck running Windows containers…

Comments closed

Building a Big Data Cluster

Published 2019-06-26 by Kevin Feasel

Mohammad Darab continues a series on SQL Server Big Data Clusters in Azure Kubernetes Service:

To kick off the Big Data Cluster “Default configuration” creation, we will execute the following Powershell command:

mssqlctl cluster create

That will first prompt us to accept the license terms. Type y and Enter. 

Mohammad takes us through the default installation, which requires only a few parameters before it can go on its merry way.

Comments closed

Feeding Kubernetes Log Data to Logstash and Kibana

Published 2019-06-24 by Kevin Feasel

Aayushi Johari shows how you can stand up a Kubernetes cluster and review log data using Logstash and Kibana:

In this article, you will learn how to publish Kubernetes cluster events data to Amazon Elastic Search using Fluentd logging agent. The data will then be viewed using Kibana, an open-source visualization tool for Elasticsearch. Amazon ES consists of integrated Kibana integration.

We will walk you through with the following process:
– Creating a Kubernetes Cluster
– Creating an Amazon ES cluster
– Deploy Fluentd logging agent on Kubernetes cluster
– Visualize kubernetes date in Kibana

Click through for the full article.

Comments closed

Building an AKS Cluster

Published 2019-06-24 by Kevin Feasel

Mohammad Darab continues a series on Big Data Clusters by creating a Kubernetes pod in Azure Kubernetes Service:

Next, we will create a resource group by executing the following command:
az group create –name nameOfMyresourceGroup –location eastus2

Once you execute the above command, you can go into the Azure portal and refresh your resource group pane and see the newly created resource group.

Once that is setup, it’s time to create the actual Kubernetes cluster.

Click through for the full set of instructions.

Comments closed

Running Confluent Platform with .NET

Published 2019-06-20 by Kevin Feasel

Niels Berglund shows how you can install Confluent Platform as a Docker container and use the .NET client against it:

What we see in Figure 16 are the various project related files, including the source file Program.cs. What is missing now is a Kafka client. For .NET there exists a couple of clients, and theoretically, you can use any one of them. However, in practice, there is only one, and that is the Confluent Kafka DotNet client. The reason I say this is because it has the best parity with the original Java client. The client has NuGet packages, and you install it via VS Code’s integrated terminal: dotnet add package Confluent.Kafka --version 1.0.1.1:

Definitely use the Confluent client. The others were from a time when there was no official driver; most aren’t even maintained anymore.

Comments closed

Docker Desktop Resource Limits

Published 2019-06-20 by Kevin Feasel

Andrew Pruski lays out the default resource limits for Linux and Windows containers using Docker Desktop on Windows:

Docker Desktop is a great product imho. The ability to run Windows and Linux containers locally is great for development and has allowed me to really dig into SQL Server on Linux. I also love the fact that I no longer need to install SQL 2016/2017, I can run it in Windows containers.

However there are some differences between how Windows and Linux containers run in Docker Desktop. One of those differences being the default resource limits that are set.

Read on to see how they differ and how to deal with Windows container resources.

Comments closed