Press "Enter" to skip to content

Day: November 2, 2020

Dotnet-Spark UDFs and Missing Shared State

Published 2020-11-02 by Kevin Feasel

Ed Elliott uncovers a mystery:

To understand this we need to take a look at how we can create a UDF in .NET that is called by the Java VM Apache Spark code because, that is logically, what happens. In our application we call into Apache Spark and ask it to do things like read from a file, run some transformation and write files back out again. With UDF’s, we ask Spark to run a UDF and Spark comes back to our UDF, passing it some data and asks the UDF to execute but the Java VM does not understand how to execute .NET code.

Read the whole thing.

Comments closed

Migrating SQL Server Container Images to GitHub

Published 2020-11-02 by Kevin Feasel

Andrew Pruski has moved some images around:

A couple of months ago Docker announced that they would be implementing a 6 month retention policy for unused images in the Docker Hub.

This was due to kick in on the 1st of November but has now been pushed back until mid 2021.

I’ve had multiple Windows SQL Server container images up on the Docker Hub for years now. It’s been a great platform and I’m very thankful to them for hosting my images.

That being said, I want to make sure that the images that I’ve built are always going to be available for the community so I have pushed my SQL Server images to the Github Container Registry.

I guess I should do the same.

Comments closed

Parallelism and Nested Loops Joins

Published 2020-11-02 by Kevin Feasel

Erik Darling talks about the intersection of two performance tuning topics:

Yesterday we saw a case where the Gather Streams operator was costed quite highly, and it prevented a parallel plan from being chosen, despite the parallel plan in this case being much faster.

It’s important to note that costing for plans is not a direct reflection of actual time or effort, nor is it accurate to your local configuration.

They’re estimates used to come up with a plan. When you get an actual plan, there are no added-in “Actual Cost” metrics.

Read on to see how you can monkey’s paw your way through this problem by introducing exciting, new problems.

Comments closed

Avoid Backup-and-Restore of SSISDB for Deployment

Published 2020-11-02 by Kevin Feasel

Andy Leonard recommends not using backup-and-restore as an approach of moving SSIS packages around:

First, please do not misunderstand. You should back up SSISDB just like you back up all other databases – especially in Production. You should also conduct Disaster Recovery exercises in which you restore SSISDB from the latest backup, or avail yourself of Always On availability groups and / or Windows Server Failover Clustering.

With that caveat in mind, read on to see why.

Comments closed

Query Processor Ran Out of Internal Resources

Published 2020-11-02 by Kevin Feasel

Andy Galbraith troubleshoots a problem:

 Unfortunately a common error in many of our client environments is this:

—

Error: 8623, Severity: 16, State: 1.

The query processor ran out of internal resources and could not produce a query plan. This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partitions. Please simplify the query. If you believe you have received this message in error, contact Customer Support Services for more information.–There are many potential causes for this, and the text of this particular error is very well-written because the primary cause is exactly what is listed – a complex query.

Read on to see how to find this complex query, as well as a few examples of complex queries.

Comments closed

SQL Server Installation Options for Testing Azure DevOps Deployments

Published 2020-11-02 by Kevin Feasel

Kevin Chant looks at the different options available when trying to set up local testing of SQL Server databases using Azure DevOps deployments:

One way you can work around the above scenario is to install multiple virtual machines. Now the first thing you might realize is that this will also take up a lot of compute and storage.

In reality, I use to use this method myself in the past using Hyper-V. To reduce the amount of storage the virtual machines used in Hyper-V I use to used parenting disks.

Since the introduction of containers and Docker this has become a less popular option. However, you can still read an old post of mine with tips in here.

Click through for additional options.

Comments closed