A Fine Slice Of SQL Server
Tomaz Kastrun is already counting down to next year:
Progress bar showing the number of days until end of the year. Yes, only based on the days and not hour. And needless to say, the progress is soothing using ASCII charts.
So the function is fair to say, simple and useless.
In fairness, the function isn’t that useful but the technique is, particularly if you’re installing something or running a long process.
Comments closedThe Hadoop in Real World team give us three methods (and one synonym) to organize results in Hive:
Hive provides 3 options to order or sort the result of records – order by, sort by, cluster by and distribute by. Which option you choose has performance implications. So it is important to understand the difference between the options and choose the right one for the use case at hand.
Click through for a high-level overview of the techniques.
Comments closedKeqiu Hu, et al, take us through a problem of scale:
At LinkedIn, we use Hadoop as our backbone for big data analytics and machine learning. With an exponentially growing data volume, and the company heavily investing in machine learning and data science, we have been doubling our cluster size year over year to match the compute workload growth. Our largest cluster now has ~10,000 nodes, one of the largest (if not the largest) Hadoop clusters on the planet. Scaling Hadoop YARN has emerged as one of the most challenging tasks for our infrastructure over the years.
In this blog post, we will first discuss the YARN cluster slowdowns we observed as we approached 10,000 nodes and the fixes we developed for these slowdowns. Then, we will share the ways we proactively monitored for future performance degradations, including a now open-sourced tool we wrote called DynoYARN, which reliably forecasts performance for YARN clusters of arbitrary size. Finally, we will describe Robin, an in-house service which enables us to horizontally scale our clusters beyond 10,000 nodes.
Read on to learn about the problems they experienced and how they resolved them.
Comments closedSteve REzhener has a warning for us:
Amazon Web Services Elastic Cloud Computing (a.k.a. EC2) is a service that lets anyone with a credit card rent a virtualized server from Amazon. To cater to different clients’ needs, AWS provides various instance types that are either general instance or specific-purpose instances (focused on CPU, RAM, IO). You can see the different types in Fig 1. This blog post is going to talk about a storage optimized instance. the I3 instance type family, its little-known problem, and the solution in the form of Elastic Block Storage (a.k.a. EBS).
Click through for the warning, more explanation, and what you can do about it. H/T the SQLServerCentral newsletter.
Comments closedJeet Kainth takes us through the process of working with variables in DAX:
Variables can simplify your DAX code, help with debugging and help improve performance. To use variables in your DAX measure you need to declare a variable using the VAR keyword, give the variable a name, and then assign an expression to the variable. You can use multiple variables in a measure but when using variables you must use a RETURN statement to specify the final output to be returned.
Read on for a demonstration, as well as several examples of how you can use variables to make your DAX-writing life easier.
Comments closedErik Darling looks at exchange spills:
There are quite high waits on PAGEIOLATCH_EX, SLEEP_TASK, and SLEEP_BPOOL_STEAL rounding out the top five. This is quite interesting, because I’ve never explicitly thought of PAGEIOLATCH_EX waits in the context of exchange spills. Normally, I think of them when queries read pages from disk into memory for modification.
Going down the line, SLEEP_TASK is familiar from our time spent with hash spills, but SLEEP_BPOOL_STEAL is so far undocumented anywhere.
Erik also does the math on this query and recommends that you not write a query like this one.
Comments closedChris Webb has a follow-up to a prior post:
My post earlier this year on enabling query folding when using SQL queries as a data source in Power Query provoked a lot of interest. This post adds one more useful detail: how to preserve the original data types of the columns in your query when using this technique with SQL Server-related sources.
Read on for an example using the DATE datatype.