Press "Enter" to skip to content

Curated SQL Posts

Securing Elasticsearch And Kibana

Published 2016-09-13 by Kevin Feasel

Vikash Selvin shows how to secure instances of Elasticsearch and Kibana:

The most popular options for securing Elasticsearch and Kibana are compared in the table below.

Shield is a security plugin developed by the same company that developed Elasticsearch. It allows you to easily protect this data with a username and password while simplifying your architecture. Advanced security features like encryption, role-based access control, IP filtering, and auditing are also available when you need them.

NGINX is an open source web server. It can act as a proxy server and can do load balancing, among other things. In combination with LUA and external scripts, it can be used for securing Elasticsearch and Kibana. We will be using this approach in this tutorial.

Searchguard is an open source alternative for Shield. It provides almost all the same functionalities as Shield, except for some features like LDAP authentication. However, these features are available in the paid variant.

Click through for a detailed NGINX setup.

Comments closed

Shiny 0.14 Released

Published 2016-09-13 by Kevin Feasel

Winston Chang reports that Shiny version 0.14 is now available:

If your Shiny app contains computations that take a long time to complete, a progress bar can improve the user experience by communicating how far along the computation is, and how much is left. Progress bars were added in Shiny 0.10.2. In Shiny 0.14, we’ve changed them to use the notifications system, which gives them a different look.

Important note: If you were already using progress bars and had customized them with your own CSS, you can add the style = "old" argument to yourwithProgress() call (or Progress$new()). This will result in the same appearance as before. You can also call shinyOptions(progress.style = "old") in your app’s server function to make all progress indicators use the old styling.

It looks like they’ve made some good progress with Shiny.

Comments closed

Administrative Scripts

Published 2016-09-13 by Kevin Feasel

Slava Murygin has ten Powershell scripts to help administer a SQL Server instance:

Script #5. Read SQL Server Error Log file.

That is extremely important troubleshooting script. When you start/restart the SQL Server service and it does not come up, you can run this script to see what was going on during the SQL Server startup and what was the problem (just note that value of “$SQLInstancePath” must be pre-set by previous script):

Click through for all of the scripts.

Comments closed

Understand Your Testing Utilities

Published 2016-09-13 by Kevin Feasel

David Klee shows an issue with using iperf for load testing:

The load test utility had maxed out the compute resource that it had available, due to internal limitations within iperf itself.  It’s a shame that this utility is not multi-threaded, because I think we could have a much greater result of improvement on this system.

Monitor the utilities that you’re using to do load testing, because limitations like this might skew your results!

Everything eventually hits a bottleneck.  In David’s case, the bottleneck was in the testing tool itself.

Comments closed

Query Performance Insight

Published 2016-09-13 by Kevin Feasel

Arun Sirpal discusses Query Performance Insight in Azure SQL Databases:

Here you will be presented with the TOP X queries based on CPU, Duration or Execution count. You will have the ability to change the time period of analysis, return 5, 10 or 20 queries using aggregations SUM, MAX or AVG.

So let’s look at what information is provided based on queries with high AVG duration over the last 6 hours.

Looks like an interesting way to get information on the few most heavily used queries.

Comments closed

Non-Clustered Index Design

Published 2016-09-13 by Kevin Feasel

Derik Hammer has an article on non-clustered index design:

In general, non-clustered indexes are a positive force in your database. Indexes are a form of data duplication, however, which costs disk space. In addition, non-clustered indexes need to be maintained. This can increase the number of writes which occur during INSERT or UPDATE operations and increase the number of index rebuilds or reorganizations that need to be performed.

Create non-clustered indexes to support all of your queries but be careful not to create duplicates and regularly purge indexes which are no longer necessary.

Worth reading the whole thing.

Comments closed

SQL Agent Job Durations As TimeSpans

Published 2016-09-13 by Kevin Feasel

Rob Sewell shows us how to convert the SQL Agent job duration into a .NET TimeSpan using Powershell:

The first job took 15 hours 41 minutes  53 seconds, the second 1 minute 25 seconds, the third 21 seconds. This makes it quite tricky to calculate the duration in a suitable datatype. In T-SQL people use scripts like the following from MSSQLTips.com

((run_duration/10000*3600 + (run_duration/100)%100*60 + run_duration%100 + 31 ) / 60)  as ‘RunDurationMinutes’

I wish that some version of SQL Server would fix this “clever” duration.  We’ve had the time datatype since 2008; at least add a new column with run duration as a time value if you’re that concerned with backwards compatibility.

Comments closed

More NFL Alerts

Published 2016-09-12 by Kevin Feasel

Allison Tharp has an update to her NFL Alerts Python script:

Next, I wanted to make the alerts be a little more meaningful.  The alert for a scoring play was already pretty good – it sends something like: BUF – Q4 – TD – J.Boykin 4 yd. pass from C.Jones (pass failed) Drive: 8 plays, 83 yards in 1:08 IND (19) at BUF (18).  This is good, and in fact it is what I want the rest of the alerts to look like.  However, I’d like the subject of the email to have the name of the team that scored (before it was just ‘Scoring Play’).

To do that, I needed to find out how to get the name of the scoring team.  This was a little tricky because the documentation for the nflgame library, though pretty good, doesn’t give a good indication on how to find this.

Read on for more details, including specifics on turnovers and penalties.

Comments closed

The SQLOS Scheduler

Published 2016-09-12 by Kevin Feasel

Ewald Cress builds an extended metaphor for the SQLOS scheduler:

When the time comes for a bunny to pass the battery, it may be out of free choice, or it might be because its script went down a path where passing it along is The Thing To Do. At this juncture, the team’s collective memory and playbook comes to the fore, and agreed rules dictate who the battery goes to. It doesn’t really matter what those rules are for the moment. The important point is that control is transferred by the players themselves using shared rules and a team whiteboard tracking who is ready to go, which team member might be most deserving, who has been waiting the longest etc. This code of conduct and state, this bushido of bunny bonhomie, is what we call a scheduler.

This is building up to something big…

Comments closed