THE EASY FIX: RUN FULL NATIVE BACKUPS EVERY DAY, AND FREAK OUT WHEN THEY FAIL.
Backup failures aren’t normally a big deal, but if you use in-memory OLTP on a standalone server or a failover clustered instance, backup failures are all-out emergencies. You need to immediately find out if the backup just ran out of drive space or lost its network connection, or if you have game-over Hekaton corruption.
Note that you can’t use SAN snapshot backups here. SQL Server won’t read the In-Memory OLTP pages during a snapshot backup, which means they can still be totally corrupt.
This works fine for shops with relatively small databases, say under 500GB.
Brent also has a Connect item to fix this. Upvote if you have memory-optimized, durable tables.
If you’ve been following along in this series, you’ll have created the necessary components of the service architecture, namely the message types, contract, queues, and services. You will also have started a conversation between your initiator and target services. You’re finally ready to start sending and receiving messages in your Service Broker app. Whoohoo! Let’s get to it!
Here’s what we’ve been waiting for, and it’s a good example going through message handling. I also hope Colleen handles poison pill messages and other Service Broker-related traps.
It’s really tempting to implement cool-sounding features. It’s really tempting to hyper-tune solutions to be the absolutely perfect, most-optimal solution. But it takes a real expert to realize when you’re over-engineering a solution.
Take a moment to appreciate your own restraint. Appreciate all the features that you didn’t implement because you didn’t have to. Be happy that you looked at the big picture and decided the best solution was the one you were able to support.
It’s definitely an interesting approach to the T-SQL Tuesday challenge, so full credit for that.
In the field, I see a lot of people using Availability Groups to have a near real-time replica for reporting. I talked a little bit about this above. What isn’t mentioned here is you have to maintain a Windows Failover Cluster, Quorum, Active Directory (Unless using Windows 2016 Preview) and more. This gets you a replica that is just a copy of the database. What does this mean? You cannot change database objects like security, indexes, etc. Also, what if you don’t need the whole database(s) for reporting? If not, you can replicate only the data you truly need.
So, let’s recap here. You only have to replicate the data that you need. You can have different security and indexes on your reporting subscriber database(s). The reporting subscriber database can be scaled up or down as needed based on your needs. The reporting database can now be an Azure Database. Folks, I call this a huge win!
There’s a lot more replication love out there than I’d expect. John promises to follow up with a guide on how to implement this, so keep an eye out for that.
Good Change Management makes your team smarter. When you change things a lot, things are going to break sometimes. If you’ve done a good change request, you’ll know exactly what to do when something breaks: either you’ll roll the change back, or have a Plan B to execute on. Good change requests also mean that business owners understand the risks of the changes and have approved them, and that teammates have reviewed them: good changes aren’t done in isolation.
Change Management isn’t just for IT. If you’re a developer who deploys changes to production, you need this as well.
Change management doesn’t need to be a complex endeavor. It should, however, give you enough information to save yourself if things get crazy.
With the end of SQL Server 2005, we also will soon see the end of database mirroring. There is a new feature releasing with SQL Server 2016 called Basic Availability Groups. This is the replacement for database mirroring. The use cases and limitations will appear very similar to database mirroring but it will use the Availability Group technology. In theory this will be like a stim-pack for the database mirroring feature while leaving it available in Standard Edition. Let’s cross our fingers that the Windows Failover Cluster components don’t slow down the failovers like it did with AGs.
A bold choice, but that “available in standard edition” thing is huge for smaller organizations which can’t afford Enterprise (especially with The Licensing Changes of 2012).
In the end I decided it would be fun to post about one of the newer features;Delayed Durability. To understand DD (delayed durability) you have to understand how SQL Server implements durability. To start with there are two pieces of any data change. The actual data page and any log records that are created. In order to keep SQL Server as fast as possible the data pages are changed in memory and then written to disk as part of a background process. Unfortunately this means that you could lose those changes if the server crashes or shuts down before they are written to disk. In order to avoid this, and keep the changes durable, the transaction can not complete until the log information is actually written to disk. This means that one of the big bottlenecks in any given transaction is writing that log record to disk. The DD option basically changes that write from a synchronous to an asynchronous operation. The log writes are bundled together in 60k chunks before being written to disk. This can (depending on your work load) provide a huge increase in speed. I’m not going to bother discussing that part of it since Aaron Bertrand (b/t) and Melissa Connors (b/t) wrote great posts on just that (Aaron’s and Melissa’s).
What I want to discuss is the actual risk of data loss.
My philosophy on this is simple: if you need delayed durability and you can afford the data loss, then maybe there’s a better data platform. I want my SQL Server like I want my Grateful Dead concert: ACID compliant.
Seriously though, replication has been around since the beginning and it’s not going anywhere. I can’t think of any other feature more prolific than replication. Name another SQL Server HA/DR technology that is as extensible as replication. Replication has gotten a bad rap over the years mostly on anecdotal comments that it “breaks all the time” or “it takes too much time to manage”. I’ve worked in many environments and have setup dozens and dozens of instances of log shipping, mirroring, clusters, availability groups, and replication. From my anecdotal experience, I can tell you I’ve had more trouble with availability groups than I have with replication. If you have a good DBA that understands replication, uses it correctly, and is provided the correct tools (read $ for hardware/infrastructure) replication works just fine. I have setup replication in a global environment in which multiple databases, publications, subscriptions, and agents ran around the clock and without issue.
Replication is very powerful, I agree…but it still breaks. A lot. I’m grateful for its existence and also for the fact that I’m not the one maintaining it…
Chrissy LeMaire shows us how to connect to AD from Ubuntu:
Since 2009, it seems that a couple things have changed in the client realm. In particular, winbindfell out of favor to Likewise Open (which I used to <3) which was bought by BeyondTrust and turned into PowerBroker Open. But that’s since fallen out of favor to the SSSD or “System Security Services Daemon“. SSSD seems pretty cool but everyone hates its name and assume that its name is keeping it from greater adoption.
Sometimes when researching SSSD, you’ll come across a few mentions of FreeIPA which is similar to Active Directory, OpenLDAP, and ApacheDS. Oh, and I recently found out thatSamba4 allows Linux servers to join Active Directory as Domain Controllers (!!) but I can’t tell if it can be a forest of its own (reddit review here).
There are other players I’m leaving out but after a bit of casual research, no others seem to stand out. Ultimately, while there are a number of ways to setup AD/Linux authentication with Ubuntu, it appears that SSSD is the current way to go. Let’s go ahead and set that up.
Cf Ryan Adams and LeMaire’s separate posts back in March on the topic. As Microsoft gets serious about Linux integration, I would love to see them simplify this process significantly, either by updating an existing open-source project (my preference) or creating their own open-source project.
The organization will need to take a step back to understand better their existing status. Are they just starting out? Are other departments which are doing the same thing, perhaps in the local organization or somewhere else in the world? Once the organization understands their state better, they can start to broadly work out the strategy that the Data Lake is intended to provide.
As part of this understanding, the objective of the Data Lake will need to be identified. Is it for data science? Or, for example, is the Data Lake simply to store data in a holding pattern for data discovery? Identifying the objective will help align the vision and the goals, and set the scene for communication to move forward.
I would like to popularize the term Data Swamp for “that place you store a whole bunch of data of dubious origin and value.” It’s the place that you promise management of course you can get the data back…as long as they never actually ask for it or are okay with reading terabytes of flat files from backup tapes. The Data Swamp is the Aristotelian counterpart to the Data Lake, Goofus to its Gallant. It will also, to my estimate, be the more common version.