Press "Enter" to skip to content

Category: Replication

GTIDs for Replication in MySQL

Published 2023-08-09 by Kevin Feasel

Aisha Bukar continues a series on replication in MySQL:

Welcome back to another replication series! As a quick reminder, we explored various methods of using MySQL’s replication capabilities in our previous discussions. Initially, we employed the traditional binary-log-based replication approach to set up our replication servers. This involved tracking the binary log file and its positions to facilitate replication.

In this article, we will dive into a more recent and acceptable approach to creating replication – using the Global Transaction Identifiers (GTID) based replication.

Click through to understand how they work and the trade-offs you’ll need to make regarding them.

Comments closed

Multi-Source Replication in MySQL

Published 2023-07-19 by Kevin Feasel

Aisha Bukar continues a series on replication in MySQL:

MySQL’s multi-source replication allows a replica server to receive data from multiple source servers. Let’s say you have a replica server at your workplace, and there are multiple source servers in different locations, you need a way to directly receive data from these source servers to your replica server. This is where the multi-source replication technique comes into play. It allows you to efficiently gather data from various sources and consolidate it on your replica server.

Note that this is quite different from merge replication or peer-to-peer replication in SQL Server and there are some limits to its capabilities. That said, I could see this being really useful for performing ELT into a warehouse: use replication to keep the staging tables in sync and then run a job to perform transformations into facts and dimensions periodically.

Comments closed

Data Syncs between Azure SQL DB and Amazon RDS

Published 2023-07-19 by Kevin Feasel

Joey D’Antoni crosses clouds:

A while back, a client, who host user-facing databases in Azure SQL Database, had a novel problem. One of their customers, had all of their infrastructure in AWS, and wanted to be able to access my client’s data in an RDS instance. There aren’t many options for doing this–replication doesn’t work with Azure SQL Database as a publisher because there’s no SQL Agent. Managed Instance would have been messy from a network perspective, as well as cost prohibitive compared to Azure SQL DB serverless. Even using an ETL tool like Azure Data Factory would have worked, but would have required a rather large amount of dev cycles to check for changed data. Enter Azure Data Sync.

Read on to see what Azure Data Sync is and how it helps solve this problem.

Comments closed

Azure DataSync: Cannot Insert NULL Value

Published 2023-07-07 by Kevin Feasel

Jose Manuel Jurado Diaz does some sleuthing:

In this blog article, we will delve into a common error encountered when synchronizing data with Azure SQL DataSync. We’ll explore the error message “Error #1: SqlException Error Code: -2146232060 – SqlError Number: 515, Message: Cannot insert the value NULL into column ‘ID’, table ‘dbo.Customers’; column does not allow nulls. INSERT fails. SqlError Number: 3621, Message: The statement has been terminated.” We will provide a detailed explanation of the error and its possible causes, followed by a T-SQL code snippet that reproduces the error scenario.

Click through for four possible causes.

Comments closed

Replication from 2000 to 2012

Published 2023-01-12 by Kevin Feasel

Deepthi Gogrui pulls a fast one:

The scenario that I faced was little challenging. We had SQL Server 2012 production server replicating data to a Server 2000 which is used for reporting purposes. Subscriber SQL Server 2000 used by the reporting team were not ready to upgrade the Server as they need to rewrite their entire application as it was using vb6 code. They need a strategy where the data can still be replicated without upgrading the Server.

As I researched, I found that it is not compatible version but planned to test the replication to see if somehow it works. I tested the replication between SQL Server 2012 as a publisher and SQL Server 2000 as subscriber. I was able to setup the transactional replication between the servers for the database but found during the initial initialization snapshot, the ANSI_PADDING setting in the snapshot generated .sch files caused the issue while the distribution job runs. 

Read on for the solution. This turned out to work despite Microsoft’s official guidance that they only support replication between SQL Server instances within two versions of each other.

Comments closed

Bidirectional Transactional Replication and Managed Instances

Published 2022-08-02 by Kevin Feasel

Holger Linke builds a transactional replication topology with a couple of twists:

Bidirectional transactional replication is a specific Transactional Replication topology that allows two SQL Server instances or databases to replicate changes to each other. Each of the two databases publishes data and then subscribes to a publication with the same data from the other database. The “@loopback_detection” feature ensures that changes are only sent to the Subscriber and do not result in the changes being sent back to the Publisher.

The databases that are providing the publication/subscription pairs can be hosted either on the same SQL instance or on two different SQL instances. The SQL instances can either be SQL Server on-premise, SQL Server hosted in a Virtual Machine, SQL Managed Instance on Azure, or a combination of each. You just have to make sure that the instances can connect to each other. If you add a subscription by using the fully-qualified domain name (FQDN), verify that the server name (@@SERVERNAME) of the Subscriber returns the FQDN. If the Subscriber server name does not return the FQDN, changes that originate from that Subscriber may cause primary key violations.

Read on for the scripts.

Comments closed

The Joys of Replication, Example 48,106

Published 2022-07-14 by Kevin Feasel

Garry Bargsley troubleshoots a transactional replication issue:

Use transactional replication to replicate similar data from four databases to one Azure DB.  This sounds pretty straightforward for anyone who has done any replication work.

However, once I had everything setup and working, things stopped working, and it was a head-scratcher as to why.  I made the proper settings configurations on each article, or so I thought.  Let me show you the scenario in more detail.

Read on for the answer.

Comments closed

Finding Articles in a SQL Server Publication

Published 2022-05-23 by Kevin Feasel

Kenneth Fisher disturbs the slumber of the forces of replication:

The other day I was asked to supply a list of all of the tables being replicated into a given database. Now, for those of you that aren’t aware, if I replicate a group of tables from database SourceDB into DestDB I can still have additional tables in DestDB that have nothing to do with the replication. So this wasn’t just a matter of getting a list of tables from the database.

Click through for queries which work for transactional replication as well as merge replication.

Comments closed

Replication in Azure DB for MySQL

Published 2022-02-16 by Kevin Feasel

Arun Sirpal explains how you can set up replication with Azure DB for MySQL:

No doubt there will be a need for you to split off your analytical queries from the main database for performance reasons.

If you have been following me in the past with Azure SQL DB you would use failover group read endpoints. With MySQL we would need to build a replica (read only) to another server. This uses MySQL’s native feature binlog replication which is great to hear. This form is asynchronous.

Read on to see how.

Comments closed

Bidirectional Transactional Replication and Server Names

Published 2022-01-20 by Kevin Feasel

Mousa Janini points out a requirement of bidirectional transactional replication:

The steps to create a Bi-directional replication is simple, and similar to the steps for configuring transnational replication with extra step to enable the @loopback_detection parameter of sp_addsubscription to ensure that changes are only sent to the Subscriber and do not result in the change being sent back to the Publisher.

The most common issue for the Bi-directional replication is when the loop back detection is not working as expected; which results in data conflicts and Primary Key Violations.

Read on to see what is the cause of this problem and what you can do to solve it.

Comments closed