MySQL Replication for Scale-Out : Managing Consistency of Data

One of the problems with asynchronous replication is managing consistency. To illustrate the problem, let’s imagine you have an e-commerce site where customers can browse for items they want to purchase and put them in a cart. You’ve set up your servers so that when a user adds an item to the cart, the change request goes to the master, but when the web server requests information about the contents of the cart, the query goes to one of the slaves tasked with answering such queries. Since the master is ahead of the slave, it is possible that the change has not reached the slave yet, so a query to the slave will then find the cart empty. This will, of course, come as a big surprise to the customer, who will then promptly add the item to the cart again only to discover that the cart now contains two items, because this time the slave managed to catch up and replicate both changes to the cart. This situation clearly needs to be avoided or you will risk a bunch of irritated customers.

To avoid getting data that is too old, it is necessary to somehow ensure that the data provided by the slave is recent enough to be useful. As you will see, the problem becomes even trickier when a relay server is added to the mix. The basic idea of handling this is to somehow mark each transaction committed on the master, and then wait for the slave to reach that transaction (or later) before trying to execute a query on the slave.

The problem needs to be handled in different ways depending on whether there are any relay slaves between the master and the slave.

1. Consistency in a Nonhierarchal Deployment

When all the slaves are connected directly to the master, it is very easy to check for consistency. In this case, it is sufficient to record the binlog position after the transaction has been committed and then wait for the slave to reach this position using the previously introduced MASTER_POS_WAIT function. It is, however, not possible to get the exact position where a transaction was written in the binlog. Why? Because in the time between the commit of a transaction and the execution of SHOW MASTER STATUS, several events can be written to the binlog.

This does not matter, since in this case it is not necessary to get the exact binlog position where the transaction was written; it is sufficient to get a position that is at or later than the position of the transaction. Since the SHOW MASTER STATUS command will show the position where replication is currently writing events, executing this after the transaction has committed will be sufficient for getting a binlog position that can be used for checking consistency.

Example 1 shows the PHP code for processing an update to guarantee that the data presented is not stale.

Example 1. PHP code for avoiding read of stale data

function fetch_master_pos($server) {
  $result = $server->query('SHOW MASTER STATUS');
  if ($result == NULL)
    return NULL;                              // Execution failed
  $row = $result->fetch_assoc();
  if ($row == NULL)
    return NULL;                              // No binlog enabled
  $pos = array($row['File'], $row['Position']);
  $result->close();
  return $pos;
}

function sync_with_master($master, $slave) {
  $pos = fetch_master_pos($master);
  if ($pos == NULL)
    return FALSE;
  if (!wait_for_pos($slave, $pos[0], $pos[1]))
    return FALSE;
  return TRUE;
}

function wait_for_pos($server, $file, $pos) {
  $result = $server->query("SELECT MASTER_POS_WAIT('$file', $pos)");
  if ($result == NULL)
    return FALSE;                                // Execution failed
  $row = $result->fetch_row();
  if ($row == NULL)
    return FALSE;                                // Empty result set ?!
  if ($row[0] == NULL || $row[0] < 0)
    return FALSE;                                // Sync failed
  $result->close();
  return TRUE;
}

function commit_and_sync($master, $slave) {
  if ($master->commit()) {
    if (!sync_with_master($master, $slave))
      return NULL;                     // Synchronization failed
    return TRUE;                       // Commit and sync succeeded
  }
  return FALSE;                        // Commit failed (no sync done)
}

function start_trans($server) {
  $server->autocommit(FALSE);
}

					  

In Example 1, you see the functions commit_and_sync and start_trans together with the three support functions, fetch_master_pos, wait_for_pos, and sync_with_master. The commit_and_sync function commits a transaction and waits for it to reach a designated slave. It accepts two arguments, a connection object to a master and a connection object to the slave. The function will return TRUE if the commit and the sync succeeded, FALSE if the commit failed, and NULL if the commit succeeded but the synchronization failed (either because there was an error in the slave or because the slave lost the master).

The function works by committing the current transaction and then, if that succeeds, fetching the current master binlog position through SHOW MASTER STATUS. Since other threads may have executed updates to the database between the commit and the call to SHOW MASTER STATUS, it is possible (even likely) that the position returned is not at the end of the transaction, but rather somewhere after where the transaction was written in the binlog. As mentioned earlier, this does not matter from an accuracy perspective, since the transaction will have been executed anyway when we reach this later position.

After fetching the binlog position from the master, the function proceeds by connecting to the slave and executing a wait for the master position using the MASTER_POS_WAIT function. If the slave is running, a call to this function will block and wait for the position to be reached, but if the slave is not running, NULL will be returned immediately. This is also what will happen if the slave stops while the function is waiting, for example, if an error occurs when the slave thread executes a statement. In either case, NULL indicates the transaction has not reached the slave, so it’s important to check the result from the call. If MASTER_POS_WAIT returns 0, it means that the slave had already seen the transaction and therefore synchronization succeeds trivially.

To use these functions, it is sufficient to connect to the server as usual, but then use the functions to start, commit, and abort transactions. Example 2 shows examples of their use in context, but the error checking has been omitted since it is dependent on how errors are handled.

Example 2. Using the start_trans and commit_and_sync functions

require_once './database.inc';

start_trans($master);
$master->query('INSERT INTO t1 SELECT 2*a FROM t1');
commit_and_sync($master, $slave);