<p>The Berkeley DB replication implementation can be affected by network
partitioning problems.</p>
<p>For example, consider a replication group with N members. The network
partitions with the master on one side and more than N/2 of the sites
on the other side. The sites on the side with the master will continue
forward, and the master will continue to accept write queries for the
databases. Unfortunately, the sites on the other side of the partition,
realizing they no longer have a master, will hold an election. The
election will succeed as there are more than N/2 of the total sites
participating, and there will then be two masters for the replication
group. Since both masters are potentially accepting write queries, the
databases could diverge in incompatible ways.</p>
<p>If multiple masters are ever found to exist in a replication group, a
master detecting the problem will return <ahref="../api_reference/C/repmessage.html#repmsg_DB_REP_DUPMASTER"class="olink">DB_REP_DUPMASTER</a>. If
the application sees this return, it should reconfigure itself as a
client (by calling <ahref="../api_reference/C/repstart.html"class="olink">DB_ENV->rep_start()</a>), and then call for an election
(by calling <ahref="../api_reference/C/repelect.html"class="olink">DB_ENV->rep_elect()</a>). The site that wins the election may be
one of the two previous masters, or it may be another site entirely.
Regardless, the winning system will bring all of the other systems into
conformance.</p>
<p>As another example, consider a replication group with a master
environment and two clients A and B, where client A may upgrade to
master status and client B cannot. Then, assume client A is partitioned
from the other two database environments, and it becomes out-of-date
with respect to the master. Then, assume the master crashes and does
not come back on-line. Subsequently, the network partition is restored,
and clients A and B hold an election. As client B cannot win the
election, client A will win by default, and in order to get back into
sync with client B, possibly committed transactions on client B will be
unrolled until the two sites can once again move forward together.</p>
<p>In both of these examples, there is a phase where a newly elected master
brings the members of a replication group into conformance with itself
so that it can start sending new information to them. This can result
in the loss of information as previously committed transactions are
unrolled.</p>
<p>In architectures where network partitions are an issue, applications
may want to implement a heart-beat protocol to minimize the consequences
of a bad network partition. As long as a master is able to contact at
least half of the sites in the replication group, it is impossible for
there to be two masters. If the master can no longer contact a
sufficient number of systems, it should reconfigure itself as a client,
and hold an election. Replication Manager does not currently
implement such a feature, so this technique is only available to Base API
applications.</p>
<p>There is another tool applications can use to minimize the damage in
the case of a network partition. By specifying an <spanclass="bold"><strong>nsites</strong></span>
argument to <ahref="../api_reference/C/repelect.html"class="olink">DB_ENV->rep_elect()</a> that is larger than the actual number of
database environments in the replication group, Base API applications can keep
systems from declaring themselves the master unless they can talk to
a large percentage of the sites in the system. For example, if there
are 20 database environments in the replication group, and an argument
of 30 is specified to the <ahref="../api_reference/C/repelect.html"class="olink">DB_ENV->rep_elect()</a> method, then a system will have
to be able to talk to at least 16 of the sites to declare itself the
master.</p>
<p>Replication Manager uses the value of <spanclass="bold"><strong>nsites</strong></span> (configured by
the <ahref="../api_reference/C/repnsites.html"class="olink">DB_ENV->rep_set_nsites()</a> method) for elections as well as in calculating how
many acknowledgements to wait for when sending a
<ahref="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT"class="olink">DB_REP_PERMANENT</a> message. So this technique may be useful here
as well, unless the application uses the <ahref="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL"class="olink">DB_REPMGR_ACKS_ALL</a> or
<p>Specifying a <spanclass="bold"><strong>nsites</strong></span> argument to <ahref="../api_reference/C/repelect.html"class="olink">DB_ENV->rep_elect()</a> that is
smaller than the actual number of database environments in the
replication group has its uses as well. For example, consider a
replication group with 2 environments. If they are partitioned from
each other, neither of the sites could ever get enough votes to become
the master. A reasonable alternative would be to specify a
<spanclass="bold"><strong>nsites</strong></span> argument of 2 to one of the systems
and a <spanclass="bold"><strong>nsites</strong></span>
argument of 1 to the other. That way, one of the systems could win
elections even when partitioned, while the other one could not. This
would allow one of the systems to continue accepting write
queries after the partition.</p>
<p>In a 2-site group, Replication Manager by default reacts to the loss of
communication with the master by observing a strict majority rule that
prevents the survivor from taking over. Thus it avoids multiple masters and
the need to unroll some transactions if both sites are running but cannot
communicate. But it does leave the group in a read-only state until both
sites are available. If application availability while one site is down is a
priority and it is acceptable to risk unrolling some transactions, there
is a configuration option to turn off the strict majority rule and allow
the surviving client to declare itself to be master. See the <ahref="../api_reference/C/repconfig.html"class="olink">DB_ENV->rep_set_config()</a>
method <ahref="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT"class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag for more information.</p>
