je/docs/TransactionGettingStarted/txnexample_java.html

768 lines
32 KiB
HTML
Raw Permalink Normal View History

2021-06-06 17:46:45 +00:00
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Base API Transaction Example</title>
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
<link rel="start" href="index.html" title="Getting Started with Berkeley DB, Java Edition Transaction Processing" />
<link rel="up" href="wrapup.html" title="Chapter 6. Summary and Examples" />
<link rel="prev" href="wrapup.html" title="Chapter 6. Summary and Examples" />
<link rel="next" href="txnexample_dpl.html" title="DPL Transaction Example" />
</head>
<body>
<div xmlns="" class="navheader">
<div class="libver">
<p>Library Version 12.2.7.5</p>
</div>
<table width="100%" summary="Navigation header">
<tr>
<th colspan="3" align="center">Base API Transaction Example</th>
</tr>
<tr>
<td width="20%" align="left"><a accesskey="p" href="wrapup.html">Prev</a> </td>
<th width="60%" align="center">Chapter 6. Summary and Examples</th>
<td width="20%" align="right"> <a accesskey="n" href="txnexample_dpl.html">Next</a></td>
</tr>
</table>
<hr />
</div>
<div class="sect1" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h2 class="title" style="clear: both"><a id="txnexample_java"></a>Base API Transaction Example</h2>
</div>
</div>
</div>
<div class="toc">
<dl>
<dt>
<span class="sect2">
<a href="txnexample_java.html#txnguideexample">TxnGuide.java</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="txnexample_java.html#payloaddata">PayloadData.java</a>
</span>
</dt>
<dt>
<span class="sect2">
<a href="txnexample_java.html#dbwriter">DBWriter.java</a>
</span>
</dt>
</dl>
</div>
<p>
The following Java code provides a fully functional example of a
multi-threaded transactional JE application.
The example opens an environment and database, and then creates 5
threads, each of which writes 500 records to the database. The keys
used for these writes are pre-determined strings, while the data is
a class that contains randomly generated data. This means that the actual
data is arbitrary and therefore uninteresting; we picked it only
because it requires minimum code to implement and therefore will
stay out of the way of the main points of this example.
</p>
<p>
Each thread writes 10 records under a single transaction
before committing and writing another 10 (this is repeated 50
times). At the end of each transaction, but before committing, each
thread calls a function that uses a cursor to read every record in
the database. We do this in order to make some points about
database reads in a transactional environment.
</p>
<p>
Of course, each writer thread performs deadlock detection as
described in this manual. In addition, normal recovery is performed
when the environment is opened.
</p>
<p>
To implement this example, we need three classes:
</p>
<div class="itemizedlist">
<ul type="disc">
<li>
<p>
<code class="literal">TxnGuide.java</code>
</p>
<p>
This is the main class for the application. It performs
environment and database management, spawns threads, and
creates the data that is placed in the database. See <a class="xref" href="txnexample_java.html#txnguideexample" title="TxnGuide.java">TxnGuide.java</a> for implementation details.
</p>
</li>
<li>
<p>
<code class="literal">DBWriter.java</code>
</p>
<p>
This class extends <code class="literal">java.lang.Thread</code>, and
as such it is our thread implementation. It is responsible
for actually reading and writing to the database. It also
performs all of our transaction management. See <a class="xref" href="txnexample_java.html#dbwriter" title="DBWriter.java">DBWriter.java</a> for
implementation details.
</p>
</li>
<li>
<p>
<code class="literal">PayloadData.java</code>
</p>
<p>
This is a data class used to encapsulate several data
fields. It is fairly uninteresting, except that the usage
of a class means that we have to use the bind APIs to
serialize it for storage in the database. See <a class="xref" href="txnexample_java.html#payloaddata" title="PayloadData.java">PayloadData.java</a> for
implementation details.
</p>
</li>
</ul>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="txnguideexample"></a>TxnGuide.java</h3>
</div>
</div>
</div>
<p>
The main class in our example application is used to open and
close our environment and database. It also spawns all the
threads that we need. We start with the normal series
of Java package and import statements, followed by our class
declaration:
</p>
<pre class="programlisting">// File TxnGuide.java
package je.txn;
import com.sleepycat.bind.serial.StoredClassCatalog;
import com.sleepycat.je.Database;
import com.sleepycat.je.DatabaseConfig;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.Environment;
import com.sleepycat.je.EnvironmentConfig;
import java.io.File;
import java.io.FileNotFoundException;
public class TxnGuide { </pre>
<p>
Next we declare our class' private data members. Mostly these are used
for constants such as the name of the database that we are opening and
the number of threads that we are spawning. However, we also declare
our environment and database handles here.
</p>
<pre class="programlisting"> private static String myEnvPath = "./";
private static String dbName = "mydb.db";
private static String cdbName = "myclassdb.db";
// DB handles
private static Database myDb = null;
private static Database myClassDb = null;
private static Environment myEnv = null;
private static final int NUMTHREADS = 5; </pre>
<p>
Next, we implement our <code class="function">usage()</code> method. This
application optionally accepts a single command line argument which is
used to identify the environment home directory.
</p>
<pre class="programlisting"> private static void usage() {
System.out.println("TxnGuide [-h &lt;env directory&gt;]");
System.exit(-1);
} </pre>
<p>
Now we implement our <code class="function">main()</code> method. This method
simply calls the methods to parse the command line arguments and open
the environment and database. It also creates the stored class catalog
that we use for serializing the data that we want to store in our
database. Finally, it creates and then joins the database writer
threads.
</p>
<pre class="programlisting"> public static void main(String args[]) {
try {
// Parse the arguments list
parseArgs(args);
// Open the environment and databases
openEnv();
// Get our class catalog (used to serialize objects)
StoredClassCatalog classCatalog =
new StoredClassCatalog(myClassDb);
// Start the threads
DBWriter[] threadArray;
threadArray = new DBWriter[NUMTHREADS];
for (int i = 0; i &lt; NUMTHREADS; i++) {
threadArray[i] = new DBWriter(myEnv, myDb, classCatalog);
threadArray[i].start();
}
// Join the threads. That is, wait for each thread to
// complete before exiting the application.
for (int i = 0; i &lt; NUMTHREADS; i++) {
threadArray[i].join();
}
} catch (Exception e) {
System.err.println("TxnGuide: " + e.toString());
e.printStackTrace();
} finally {
closeEnv();
}
System.out.println("All done.");
} </pre>
<p>
Next we implement <code class="function">openEnv()</code>. This method is used
to open the environment and then a database in that environment. Along
the way, we make sure that the transactional subsystem is correctly
initialized.
</p>
<p>
For the database open, notice that we open the database such that it
supports duplicate records. This is required purely by the data that
we are writing to the database, and it is only necessary if you run the
application more than once without first deleting the environment.
</p>
<pre class="programlisting"> private static void openEnv() throws DatabaseException {
System.out.println("opening env");
// Set up the environment.
EnvironmentConfig myEnvConfig = new EnvironmentConfig();
myEnvConfig.setAllowCreate(true);
myEnvConfig.setTransactional(true);
// Environment handles are free-threaded by default in JE,
// so we do not have to do anything to cause the
// environment handle to be free-threaded.
// Set up the database
DatabaseConfig myDbConfig = new DatabaseConfig();
myDbConfig.setAllowCreate(true);
myDbConfig.setTransactional(true);
myDbConfig.setSortedDuplicates(true);
// Open the environment
myEnv = new Environment(new File(myEnvPath), // Env home
myEnvConfig);
// Open the database. Do not provide a txn handle. This open
// is auto committed because DatabaseConfig.setTransactional()
// is true.
myDb = myEnv.openDatabase(null, // txn handle
dbName, // Database file name
myDbConfig);
// Used by the bind API for serializing objects
// Class database must not support duplicates
myDbConfig.setSortedDuplicates(false);
myClassDb = myEnv.openDatabase(null, // txn handle
cdbName, // Database file name
myDbConfig);
} </pre>
<p>
Finally, we implement the methods used to close our environment and
databases, parse the command line arguments, and provide our class
constructor. This is fairly standard code and it is mostly
uninteresting from the perspective of this manual. We include it here
purely for the purpose of completeness.
</p>
<pre class="programlisting"> private static void closeEnv() {
System.out.println("Closing env and databases");
if (myDb != null ) {
try {
myDb.close();
} catch (DatabaseException e) {
System.err.println("closeEnv: myDb: " +
e.toString());
e.printStackTrace();
}
}
if (myClassDb != null ) {
try {
myClassDb.close();
} catch (DatabaseException e) {
System.err.println("closeEnv: myClassDb: " +
e.toString());
e.printStackTrace();
}
}
if (myEnv != null ) {
try {
myEnv.close();
} catch (DatabaseException e) {
System.err.println("closeEnv: " + e.toString());
e.printStackTrace();
}
}
}
private TxnGuide() {}
private static void parseArgs(String args[]) {
for(int i = 0; i &lt; args.length; ++i) {
if (args[i].startsWith("-")) {
switch(args[i].charAt(1)) {
case 'h':
myEnvPath = new String(args[++i]);
break;
default:
usage();
}
}
}
}
} </pre>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="payloaddata"></a>PayloadData.java</h3>
</div>
</div>
</div>
<p>
Before we show the implementation of the database writer thread, we
need to show the class that we will be placing into the database. This
class is fairly minimal. It simply allows you to store and retrieve an
<code class="literal">int</code>, a <code class="literal">String</code>, and a
<code class="literal">double</code>. We will be using the JE bind API from
within the writer thread to serialize instances of this class and place
them into our database.
</p>
<pre class="programlisting">package je.txn;
import java.io.Serializable;
public class PayloadData implements Serializable {
private int oID;
private String threadName;
private double doubleData;
PayloadData(int id, String name, double data) {
oID = id;
threadName = name;
doubleData = data;
}
public double getDoubleData() { return doubleData; }
public int getID() { return oID; }
public String getThreadName() { return threadName; }
} </pre>
</div>
<div class="sect2" lang="en" xml:lang="en">
<div class="titlepage">
<div>
<div>
<h3 class="title"><a id="dbwriter"></a>DBWriter.java</h3>
</div>
</div>
</div>
<p>
<code class="literal">DBWriter.java</code> provides the implementation
for our database writer thread. It is responsible for:
</p>
<div class="itemizedlist">
<ul type="disc">
<li>
<p>
All transaction management.
</p>
</li>
<li>
<p>
Responding to deadlock exceptions.
</p>
</li>
<li>
<p>
Providing data to be stored into the database.
</p>
</li>
<li>
<p>
Serializing and then writing the data to the database.
</p>
</li>
</ul>
</div>
<p>
In order to show off some of the ACID properties provided
by JE's transactional support,
<code class="literal">DBWriter.java</code> does some things in a less
efficient way than you would probably decide to use in a
true production application. First, it groups 10 database
writes together in a single transaction when you could just
as easily perform one write for each transaction. If you
did this, you could use auto commit for the individual
database writes, which means your code would be slightly
simpler and you would run a <span class="emphasis"><em>much</em></span>
smaller chance of encountering blocked and deadlocked
operations. However, by doing things this way, we are able
to show transactional atomicity, as well as deadlock
handling.
</p>
<p>
At the end of each transaction,
<code class="literal">DBWriter.java</code> runs a cursor over the
entire database by way of counting the number of records
currently existing in the database. There are better ways
to discover this information, but in this case we want to
make some points regarding cursors, transactional
applications, and deadlocking (we get into this in more
detail later in this section).
</p>
<p>
To begin, we provide the usual package and import statements, and we declare our class:
</p>
<pre class="programlisting">package je.txn;
import com.sleepycat.bind.EntryBinding;
import com.sleepycat.bind.serial.StoredClassCatalog;
import com.sleepycat.bind.serial.SerialBinding;
import com.sleepycat.bind.tuple.StringBinding;
import com.sleepycat.je.Cursor;
import com.sleepycat.je.CursorConfig;
import com.sleepycat.je.Database;
import com.sleepycat.je.DatabaseEntry;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.Environment;
import com.sleepycat.je.LockMode;
import com.sleepycat.je.LockConflictException;
import com.sleepycat.je.OperationStatus;
import com.sleepycat.je.Transaction;
import java.io.UnsupportedEncodingException;
import java.util.Random;
public class DBWriter extends Thread
{ </pre>
<p>
Next we declare our private data members. Notice that we get handles
for the environment and the database. We also obtain a handle for an
<code class="classname">EntryBinding</code>. We will use this to serialize
<code class="classname">PayloadData</code> class instances (see <a class="xref" href="txnexample_java.html#payloaddata" title="PayloadData.java">PayloadData.java</a>) for storage in
the database. The random number generator that we instantiate is used
to generate unique data for storage in the database. The
<code class="literal">MAX_RETRY</code> variable is used to define how many times
we will retry a transaction in the face of a deadlock. And, finally,
<code class="literal">keys</code> is a <code class="classname">String</code> array that
holds the keys used for our database entries.
</p>
<pre class="programlisting"> private Database myDb = null;
private Environment myEnv = null;
private EntryBinding dataBinding = null;
private Random generator = new Random();
private static final int MAX_RETRY = 20;
private static String[] keys = {"key 1", "key 2", "key 3",
"key 4", "key 5", "key 6",
"key 7", "key 8", "key 9",
"key 10"}; </pre>
<p>
Next we implement our class constructor. The most interesting thing
we do here is instantiate a serial binding for serializing
<code class="classname">PayloadData</code> instances.
</p>
<pre class="programlisting"> // Constructor. Get our DB handles from here
DBWriter(Environment env, Database db, StoredClassCatalog scc)
throws DatabaseException {
myDb = db;
myEnv = env;
dataBinding = new SerialBinding(scc, PayloadData.class);
} </pre>
<p>
Now we implement our thread's <code class="methodname">run()</code> method.
This is the method that is run when <code class="classname">DBWriter</code>
threads are started in the main program (see <a class="xref" href="txnexample_java.html#txnguideexample" title="TxnGuide.java">TxnGuide.java</a>).
</p>
<pre class="programlisting"> // Thread method that writes a series of records
// to the database using transaction protection.
// Deadlock handling is demonstrated here.
public void run () { </pre>
<p>
The first thing we do is get a <code class="literal">null</code> transaction
handle before going into our main loop. We also begin the top transaction loop here that causes our application to
perform 50 transactions.
</p>
<pre class="programlisting"> Transaction txn = null;
// Perform 50 transactions
for (int i=0; i&lt;50; i++) { </pre>
<p>
Next we declare a <code class="literal">retry</code> variable. This is used to
determine whether a deadlock should result in our retrying the
operation. We also declare a <code class="literal">retry_count</code> variable
that is used to make sure we do not retry a transaction forever in the
unlikely event that the thread is unable to ever get a necessary lock.
(The only thing that might cause this is if some other thread dies
while holding an important lock. This is the only code that we have to
guard against that because the simplicity of this application makes it
highly unlikely that it will ever occur.)
</p>
<pre class="programlisting"> boolean retry = true;
int retry_count = 0;
// while loop is used for deadlock retries
while (retry) { </pre>
<p>
Now we go into the <code class="literal">try</code> block that we use for
deadlock detection. We also begin our transaction here.
</p>
<pre class="programlisting"> // try block used for deadlock detection and
// general db exception handling
try {
// Get a transaction
txn = myEnv.beginTransaction(null, null); </pre>
<p>
Now we write 10 records under the transaction that we have just begun.
By combining multiple writes together under a single transaction,
we increase the likelihood that a deadlock will occur. Normally,
you want to reduce the potential for a deadlock and in this case
the way to do that is to perform a single write per transaction. In
other words, we <span class="emphasis"><em>should</em></span> be using auto commit to
write to our database for this workload.
</p>
<p>
However, we want to show deadlock handling and by performing
multiple writes per transaction we can actually observe deadlocks
occurring. We also want to underscore the idea that you can
combing multiple database operations together in a single atomic
unit of work. So for our example, we do the (slightly) wrong thing.
</p>
<p>
Further, notice that we store our key into a
<code class="classname">DatabaseEntry</code> using
<code class="classname">com.sleepycat.bind.tuple.StringBinding</code> to
perform the serialization. Also, when we instantiate the
<code class="classname">PayloadData</code> object, we call
<code class="methodname">getName()</code> which gives us the string
representation of this thread's name, as well as
<code class="methodname">Random.nextDouble()</code> which gives us a random
double value. This latter value is used so as to avoid duplicate
records in the database.
</p>
<pre class="programlisting">
// Write 10 records to the db
// for each transaction
for (int j = 0; j &lt; 10; j++) {
// Get the key
DatabaseEntry key = new DatabaseEntry();
StringBinding.stringToEntry(keys[j], key);
// Get the data
PayloadData pd = new PayloadData(i+j, getName(),
generator.nextDouble());
DatabaseEntry data = new DatabaseEntry();
dataBinding.objectToEntry(pd, data);
// Do the put
myDb.put(txn, key, data);
} </pre>
<p>
Having completed the inner database write loop, we could simply
commit the transaction and continue on to the next block of 10
writes. However, we want to first illustrate a few points about
transactional processing so instead we call our
<code class="function">countRecords()</code> method before calling the transaction
commit. <code class="function">countRecords()</code> uses a cursor to read every
record in the database and return a count of the number of records
that it found.
</p>
<p>
Because
<code class="function">countRecords()</code>
reads every record in the database, if used incorrectly the thread
will self-deadlock. The writer thread has just written 500 records
to the database, but because the transaction used for that write
has not yet been committed, each of those 500 records are still
locked by the thread's transaction. If we then simply run a
non-transactional cursor over the database from within the same
thread that has locked those 500 records, the cursor will
block when it tries to read one of those transactional
protected records. The thread immediately stops operation at that
point while the cursor waits for the read lock it has
requested. Because that read lock will never be released (the thread
can never make any forward progress), this represents a
self-deadlock for the thread.
</p>
<p>
There are three ways to prevent this self-deadlock:
</p>
<div class="orderedlist">
<ol type="1">
<li>
<p>
We can move the call to
<code class="function">countRecords()</code> to a point after the
thread's transaction has committed.
</p>
</li>
<li>
<p>
We can allow <code class="function">countRecords()</code> to
operate under the same transaction as all of the writes
were performed.
</p>
</li>
<li>
<p>
We can reduce our isolation guarantee for the application
by allowing uncommitted reads.
</p>
</li>
</ol>
</div>
<p>
For this example, we choose to use option 3 (uncommitted reads) to avoid
the deadlock. This means that we have to open our cursor handle
so that it knows to perform uncommitted reads.
</p>
<pre class="programlisting"> // commit
System.out.println(getName() + " : committing txn : "
+ i);
// Using uncommitted reads to avoid the deadlock, so
// null is passed for the transaction here.
System.out.println(getName() + " : Found " +
countRecords(null) + " records in the database.");</pre>
<p>
Having performed this somewhat inelegant counting of the records in the
database, we can now commit the transaction.
</p>
<pre class="programlisting"> try {
txn.commit();
txn = null;
} catch (DatabaseException e) {
System.err.println("Error on txn commit: " +
e.toString());
}
retry = false; </pre>
<p>
If all goes well with the commit, we are done and we can move on to the
next batch of 10 records to add to the database. However, in the event
of an error, we must handle our exceptions correctly. The first of
these is a deadlock exception. In the event of a deadlock, we want to
abort and retry the transaction, provided that we have not already
exceeded our retry limit for this transaction.
</p>
<pre class="programlisting"> } catch (LockConflictException le) {
System.out.println("################# " + getName() +
" : caught deadlock");
// retry if necessary
if (retry_count &lt; MAX_RETRY) {
System.err.println(getName() +
" : Retrying operation.");
retry = true;
retry_count++;
} else {
System.err.println(getName() +
" : out of retries. Giving up.");
retry = false;
} </pre>
<p>
In the event of a standard, non-specific database exception, we simply
log the exception and then give up (the transaction is not retried).
</p>
<pre class="programlisting"> } catch (DatabaseException e) {
// abort and don't retry
retry = false;
System.err.println(getName() +
" : caught exception: " + e.toString());
e.printStackTrace(); </pre>
<p>
And, finally, we always abort the transaction if the transaction handle
is not null. Note that immediately after committing our transaction, we
set the transaction handle to null to guard against aborting a
transaction that has already been committed.
</p>
<pre class="programlisting"> } finally {
if (txn != null) {
try {
txn.abort();
} catch (Exception e) {
System.err.println("Error aborting txn: " +
e.toString());
e.printStackTrace();
}
}
}
}
}
} </pre>
<p>
The final piece of our <code class="classname">DBWriter</code> class is the
<code class="methodname">countRecords()</code> implementation. Notice how in
this example we open the cursor such that it performs uncommitted
reads:
</p>
<pre class="programlisting"> // A method that counts every record in the database.
// Note that this method exists only for illustrative purposes.
// A more straight-forward way to count the number of records in
// a database is to use the Database.getStats() method.
private int countRecords(Transaction txn) throws DatabaseException {
DatabaseEntry key = new DatabaseEntry();
DatabaseEntry data = new DatabaseEntry();
int count = 0;
Cursor cursor = null;
try {
// Get the cursor
CursorConfig cc = new CursorConfig();
cc.setReadUncommitted(true);
cursor = myDb.openCursor(txn, cc);
while (cursor.getNext(key, data, LockMode.DEFAULT) ==
OperationStatus.SUCCESS) {
count++;
}
} finally {
if (cursor != null) {
cursor.close();
}
}
return count;
}
} </pre>
</div>
<p>
This completes our transactional example. If you would like to
experiment with this code, you can find the example in the following
location in your JE distribution:
</p>
<pre class="programlisting"><span class="emphasis"><em>JE_HOME</em></span>/examples/je/txn</pre>
</div>
<div class="navfooter">
<hr />
<table width="100%" summary="Navigation footer">
<tr>
<td width="40%" align="left"><a accesskey="p" href="wrapup.html">Prev</a> </td>
<td width="20%" align="center">
<a accesskey="u" href="wrapup.html">Up</a>
</td>
<td width="40%" align="right"> <a accesskey="n" href="txnexample_dpl.html">Next</a></td>
</tr>
<tr>
<td width="40%" align="left" valign="top">Chapter 6. Summary and Examples </td>
<td width="20%" align="center">
<a accesskey="h" href="index.html">Home</a>
</td>
<td width="40%" align="right" valign="top"> DPL Transaction Example</td>
</tr>
</table>
</div>
</body>
</html>