mirror of
https://github.com/berkeleydb/libdb.git
synced 2024-11-16 17:16:25 +00:00
466 lines
19 KiB
HTML
466 lines
19 KiB
HTML
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
|
||
<title>Configuring the Logging Subsystem</title>
|
||
<link rel="stylesheet" href="gettingStarted.css" type="text/css" />
|
||
<meta name="generator" content="DocBook XSL Stylesheets V1.73.2" />
|
||
<link rel="start" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" />
|
||
<link rel="up" href="filemanagement.html" title="Chapter 5. Managing DB Files" />
|
||
<link rel="prev" href="logfileremoval.html" title="Removing Log Files" />
|
||
<link rel="next" href="wrapup.html" title="Chapter 6. Summary and Examples" />
|
||
</head>
|
||
<body>
|
||
<div xmlns="" class="navheader">
|
||
<div class="libver">
|
||
<p>Library Version 11.2.5.3</p>
|
||
</div>
|
||
<table width="100%" summary="Navigation header">
|
||
<tr>
|
||
<th colspan="3" align="center">Configuring the Logging Subsystem</th>
|
||
</tr>
|
||
<tr>
|
||
<td width="20%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a> </td>
|
||
<th width="60%" align="center">Chapter 5. Managing DB Files</th>
|
||
<td width="20%" align="right"> <a accesskey="n" href="wrapup.html">Next</a></td>
|
||
</tr>
|
||
</table>
|
||
<hr />
|
||
</div>
|
||
<div class="sect1" lang="en" xml:lang="en">
|
||
<div class="titlepage">
|
||
<div>
|
||
<div>
|
||
<h2 class="title" style="clear: both"><a id="logconfig"></a>Configuring the Logging Subsystem</h2>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="toc">
|
||
<dl>
|
||
<dt>
|
||
<span class="sect2">
|
||
<a href="logconfig.html#logfilesize">Setting the Log File Size</a>
|
||
</span>
|
||
</dt>
|
||
<dt>
|
||
<span class="sect2">
|
||
<a href="logconfig.html#logregionsize">Configuring the Logging Region Size</a>
|
||
</span>
|
||
</dt>
|
||
<dt>
|
||
<span class="sect2">
|
||
<a href="logconfig.html#inmemorylogging">Configuring In-Memory Logging</a>
|
||
</span>
|
||
</dt>
|
||
<dt>
|
||
<span class="sect2">
|
||
<a href="logconfig.html#logbuffer">Setting the In-Memory Log Buffer Size</a>
|
||
</span>
|
||
</dt>
|
||
</dl>
|
||
</div>
|
||
<p>
|
||
You can configure the following aspects of the logging
|
||
subsystem:
|
||
</p>
|
||
<div class="itemizedlist">
|
||
<ul type="disc">
|
||
<li>
|
||
<p>
|
||
Size of the log files.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
Size of the logging subsystem's region.
|
||
See <a class="xref" href="logconfig.html#logregionsize" title="Configuring the Logging Region Size">Configuring the Logging Region Size</a>.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
Maintain logs entirely in-memory.
|
||
See <a class="xref" href="logconfig.html#inmemorylogging" title="Configuring In-Memory Logging">Configuring In-Memory Logging</a>
|
||
for more information.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
Size of the log buffer in memory.
|
||
See <a class="xref" href="logconfig.html#logbuffer" title="Setting the In-Memory Log Buffer Size">Setting the In-Memory Log Buffer Size</a>.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
On-disk location of your log files. See
|
||
<a class="xref" href="enabletxn.html#splittingdata" title="Identifying Specific File Locations">Identifying Specific File Locations</a>.
|
||
</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="sect2" lang="en" xml:lang="en">
|
||
<div class="titlepage">
|
||
<div>
|
||
<div>
|
||
<h3 class="title"><a id="logfilesize"></a>Setting the Log File Size</h3>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<p>
|
||
Whenever a pre-defined amount of data is written to a
|
||
log file (10 MB by default), DB stops using the
|
||
current log file and starts writing to a new file.
|
||
You can change the maximum amount of data contained in each
|
||
log file by using the
|
||
<span>
|
||
<code class="methodname">DB_ENV->set_lg_max()</code>
|
||
|
||
method.
|
||
</span>
|
||
|
||
Note that this method can be used at any time
|
||
during an application's lifetime.
|
||
</p>
|
||
<p>
|
||
Setting the log file size to something larger than its
|
||
default value is largely a matter of
|
||
convenience and a reflection of the application's
|
||
preference in backup media and frequency. However, if
|
||
you set the log file size too low relative to your
|
||
application's traffic patterns, you can cause
|
||
yourself trouble.
|
||
</p>
|
||
<p>
|
||
From a performance perspective, setting the log file
|
||
size to a low value can cause your active transactions to
|
||
pause their writing activities more frequently than would
|
||
occur with larger log file sizes. Whenever a
|
||
transaction completes the log buffer is flushed to
|
||
disk. Normally other transactions can continue to
|
||
write to the log buffer while this flush is in
|
||
progress. However, when one log file is being closed
|
||
and another created, all transactions must cease
|
||
writing to the log buffer until the switch over is
|
||
completed.
|
||
</p>
|
||
<p>
|
||
Beyond performance concerns, using smaller log files
|
||
can cause you to use more physical files on disk.
|
||
As a result, your application could run out
|
||
of log sequence numbers, depending on how busy your
|
||
application is.
|
||
</p>
|
||
<p>
|
||
Every log file is identified with a 10 digit number.
|
||
Moreover, the maximum number of log files that your application
|
||
is allowed to create in its lifetime is 2,000,000,000.
|
||
</p>
|
||
<p>
|
||
For example, if your application performs 6,000 transactions per
|
||
second for 24 hours a day, and you are logging 500 bytes of
|
||
data per transaction into 10 MB log files, then you
|
||
will run out of log files in around 221 years:
|
||
</p>
|
||
<pre class="programlisting"> (10 * 2^20 * 2000000000) / (6000 * 500 * 365 * 60 *60 * 24) = 221 </pre>
|
||
<p>
|
||
However, if you were writing 2000 bytes of data per
|
||
transaction, and using 1 MB log files, then the same
|
||
formula shows you running out of log files in 5 years time.
|
||
</p>
|
||
<p>
|
||
All of these time frames are quite long, to be sure,
|
||
but if you do run out of log files after, say, 5 years
|
||
of continuous operations, then you must reset your log
|
||
sequence numbers. To do so:
|
||
</p>
|
||
<div class="orderedlist">
|
||
<ol type="1">
|
||
<li>
|
||
<p>
|
||
Backup your databases as if to prepare for
|
||
catastrophic failure. See
|
||
<a class="xref" href="backuprestore.html" title="Backup Procedures">Backup Procedures</a>
|
||
for more information.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
Reset the log file's sequence number using the
|
||
<span class="command"><strong>db_load</strong></span> utility's
|
||
<code class="literal">-r</code> option.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
Remove all of the log files from your
|
||
environment. Note that this is the only
|
||
situation in which all of the log files are
|
||
removed from an environment; in all other
|
||
cases, at least a single log file is retained.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
Restart your application.
|
||
</p>
|
||
</li>
|
||
</ol>
|
||
</div>
|
||
</div>
|
||
<div class="sect2" lang="en" xml:lang="en">
|
||
<div class="titlepage">
|
||
<div>
|
||
<div>
|
||
<h3 class="title"><a id="logregionsize"></a>Configuring the Logging Region Size</h3>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<p>
|
||
The logging subsystem's default region size is 60 KB. The
|
||
logging region is used to store filenames, and so you may
|
||
need to increase its size if a large number of files (that
|
||
is, if you have a very large number of databases) will
|
||
be opened and registered with DB's log manager.
|
||
</p>
|
||
<p>
|
||
You can set the size of your logging region by using the
|
||
<span>
|
||
<code class="methodname">DB_ENV->set_lg_regionmax()</code>
|
||
|
||
</span>
|
||
|
||
method. Note that this method can only be called before the
|
||
first environment handle for your application is opened.
|
||
</p>
|
||
</div>
|
||
<div class="sect2" lang="en" xml:lang="en">
|
||
<div class="titlepage">
|
||
<div>
|
||
<div>
|
||
<h3 class="title"><a id="inmemorylogging"></a>Configuring In-Memory Logging</h3>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<p>
|
||
It is possible to configure your logging subsystem such
|
||
that logs are maintained entirely in memory. When
|
||
you do this, you give up your transactional durability
|
||
guarantee. Without log files, you have no way to run
|
||
recovery so any system or software failures that you might
|
||
experience can corrupt your databases.
|
||
</p>
|
||
<p>
|
||
However, by giving up your durability guarantees, you can
|
||
greatly improve your application's throughput by avoiding
|
||
the disk I/O necessary to write logging information to
|
||
disk. In this case, you still retain your transactional
|
||
atomicity, consistency, and isolation guarantees.
|
||
</p>
|
||
<p>
|
||
To configure your logging subsystem to maintain your logs
|
||
entirely in-memory:
|
||
</p>
|
||
<div class="itemizedlist">
|
||
<ul type="disc">
|
||
<li>
|
||
<p>
|
||
Make sure your log buffer is capable of holding all
|
||
log information that can accumulate during the longest
|
||
running transaction. See <a class="xref" href="logconfig.html#logbuffer" title="Setting the In-Memory Log Buffer Size">Setting the In-Memory Log Buffer Size</a> for details.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
Do not run normal recovery when you open your environment. In this configuration, there are no
|
||
log files available against which you can run recovery. As a result, if you specify recovery
|
||
when you open your environment, it is ignored.
|
||
</p>
|
||
</li>
|
||
<li>
|
||
<p>
|
||
Specify
|
||
<span>
|
||
<code class="literal">DB_LOG_IN_MEMORY</code> to the
|
||
<code class="methodname">DB_ENV->log_set_config()</code>
|
||
|
||
</span>
|
||
|
||
method. Note that you must specify this before
|
||
your application opens its first environment
|
||
handle.
|
||
</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<p>
|
||
For example:
|
||
</p>
|
||
<pre class="programlisting">#include <stdio.h>
|
||
#include <stdlib.h>
|
||
|
||
#include "db.h"
|
||
|
||
int
|
||
main(void)
|
||
{
|
||
int ret, ret_c;
|
||
u_int32_t db_flags, env_flags;
|
||
DB_ENV *envp;
|
||
const char *db_home_dir = "/tmp/myEnvironment";
|
||
|
||
envp = NULL;
|
||
|
||
/* Create the environment */
|
||
ret = db_env_create(&envp, 0);
|
||
if (ret != 0) {
|
||
fprintf(stderr, "Error creating environment handle: %s\n",
|
||
db_strerror(ret));
|
||
return (EXIT_FAILURE);
|
||
}
|
||
|
||
/*
|
||
* Indicate that logging is to be performed only in memory.
|
||
* Doing this means that we give up our transactional durability
|
||
* guarantee.
|
||
*/
|
||
envp->log_set_config(envp, DB_LOG_IN_MEMORY, 1);
|
||
|
||
/*
|
||
* Configure the size of our log memory buffer. This must be
|
||
* large enough to hold all the logging information likely
|
||
* to be created for our longest running transaction. The
|
||
* default size for the logging buffer is 1 MB when logging
|
||
* is performed in-memory. For this example, we arbitrarily
|
||
* set the logging buffer to 5 MB.
|
||
*/
|
||
ret = envp->set_lg_bsize(envp, 5 * 1024 * 1024);
|
||
if (ret != 0) {
|
||
fprintf(stderr, "Error setting log buffer size: %s\n",
|
||
db_strerror(ret));
|
||
goto err;
|
||
}
|
||
|
||
/*
|
||
* Set the normal flags for a transactional subsystem. Note that
|
||
* we DO NOT specify DB_RECOVER. Also, remember that the logging
|
||
* subsystem is automatically enabled when we initialize the
|
||
* transactional subsystem, so we do not explicitly enable
|
||
* logging here.
|
||
*/
|
||
env_flags = DB_CREATE | /* If the environment does not
|
||
* exist, create it. */
|
||
DB_INIT_LOCK | /* Initialize locking */
|
||
DB_INIT_LOG | /* Initialize logging */
|
||
DB_INIT_MPOOL | /* Initialize the cache */
|
||
DB_THREAD | /* Free-thread the env handle. */
|
||
DB_INIT_TXN; /* Initialize transactions */
|
||
|
||
/* Open the environment as normal */
|
||
ret = envp->open(envp, db_home_dir, env_flags, 0);
|
||
if (ret != 0) {
|
||
fprintf(stderr, "Error opening environment: %s\n",
|
||
db_strerror(ret));
|
||
goto err;
|
||
}
|
||
|
||
|
||
/*
|
||
* From here, you create transactions and perform database operations
|
||
* exactly as you would if you were logging to disk. This part is
|
||
* omitted for brevity.
|
||
*/
|
||
|
||
...
|
||
|
||
err:
|
||
/* Close the databases (omitted) */
|
||
|
||
...
|
||
|
||
/* Close the environment */
|
||
if (envp != NULL) {
|
||
ret_c = envp->close(envp, 0);
|
||
if (ret_c != 0) {
|
||
fprintf(stderr, "environment close failed: %s\n",
|
||
db_strerror(ret_c));
|
||
ret = ret_c;
|
||
}
|
||
}
|
||
|
||
return (ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE);
|
||
} </pre>
|
||
</div>
|
||
<div class="sect2" lang="en" xml:lang="en">
|
||
<div class="titlepage">
|
||
<div>
|
||
<div>
|
||
<h3 class="title"><a id="logbuffer"></a>Setting the In-Memory Log Buffer Size</h3>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<p>
|
||
When your application is configured for on-disk logging
|
||
(the default behavior for transactional applications), log
|
||
information is stored in-memory until the storage space
|
||
fills up, or a transaction commit forces the log
|
||
information to be flushed to disk.
|
||
</p>
|
||
<p>
|
||
It is possible to increase the amount of memory available
|
||
to your file log buffer. Doing so improves throughput for
|
||
long-running transactions, or for transactions that produce
|
||
a large amount of data.
|
||
</p>
|
||
<p>
|
||
When you have your logging subsystem configured to maintain
|
||
your log entirely in memory (see
|
||
<a class="xref" href="logconfig.html#inmemorylogging" title="Configuring In-Memory Logging">Configuring In-Memory Logging</a>), it is very important
|
||
to configure your log buffer size because the log buffer
|
||
must be capable of holding all log information that can
|
||
accumulate during the longest running transaction.
|
||
You must make sure that the in-memory log buffer size is
|
||
large enough that no transaction will ever span the entire
|
||
buffer. You must also avoid a state where the in-memory
|
||
buffer is full and no space can be freed because a
|
||
transaction that started the first log "file" is still
|
||
active.
|
||
</p>
|
||
<p>
|
||
When your logging subsystem is configured for on-disk
|
||
logging, the default log buffer space is 32 KB. When
|
||
in-memory logging is configured, the default log buffer
|
||
space is 1 MB.
|
||
</p>
|
||
<p>
|
||
You can increase your log buffer space using the
|
||
<span>
|
||
<code class="methodname">DB_ENV->set_lg_bsize()</code>
|
||
|
||
</span>
|
||
|
||
method. Note that this method can only be called before the
|
||
first environment handle for your application is opened.
|
||
</p>
|
||
</div>
|
||
</div>
|
||
<div class="navfooter">
|
||
<hr />
|
||
<table width="100%" summary="Navigation footer">
|
||
<tr>
|
||
<td width="40%" align="left"><a accesskey="p" href="logfileremoval.html">Prev</a> </td>
|
||
<td width="20%" align="center">
|
||
<a accesskey="u" href="filemanagement.html">Up</a>
|
||
</td>
|
||
<td width="40%" align="right"> <a accesskey="n" href="wrapup.html">Next</a></td>
|
||
</tr>
|
||
<tr>
|
||
<td width="40%" align="left" valign="top">Removing Log Files </td>
|
||
<td width="20%" align="center">
|
||
<a accesskey="h" href="index.html">Home</a>
|
||
</td>
|
||
<td width="40%" align="right" valign="top"> Chapter 6. Summary and Examples</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</body>
|
||
</html>
|