libdb/dist/ChangeLog

= Berkeley DB 5.2 Changelog =

== Database or Log File On-Disk Format Changes ==

Existing database file formats were unchanged in library version 11.2.5.2.  
However, a new database file format, "heap", was introduced.

The log file format changed in library version 11.2.5.2.

== New Features ==

Replication Manager now manages Group Membership. This allows sites to be added to
and removed from the replication group dynamically. Replication Manager
also now automatically keeps track of the group size (nsites). [#14778]

Initial allocations for various non-pagebuffer (mpool) system resources 
may now be specified, as well as a total maximum of memory to use, rather than 
specifying a maximum value for each resource. [#16334]

Implemented Berkeley DB globalization support architecture to enable localized 
and stripped error and output messages. [#16863] 

Added a new access method, DB_HEAP.  Heap aims for efficient use (and re-use) 
of disk space.  Keys in a heap database are automatically generated by BDB, it 
is recommended that one or more secondary indexes be used with a heap 
database.  For full details on DB_HEAP, see the Programmer's Reference 
Guide. [#17627]

Added a compatible mode for 32bit and 64bit Windows environment. [#18225]

For the SQL API, concurrency between read and write transactions can now be
enabled using "PRAGMA multiversion". Added several pragmas that can be
used to configure the Berkeley DB datastore. [#18521]

Add several new pragmas to provide in-process support for replication in
the SQL API. [#18528]

The Berkeley DB X/open compliant XA resource manager has been restored, 
including support for multi-threaded servers. [#18701]

Improved the ability to recover from an application crash on connections 
through the SQL API.  Berkeley DB will try to automatically clean up locks, 
mutexes and transactions from the failed process. [#18713]

Add support for sequence usage in the SQL API using SQLite custom 
functions. [#19007]

Add a pragma in the SQL API to allow execution of a cache trickle 
command. [#19202]

Add a pragma in the SQL API to allow configuration of DB_SYSTEM_MEM 
environments. [#19249]

The new db_env_set_win_security(SECURITY_ATTRIBUTES *) function allows an
application to specify the particular Microsoft Windows security attributes
to be used by Berkeley DB.  This helps support applications which reduce their
privileges after opening the environment. [#19529]


== Database Environment Changes ==

None

== Concurrent Data Store Changes ==

None

== Access Method Changes ==

Modified the queue access method so that it only uses latches on the metadata 
page rather than a latch and a lock. This was done to improve 
performance. [#18749]

Fixed several bugs that could cause an update running with MVCC to get the 
wrong version of a page or improperly update the metadata last page 
number. [#19063]

The database open code will no longer log the open and close of the master 
database in a file when opening a sub database in that file. [#19071]

Fixed a bug where an error during an update to a hash database with 
DB_NOOVERWRITE set could return DB_KEYEXIST rather than the correct 
error. [#19077]

Fixed a bug that could cause the wrong page number to be on a root or metadata 
page if DB->compact moved the page and the operation was later rolled 
forward. [#19167]

Fixed a bug that could cause the close of a secondary index database to fail 
if the transaction doing the open aborted. [#19169]

Fixed a bug that could prevent an update to a primary recno or queue database 
with DB_NOOVERWITE set. [#19230]

Fixed a bug when an update to a database with DB_NOOVERWRITE set could 
incorrectly return DB_KEYEXIST rather than the correct error 
(e.g., DB_LOCK_DEADLOCK). [#19345]

Fixed a bug preventing the use of the set_re_len and set_re_pad methods with 
a RECNO database when configuring with --disable-queue. [#19367]

Fixed a bug in DB->compact on BTREE databases that did not check if the last 
page in the database could be moved to a lower numbered page. [#19394]

Fixed a bug that could cause a Log Sequence Error when recovering the 
deallocation of a multiple page overflow chain. [#19474]

Fixed a bug that could cause a diagnostic assertion if MVCC was in use
and multiple levels of a btree needed to be split. [#19481]

Fixed a few error paths that could cause a Panic with an "unpinned page 
returned" error. [#19493]

Fixed a bug that closed a race condition that under heavy mult-threaded 
appending to a queue database could cause some records to be lost. [#19498]

Fixed a bug that might cause DB->compact to mis-estimate the size of an 
overflow record when merging two pages. This may cause the page to have more 
data than desired. [#19562]

Fixed a bug in DB_ENV->fileid_reset that did not update the fileid's on the 
metadata pages of subdatabases if the database file was not in native 
byte order. [#19608]

Fixed a bug that caused the first directory specified in the create of 
a partitioned database to get too many partitions. [#20041]

== SQL API Changes ==

Fixed a race condition that would cause a corruption error in one process when 
two processes created the same SQL database. [#18929]

Fixed a bug that would cause a constraint violation when updating the 
primary key with the same value. [#18976]

Overwriting an old backup with a new backup using the SQL online backup API 
will no longer double the size of the database. [#19021]

Implemented index optimizations for indexes on large values. [#19094]

Fixed a bug that could cause an undetected deadlock between a 
thread which moved a metadata or root page via a DB->compact operation and 
another thread trying to open the database if the old page was being removed 
from the file. [#19186]

Fix a bug in the BDBSQL_FILE_PER_TABLE option, to allow absolute 
path names. [#19190]

Add a pragma to allow configuration of DB_SYSTEM_MEM environments. [#19249]

Exclusive transactions will now block new transactions and will prevent 
existing transactions from making forward progress. [#19256]

Fixed a bug that would cause assert error when opening an in-memory hash 
database with thread count configured when compiled with 
--enable-diagnostic. [#19357] 

Upgrade the bundled version of SQLite to 3.7.6.2 [#19376]

Fixed a performance bug with the cache victim selection algorithm when there 
were multiple cache regions. [#19385]

Fixed a bug which could cause two SQL threads to have an undetected deadlock
when opening or closing tables. [#19386]

Fix a bug that could cause a hang when deleting a table if there are multiple 
connections to a database from different processes. [#19419]

Fixed a bug which could cause multiple threads performing DB->compact on 
the same database file to overrun the in-memory freelist, which could 
potentially lead to memory corruption. [#19571]

Fixed a bug in DB->compact that could cause a loop if an attempt to move a 
sub-database meta data page deadlocked. [#20028]

== C API Changes ==

Fixed a bug where encryption could not be enabled for individual databases 
in an encrypted environment. [#18891]

Removed two unused error codes, DB_NOSERVER_HOME and DB_NOSERVER_ID. [#18978]

Added a DB_DBT_READONLY flag so that users can pass in a non-usermem key 
(DB_DBT_USERMEM) for get operations. [#19360]  

Fixed a bug in DB/DBC->get/pget that the partial flags are silently ignored
with positional flags and return inconsistent DBT. [#19540]

Fixed a bug which prevented items from being deleted on a secondary 
database. [#19573] 

Fixed a bug to correctly handle the DB_BUFFER_SMALL case on delete 
operations when compression is enabled. [#19660]
	
== Tcl-specific API Changes ==

None. 
	

== C#-specific API Changes ==

Added support for partial put/get in the C# API. [#18795]

Fixed a bug in compare delegate for secondary db. [#18935]

== Replication Changes ==

Replication Manager now allows differing ack policies at different
sites throughout the group, and supports dynamic changes to the ack
policy.  (The ack policy in force is determined by the current
master.)  [#14993]

Replication Manager "channels" feature allows applications to
share repmgr's communication facilities. [#17228]

Add example program for RepMgr "channels" feature: ex_rep_chan. [#17387]

Replication Manager now allows dynamic changes to a site's
"electability" (changes between zero and non-zero priority).  This
feature should be used with care, because electability changes can in
boundary cases invalidate durability guarantees granted for previous
transactions. [#17497]

Changed election criteria so that later group transactions
won't get overwritten by earlier generations with more log. [#17815]

Added changes to master lease checks that result in improved 
performance when using master leases. [#18960]

A log write failure on a replication master will now cause 
a panic since the transaction may be committed on some clients. [#19054]

Fixed a few memory leak conditions on error paths. [#19131]

Change lease code so that zero priority sites do not count
in lease guarantees since they cannot be elected. [#19154]

Repmgr rerequest processing is moved from a dedicated thread to heartbeat 
messages.  Repmgr clients using heartbeats can now detect and rerequest 
missing final master log records without master activity. [#19197]

Repmgr statistics are now included in full statistics output for
an environment. [#19198]

Fix an inefficiency in mixed version elections.  We now check
if an election is won via the EID instead of priority. [#19254]

Changed election LSNs to use the last txn commit LSN instead
of the end of the log. [#19278]

Create replication internal database files in the environment
home directory rather than the data directory so that they are in the
same location as the other internal replication files. [#19403]

Fix a bug that was preventing repmgr from calling an election
when starting a site with the DB_REP_ELECTION flag. [#19546]

Fixed a bug which could cause a segfault at a replication master if a
named in-memory database was being created around the same time as a
client site were synchronizing (in "internal init") with the master.
[#19583]

Adjust lease code to consider timeout length when retrying. [#19705]

Fixed a bug that could cause a crash in replication groups of more
than 10 sites, with multiple processes sharing each DB environment
concurrently. [#19818]

Fix a bug where an assertion failure could happen if pages in a database 
were deallocated during a client internal initialization.[#19851]

Fix a bug where an internal initialization of a queue database with
non-contiguous extent files could return an error. [#19925]

The 2SITE_STRICT replication configuration parameter is now turned on
by default.  It can be turned off via a call to
DB_ENV->rep_set_config(). [#19937]

Repmgr heartbeats can now help detect a duplicate master without the
need for application activity. [#19950]

== Locking Subsystem Changes ==

Fixed a bug where an updater supporting DB_READ_UNCOMMITED might downgrade 
its lock too soon if there was an error during the update. [#19155]

Fixed a bug where transaction timeouts could have been specified in a 
database environment where the locking subsystem was disabled. [#19582]

Fixed a bug in a diagnostic assertion that was improperly triggered by the 
removal of a sub-database. [#19683]

Fixed a bug that would cause DB_ENV->failcheck to free locks for a locker 
associated with a database handle after the thread that opened the handle 
exited. [#19881]

== Logging Subsystem Changes ==

Enhanced recovery so that it will not output extra checkpoint or transaction 
id recycle log records if there was no activity since the last 
checkpoint. [#15330]

Log checksums can now be disabled using the compile argument 
--disable-log-checksum.  This will give a performance increase at the risk 
of undetectable corruption in the log records, which would make recovery 
impossible. [#19143]

Fixed a bug that could cause a page that should have been removed from the 
end of a file still be in the copy of the file in a hot backup. [#19996]

== Memory Pool Subsystem Changes ==

Fixed a bug in MPOOLFILE->get that did not permit the DB_MPOOL_DIRTY flag 
to be used with other flags. [#19421]

== Mutex Subsystem Changes ==

Fixed a bug when the mutex region needs to be larger than 4GB, the region size 
was incorrectly adjusted to be slightly too small to fit the mutexes. [#18968]

Fixed a performance problem with hybrid shared latches in which a request for
exclusive access would busy-wait (rather than put itself to sleep) if the latch
were held by a shared reader. This also fixed the timeout handling of hybrid
mutexes. In some cases the timeout would not be honored, resulting in delays
for the replication "read your writes" feature which were longer than requested.
[#18982]

Fixed the timeout handling of the pthreads mutexes used by the replication
"read your writes" feature. When a timeout occurred there was a race condition
which might result in a hang. [#19047]


== Transaction Subsystem Changes ==

Fixed a leak of log file ids when a database is closed before the end of a 
transaction that references it. [#15957]

Fixed a bug that would cause a panic if a child transaction performed a database 
rename, then aborted, and then the parent transaction committed. [#18069]

Fixed a bug where we released the metadata page lock too early if a
non-transactional update was being done. [#19036]

Removed the possibility that checkpoints will overlap in the log, decreasing 
the time to recover. [#19062]


== Test Suite Changes ==

Require Tcl 8.5 or greater.

== Utility Changes ==

Added a new utility, db_tuner, which analyzes the data in a btree database,
and suggests a reasonable pagesize. [#18910]

Fixed some bugs in log_verify when there are in-memory database logs and subdb 
logs. [#19157]

Modified db_hotbackup to not read from the file system as required on non-UNIX 
systems.  Also provided the db_copy function for this purpose. [#19863]

Fixed db_hotbackup so that when -d/-l or -D is not specified, DB_CONFIG is 
used to determine the locations of the databases and logs in the source 
environment. [#19994]

== Configuration, Documentation, Sample Apps, Portability and Build Changes ==

Changed SQL API library built on *nix to link with libpthreads when 
necessary. [#19098]

Added CPPFLAGS into our --enable-jdbc configuration. [#19234]

Added encryption support into the Windows CE build project for SQL API. [#19632]

Fixed a bug in the STAT_INC_VERB() dtrace probe that was causing compiler 
warnings. [#19707]

Fixed a bug that could cause a trap in db_dump using salvage mode if a 
page was found that was not associated with any database in the file. [#19974]

On Cygwin, circumvented a bug in libtool that is exposed when building the
BDB SQL API in a directory path containing whitespace characters. [#19812]

== Example Changes ==

Update repmgr C, C#, C++, Java examples(ex_rep_mgr, ex_rep_gsg_repmgr,
ex_rep_chan, excs_repquote, excxx_repquote, excxx_epquote_gsg, repquote,
repquote_gsg) with their related API changes for group 
membership. [#19586][#19622]

Port ex_rep_chan, ex_rep_gsg_repmgr,ex_rep_gsg_simple,
excxx_repquote_gsg_repmgr, excxx_repquote_gsg_simple to Window.[#19890]

== Miscellaneous Bug Fixes ==

Fixed a bug where memory copied from the Java API could leak if flags were not
correctly configured. [#19152]

== Deprecated Features ==

None

== Known Bugs ==

The SQL API has a known issue when using a blob field with a lot of content 
and multiple concurrent connections to the database. [#19945]

Rollback of a dropped table in the SQL layer contains a mutex leak, which 
can consume all mutex resources if enough rollbacks of table drops are 
performed. [#20077]

The DB_CONFIG configuration parameters which specify path names currently
do not support names containing any whitespace characters. [#20158]

The BFile module has a known crash issue when using BFile handle for SQL 
expressions interface on 64bit platforms. [#20193] 

On systems without FTRUNCATE, db_verify will return an error for truncated 
heap databases.  This is a bug in db_verify, the database has been truncated 
correctly and can be used in the future. [#20195]

An application using queue extents which is append mostly could see a 
decrease in the buffer pool hit rate due to the failure to remove pages 
from closed extents from the buffer pool. [#20217]