BigCouch 0.4 was released on January 26, 2012. Its emphasis is on stability, performance & compatibility. Its API is roughly equivalent to that of Apache CouchDB 1.1.1.
Important Features, Changes & Fixes
- Native SSL support (thanks, Benoit!)
- Support for FreeBSD (thanks, Benoit! — pull requests: 1, 2).
- Fix allowing database deletion & creation while nodes in the cluster are down (commits: 1, 2).
- Connect nodes to the cluster immediately on start-up, rather than wait for the first user request before doing so.
- Added _replicator database to manage replications, re-implemented in BigCouch for clustered scenarios (works similarly to the CouchDB 1.1 feature, as explained in this writeup over at Couchbase.org).
- Added CommonJS support to map functions (from CouchDB 1.1 — writeup). This resumes compatibility with the KansoJS framework.
- Added support for HTTP range requests for attachments (from CouchDB 1.1 — writeup).
- Support for stale=update_after, re-implemented in BigCouch for clustered scenarios; requesting a view with
stale=update_aftertriggers a view update, but returns a stale=ok response (pull request, writeup).
- Support for SpiderMonkey 1.8.5.
- Support for Zones, granting you improved control of the placement of shard copies (pull request).
- Many core optimizations for read, write & view performance.
- Improvements to the Chef Cookbooks (thanks, Garret Heaton! — see dedicated repo).
- Fix bug making mem3 fail if node had been deleted.
- Fix bug where the
_changesfeed would crash if it encountered docs with no revisions
- Balance the load of
stale=update_afterrequests more evenly across shards (pull request).
- Use a simple majority for read or write quorums by default (pull requests: 1, 2, 3).
- Fix reader protection (pull requests: 1, 2)
update_seqsorting (pull request)
- Report a metric called
data_sizethat represents the compacted size of database & views. The number reported is the sum of the size of the binary Erlang terms used to represent the data.
data_sizemay be bigger than the size on disk, as it is does not take into account compression. This metric also include the size of attachments. While it bears the same name, it is different from the upcoming CouchDB 1.2’s
data_sizemetric. (Pull requests: 1, 2.)
- Make progress on all pending replications instead of blocking on huge ones (pull request).
- Send HTTP code “202 Accepted” if a quorum is not met but a copy is saved (issue, pull requests: 1, 2).
- Support multiple queries in a single view POST (commit, jira).
- Support the
wquery parameter in POST requests (thanks, Mike Wallace! — pull request).
- Fix replicator bug that interrupted checkpointing on powers of 10
- Fix filtered
- Fix view reponse failures during an index compaction
- Handle empty map set correctly when using the built-in
- Wait for read-repair to complete before responding (pull request).
- Fix bug where reduce views would hang if a node was down and a shard was emitting zero rows
- Start JSON response to
_changeslazily to allow for sequence validation
- Improve error handling that occurs during a response (example for views).
- Fix timeouts that appearing when fetching the
_changesfeed continuously while doing regular writes
- Use Smart HTTP URLs in rebar
- Improve batching of writes in view updater (jira).
Backported from CouchDB 1.1.1
- Allow CommonJS modules to be an empty string.
- Fix bug where duplicates can appear in
- Fix bug where update handlers break after conflict resolution.
- Fix crashes when compacting large views.
- Fix file descriptor leak in
- Fix missing revisions in
- Improve handling of compaction at
- Permit forward slashes in path to update functions.
- Reap couchjs processes that hit
- Status code can be specified in update handlers.
- Support provides() in show functions.
_view_cleanupwhen ddoc has no views now removes all index files.
- Fix replication crash when source database has a document with empty ID.
- Fix deadlock when assigning couchjs processes to serve requests.
- Fixes to the document multipart PUT API.
Backported from CouchDB 1.1
- Added built-in filters for
- More flexible URL rewriter.
- Added OS Process module to manage daemons outside of BigCouch.
- Added HTTP Proxy handler for more scalable externals.
- Multiple micro-optimizations when reading data.
- More explicit error messages when it’s not possible to access a file due to lack of permissions.
- Added a “change password” feature to Futon.
Backported from CouchDB 1.0.3
- Fixed compatibility issues with Erlang R14B02.
- Fix bug that allows invalid UTF-8 after valid escapes.
- Enabled replication over IPv6.
- Fixed for crashes in continuous and filtered changes feeds.
- Fixed error when restarting replications in OTP R14B02.
- Fixed error with filter replication with a limit of 1.
- Upgrade ibrowse to version 2.2.0.
- Fixed OAuth signature computation in OTP R14B02.
- Handle passwords with : in them.
- Made compatible with jQuery 1.5.x.
- Added support for
inclusive_endwith reduce views.
0.3.1 (5 April 2011)
- Handle empty result set correctly in
- Allow access to view indexes of non-clustered DBs via backdoor interface.
- Inject DB name (not shard name) in JSON user context sent to externals.
- Monitor view index file descriptor before using it. Prevents crashes of long-running view queries when the index is compacted underneath.
- Support custom write quorums in
- Fix initialization of per-row sequences in
BigCouch is a highly available, fault-tolerant, clustered, mostly api-compliant version of Apache CouchDB. While it appears to the end-user as one CouchDB instance, it is in fact one or more nodes in an elastic cluster, acting in concert to store and retrieve documents, index and serve views, and serve CouchApps. BigCouch has been developed and is continually maintained by Cloudant.
Clusters behave according to concepts outlined in Amazon’s Dynamo paper, namely that each node can accept requests, data is placed on partitions based on a consistent hashing algorithm, and quorum protocols are for read/write operations.
These notes outline the major changes in this release. For details on install and build dependencies consult the main README file.
Major subcomponents of BigCouch, Chttpd, Fabric, Mem3, and Rexi, are now separate projects, and are pulled into the
$CLOUDANT_SRC/deps directory during
configure. This allows us to maintain separate branches of these components and allows developers to control which branches of each subcomponent are in use by modifying the rebar.config file. Note that the links above point to the specific tags enumerated in the config file, which can be changed if a different version is desired. As each subcomponent under the
deps directory is really a git project it can also be branched as needed for local development work. This enables these components to be developed independently. A troubleshooting wiki page contains notes on some of the more common problems.
Overview of Changes
Many of the changes are bug fixes, refactorings to simplify code, better error handling and reporting and documentation. These notes enumerate some of the highlights for each component, clicking on any section leads you to the detailed release notes for each component.
asynchronously repair inconsistent copies of a document on read
added erlang specs/doc to all the non-trivial API functions.
better percolation of errors from
rexilayer out to
aggressively kill rexi workers once quorums are satisfied, improving all around performance by reducing the number of workers
fixed issue #21 – behavior of
include_docs=truefor deleted docs in changes
fixed issue with attachment offsets causing db sizes to inflate
Rexi now keeps a circular buffer of errors it’s seen in memory. This is very useful for debugging when testing. Running a node with an interactive erlang shell, one can access these errors with a simple call:
How many are kept in memory can also be changed:
server now ignores unexpected cast messages
when possible stack traces are propagated back to fabric, this will remove the
dead shardsmessages seen in many instances.
Mem3 contains the major new enhancement in BigCouch 0.3, a new lightweight internal replicator, used for both the shards as well as the
nodes databases. We call it lightweight because it weighs in at a little less than 200 lines of code, counting blank lines. Some other changes:
better naming of shards to handle node up/down issues
shard_dbnow repairs itself and if needed shards are created when nodes that were down return to action.
Chttpd is closely aligned with CouchDB’s HTTP layer. Many of the fixes in the 1.0.2 release were incorporated in this release:
added ability to specify local dbs when replicating
better propagation of errors
fixes to _show/_list/_update functions
include unique identifiers in headers,
X-Couch-Request-ID, returned to user, which can be tracked to errors in server logs(see rexi above).
BigCouch and embedded CouchDB
The BigCouch project embeds CouchDB itself, as well as all the subcomponents and the external dependencies,
oauth. These dependencies have been upgraded to incorporate recent changes and fixes.