Home • Use • Develop • Troubleshoot • API • Changelog

Changelog

0.4.0

BigCouch 0.4 was released on January 26, 2012. Its emphasis is on stability, performance & compatibility. Its API is roughly equivalent to that of Apache CouchDB 1.1.1.

Important Features, Changes & Fixes

• Native SSL support (thanks, Benoit!)
• Support for FreeBSD (thanks, Benoit! — pull requests: 1, 2).
• Fix allowing database deletion & creation while nodes in the cluster are down (commits: 1, 2).
• Connect nodes to the cluster immediately on start-up, rather than wait for the first user request before doing so.
• Added _replicator database to manage replications, re-implemented in BigCouch for clustered scenarios (works similarly to the CouchDB 1.1 feature, as explained in this writeup over at Couchbase.org).
• Added CommonJS support to map functions (from CouchDB 1.1 — writeup). This resumes compatibility with the KansoJS framework.
• Added support for HTTP range requests for attachments (from CouchDB 1.1 — writeup).
• Support for stale=update_after, re-implemented in BigCouch for clustered scenarios; requesting a view with stale=update_after triggers a view update, but returns a stale=ok response (pull request, writeup).
• Support for SpiderMonkey 1.8.5.
• Support for Zones, granting you improved control of the placement of shard copies (pull request).
• Many core optimizations for read, write & view performance.

BigCouch-specific Updates

• Improvements to the Chef Cookbooks (thanks, Garret Heaton! — see dedicated repo).
• Fix bug making mem3 fail if node had been deleted.
• Fix bug where the _changes feed would crash if it encountered docs with no revisions
• Balance the load of stale=ok and stale=update_after requests more evenly across shards (pull request).
• Use a simple majority for read or write quorums by default (pull requests: 1, 2, 3).
• Fix reader protection (pull requests: 1, 2)
• Fix update_seq sorting (pull request)
• Report a metric called data_size that represents the compacted size of database & views. The number reported is the sum of the size of the binary Erlang terms used to represent the data. data_size may be bigger than the size on disk, as it is does not take into account compression. This metric also include the size of attachments. While it bears the same name, it is different from the upcoming CouchDB 1.2’s data_size metric. (Pull requests: 1, 2.)
• Make progress on all pending replications instead of blocking on huge ones (pull request).
• Send HTTP code “202 Accepted” if a quorum is not met but a copy is saved (issue, pull requests: 1, 2).
• Support multiple queries in a single view POST (commit, jira).
• Support the w query parameter in POST requests (thanks, Mike Wallace! — pull request).
• Fix replicator bug that interrupted checkpointing on powers of 10
• Fix filtered _changes timeouts
• Fix view reponse failures during an index compaction
• Handle empty map set correctly when using the built-in _stats reduce function
• Wait for read-repair to complete before responding (pull request).
• Fix bug where reduce views would hang if a node was down and a shard was emitting zero rows
• Start JSON response to _changes lazily to allow for sequence validation
• Improve error handling that occurs during a response (example for views).
• Fix timeouts that appearing when fetching the _changes feed continuously while doing regular writes
• Use Smart HTTP URLs in rebar
• Improve batching of writes in view updater (jira).

Backported from CouchDB 1.1.1

• Allow CommonJS modules to be an empty string.
• Fix bug where duplicates can appear in _changes feed.
• Fix bug where update handlers break after conflict resolution.
• Fix crashes when compacting large views.
• Fix file descriptor leak in _log
• Fix missing revisions in _changes?style=all_docs.
• Improve handling of compaction at max_dbs_open limit.
• JSONP responses now send “text/javascript” for Content-Type.
• Permit forward slashes in path to update functions.
• Reap couchjs processes that hit reduce_overflow error.
• Status code can be specified in update handlers.
• Support provides() in show functions.
• _view_cleanup when ddoc has no views now removes all index files.
• Fix replication crash when source database has a document with empty ID.
• Fix deadlock when assigning couchjs processes to serve requests.
• Fixes to the document multipart PUT API.

Backported from CouchDB 1.1

• Added built-in filters for _changes: _doc_ids and _design (writeup).
• More flexible URL rewriter.
• Added OS Process module to manage daemons outside of BigCouch.
• Added HTTP Proxy handler for more scalable externals.
• Multiple micro-optimizations when reading data.
• More explicit error messages when it’s not possible to access a file due to lack of permissions.
• Added a “change password” feature to Futon.

Backported from CouchDB 1.0.3

• Fixed compatibility issues with Erlang R14B02.
• Fix bug that allows invalid UTF-8 after valid escapes.
• Enabled replication over IPv6.
• Fixed for crashes in continuous and filtered changes feeds.
• Fixed error when restarting replications in OTP R14B02.
• Fixed error with filter replication with a limit of 1.
• Upgrade ibrowse to version 2.2.0.
• Fixed OAuth signature computation in OTP R14B02.
• Handle passwords with : in them.
• Made compatible with jQuery 1.5.x.
• Added support for inclusive_end with reduce views.

---

0.3.1 (5 April 2011)

• Handle empty result set correctly in _stats reduction.
• Allow access to view indexes of non-clustered DBs via backdoor interface.
• Inject DB name (not shard name) in JSON user context sent to externals.
• Monitor view index file descriptor before using it. Prevents crashes of long-running view queries when the index is compacted underneath.
• Support custom write quorums in _bulk_docs.
• Fix initialization of per-row sequences in _changes queries with since query-string parameters.

---

0.3.0

BigCouch is a highly available, fault-tolerant, clustered, mostly api-compliant version of Apache CouchDB. While it appears to the end-user as one CouchDB instance, it is in fact one or more nodes in an elastic cluster, acting in concert to store and retrieve documents, index and serve views, and serve CouchApps. BigCouch has been developed and is continually maintained by Cloudant.

Clusters behave according to concepts outlined in Amazon’s Dynamo paper, namely that each node can accept requests, data is placed on partitions based on a consistent hashing algorithm, and quorum protocols are for read/write operations.

These notes outline the major changes in this release. For details on install and build dependencies consult the main README file.

Building

Major subcomponents of BigCouch, Chttpd, Fabric, Mem3, and Rexi, are now separate projects, and are pulled into the $CLOUDANT_SRC/deps directory during configure. This allows us to maintain separate branches of these components and allows developers to control which branches of each subcomponent are in use by modifying the rebar.config file. Note that the links above point to the specific tags enumerated in the config file, which can be changed if a different version is desired. As each subcomponent under the deps directory is really a git project it can also be branched as needed for local development work. This enables these components to be developed independently. A troubleshooting wiki page contains notes on some of the more common problems.

Overview of Changes

Many of the changes are bug fixes, refactorings to simplify code, better error handling and reporting and documentation. These notes enumerate some of the highlights for each component, clicking on any section leads you to the detailed release notes for each component.

Fabric 1.2.1

• asynchronously repair inconsistent copies of a document on read

• added erlang specs/doc to all the non-trivial API functions.

• better percolation of errors from rexi layer out to chttpd

• aggressively kill rexi workers once quorums are satisfied, improving all around performance by reducing the number of workers

• fixed issue #21 – behavior of include_docs=true for deleted docs in changes

• fixed issue with attachment offsets causing db sizes to inflate

Rexi 1.4.3

Rexi now keeps a circular buffer of errors it’s seen in memory. This is very useful for debugging when testing. Running a node with an interactive erlang shell, one can access these errors with a simple call:

rexi:get_errors().

How many are kept in memory can also be changed:

rexi:set_error_limit(N).

• server now ignores unexpected cast messages

• when possible stack traces are propagated back to fabric, this will remove the dead shards messages seen in many instances.

Mem3 1.1.1

Mem3 contains the major new enhancement in BigCouch 0.3, a new lightweight internal replicator, used for both the shards as well as the shard_db and nodes databases. We call it lightweight because it weighs in at a little less than 200 lines of code, counting blank lines. Some other changes:

• better naming of shards to handle node up/down issues

• shard_db now repairs itself and if needed shards are created when nodes that were down return to action.

Chttpd 1.2.1

Chttpd is closely aligned with CouchDB’s HTTP layer. Many of the fixes in the 1.0.2 release were incorporated in this release:

• added ability to specify local dbs when replicating

• better propagation of errors

• fixes to _show/_list/_update functions

• include unique identifiers in headers, X-Couch-Request-ID, returned to user, which can be tracked to errors in server logs(see rexi above).

BigCouch and embedded CouchDB

The BigCouch project embeds CouchDB itself, as well as all the subcomponents and the external dependencies, ibrowse, mochiweb, and oauth. These dependencies have been upgraded to incorporate recent changes and fixes.

• Integrated CouchDB 1.0.2, a major release with lots of features/fixes

• Fixed inclusive_end=false for reduce queries

• Added support for compacting view shards with non-local design docs (BigCouch #16)