阿里云数据库ApsaraDB 关注
手机版

In-place update in WiredTiger

  1. 云栖社区>
  2. 阿里云数据库ApsaraDB>
  3. 博客>
  4. 正文

In-place update in WiredTiger

张友东 2018-03-08 11:50:07 浏览377 评论0

摘要: There is a great new feature in the release note of MongoDB 3.5.12. Faster In-place Updates in WiredTiger This work brings improvements to in-place ...

There is a great new feature in the release note of MongoDB 3.5.12.

Faster In-place Updates in WiredTiger

This work brings improvements to in-place update workloads for users running the WiredTiger engine, especially for updates to large documents. Some workloads may see a reduction of up to 7x in disk utilization (from 24 MB/s to 3 MB/s) as well as a 20% improvement in throughput.

I thought wiredtiger has impeletementd the delta page feature introduced in the bw-tree paper, that is, writing pages that are deltas from previously written pages. But after I read the source code, I found it's a totally diffirent idea, in-place update only impacted the in-meomry and journal format, the on disk layout of data is not changed.

I will explain the core of the in-place update implementation.

MongoDB introduced mutable bson to descirbe document update as incremental(delta) update.

Mutable BSON provides classes to facilitate the manipulation of existing BSON objects or the construction of new BSON objects from scratch in an incremental fashion.

Suppose you have a very large document, see 1MB

{
   _id: ObjectId("59097118be4a61d87415cd15"),
   name: "ZhangYoudong",
   birthday: "xxxx",
   fightvalue: 100,
   xxx: .... // many other fields
}

If the fightvalue is changed from 100 to 101, you can use a DamageEvent to describe the update, it just tells you the offset、size、content(kept in another array) of the change.

struct DamageEvent {
    typedef uint32_t OffsetSizeType;
    // Offset of source data (in some buffer held elsewhere).
    OffsetSizeType sourceOffset;

    // Offset of target data (in some buffer held elsewhere).
    OffsetSizeType targetOffset;

    // Size of the damage region.
    size_t size;
};

So if you have many small changes for a document, you will have DamageEvent array, MongoDB add a new storage interface to support inserting DamageEvent array (DamageVector).

bool WiredTigerRecordStore::updateWithDamagesSupported() const {
    return true;
}

StatusWith<RecordData> WiredTigerRecordStore::updateWithDamages(
    OperationContext* opCtx,
    const RecordId& id,
    const RecordData& oldRec,
    const char* damageSource,
    const mutablebson::DamageVector& damages) {

}

WiredTiger added a new update type called WT_UPDATE_MODIFIED to support MongoDB, when a WT_UPDATE_MODIFIED update happened, wiredTiger first logged a change list which is transformed from DamageVector into journal, then kept the change list in memory associated with the original record.

When the record is read, wiredTiger will first read the original record, then apply every operation in change list, returned the final record to the client.

So the core for in-place update:

  1. WiredTiger support delta update in memory and journal, so the IO of writing journal will be greatly reduced for large document.
  2. WiredTiger's data layout is kept unchanged, so the IO of writing data is not changed.

用云栖社区APP,舒服~

【云栖快讯】云栖社区技术交流群汇总,阿里巴巴技术专家及云栖社区专家等你加入互动,老铁,了解一下?  详情请点击

网友评论

张友东
文章93篇 | 关注483
关注
提供海量、安全和高可靠的云存储服务。RESTful API的平台无关性,容量和处理能力的弹性... 查看详情
快速、完全托管的TB/PB级数据仓库解决方案,向用户提供了完善的数据导入方案以及多种经典的分... 查看详情
MySQL 是全球最受欢迎的开源数据库,阿里云MySQL版 通过深度的内核优化和独享实例提供... 查看详情
为您提供简单高效、处理能力可弹性伸缩的计算服务,帮助您快速构建更稳定、安全的应用,提升运维效... 查看详情
建站4折

建站4折