Canal BinlogChange(mariadb5/10)

  1. 云栖社区>
  2. 博客>
  3. 正文

Canal BinlogChange(mariadb5/10)

愤怒的苹果 2016-03-30 09:24:04 浏览2855
展开阅读全文

背景

先前开源了一个开源项目: 【阿里巴巴开源项目: 基于mysql数据库binlog的增量订阅&消费】 

本文主要是介绍一下canal支持mariadb协议上的变化. 

 

协议变化

mariadb5.5

mariadb5.5主要是基于mysql5.5的原型,类型定义基本没啥变化,大体上都保持兼容

主要的变化:

1. QueryLogEvent增加了status变量.    

  • Q_HRNOW  用于记录毫秒的精度,枚举值下标为128

协议解析的时候,需要处理Q_HRNOW,需要跳过3字节的数据.  

ps.  mysql5.6后,新增了Q_MICROSECONDS来支持mariaDb中Q_HRNOW的毫秒精度的功能.  

 

2. binlog事件的变化


1./* New MySQL/Sun events are to be added right above this comment */  
2.MYSQL_EVENTS_END,  
3.  
4.MARIA_EVENTS_BEGIN= 160,  
5./* New Maria event numbers start from here */  
6.ANNOTATE_ROWS_EVENT= 160,  
7.  
8./* Add new MariaDB events here - right above this comment!  */  
9.  
10.ENUM_END_EVENT /* end marker */  


  /* New MySQL/Sun events are to be added right above this comment */
  MYSQL_EVENTS_END,

  MARIA_EVENTS_BEGIN= 160,
  /* New Maria event numbers start from here */
  ANNOTATE_ROWS_EVENT= 160,

  /* Add new MariaDB events here - right above this comment!  */

  ENUM_END_EVENT /* end marker */

新增了mariadb的binlog区间为160开始,ANNOTATE_ROWS_EVENT类型为mysql5.6中的RowsQueryLogEvent,用于记录RBR模式下insert/update/delete中执行的sql.

 

mariadb10

mariadb10主要是基于mysql5.6的原型,类型定义基本没啥变化,大体上都保持兼容(沿用了mysql5.6中TIMESTAMP2等新的时间类型和新的log_event类型)

主要的变化:

1. QueryLogEvent增加了status变量.    

  • Q_HRNOW  用于记录毫秒的精度,枚举值下标为128

协议解析的时候,需要处理Q_HRNOW,需要跳过3字节的数据.  

ps.  mysql5.6后,新增了Q_MICROSECONDS来支持mariaDb中Q_HRNOW的毫秒精度的功能.  

 

2. binlog事件的变化


1.MARIA_EVENTS_BEGIN= 160,  
2.  /* New Maria event numbers start from here */  
3.  ANNOTATE_ROWS_EVENT= 160,  
4.  /* 
5.    Binlog checkpoint event. Used for XA crash recovery on the master, not used 
6.    in replication. 
7.    A binlog checkpoint event specifies a binlog file such that XA crash 
8.    recovery can start from that file - and it is guaranteed to find all XIDs 
9.    that are prepared in storage engines but not yet committed. 
10.  */  
11.  BINLOG_CHECKPOINT_EVENT= 161,  
12.  /* 
13.    Gtid event. For global transaction ID, used to start a new event group, 
14.    instead of the old BEGIN query event, and also to mark stand-alone 
15.    events. 
16.  */  
17.  GTID_EVENT= 162,  
18.  /* 
19.    Gtid list event. Logged at the start of every binlog, to record the 
20.    current replication state. This consists of the last GTID seen for 
21.    each replication domain. 
22.  */  
23.  GTID_LIST_EVENT= 163,  
24.  
25.  /* Add new MariaDB events here - right above this comment!  */  
26.  
27.  ENUM_END_EVENT /* end marker */  


MARIA_EVENTS_BEGIN= 160,
  /* New Maria event numbers start from here */
  ANNOTATE_ROWS_EVENT= 160,
  /*
    Binlog checkpoint event. Used for XA crash recovery on the master, not used
    in replication.
    A binlog checkpoint event specifies a binlog file such that XA crash
    recovery can start from that file - and it is guaranteed to find all XIDs
    that are prepared in storage engines but not yet committed.
  */
  BINLOG_CHECKPOINT_EVENT= 161,
  /*
    Gtid event. For global transaction ID, used to start a new event group,
    instead of the old BEGIN query event, and also to mark stand-alone
    events.
  */
  GTID_EVENT= 162,
  /*
    Gtid list event. Logged at the start of every binlog, to record the
    current replication state. This consists of the last GTID seen for
    each replication domain.
  */
  GTID_LIST_EVENT= 163,

  /* Add new MariaDB events here - right above this comment!  */

  ENUM_END_EVENT /* end marker */

 新增了mariadb自己的gtid处理

 

使用注意

1.  AnnotateRowsEvent使用

  •  mariadb需要在my.cnf中设置binlog_annotate_row_events = true,开启记录annotate事件
  • canal在发送COM_BINLOG_DUMP指令中需要设置binlog_flags |= BINLOG_SEND_ANNOTATE_ROWS_EVENT,不然mariadb默认不会发送AnnotateRowsEvent,而是以空的QueryLogEvent来代替.

2.  新增的binlog类型使用

  • canal需要设置当前session变量

    1.SET @mariadb_slave_capability='" + LogEvent.MARIA_SLAVE_CAPABILITY_MINE + "'"  


网友评论

登录后评论
0/500
评论
愤怒的苹果
+ 关注