运维调试记录:Ubuntu14.04下部署Opendaylight Nitrogen集群

  1. 云栖社区>
  2. 博客>
  3. 正文

运维调试记录:Ubuntu14.04下部署Opendaylight Nitrogen集群

adoryn 2018-03-21 13:31:49 浏览1230
展开阅读全文
版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/zhaobryant/article/details/79638755

建议参考官网教程:Setting Up Clustering


一、实验环境

  • 主机节点系统版本: Ubuntu 14.04 (64bit)
odl@mpodl:~$ uname -a
Linux mpodl 4.2.0-27-generic #32~14.04.1-Ubuntu SMP Fri Jan 22 15:32:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  • 主机节点硬件配置:单核CPU+4GB内存+50GB硬盘
odl@mpodl:~$ sudo lshw        
    description: Computer
    product: Standard PC (i440FX + PIIX, 1996) ()
    vendor: QEMU
    version: pc-i440fx-trusty
    width: 64 bits
    capabilities: smbios-2.4 dmi-2.4 vsyscall32
    configuration: boot=normal uuid=053D43B6-2E3C-CEA4-4C52-833DDD1749BE
  *-core
       description: Motherboard
       physical id: 0
     # CPU信息 
     *-cpu
          description: CPU
          product: QEMU Virtual CPU version 2.0.0
          vendor: Intel Corp.
          physical id: 401
          bus info: cpu@0
          slot: CPU 1
          size: 2GHz
          capacity: 2GHz
          width: 64 bits
     # 内存信息
     *-memory
          description: System Memory
          physical id: 1000
          size: 4GiB
     # 硬盘信息
     *-disk
           description: ATA Disk
           product: QEMU HARDDISK
           physical id: 0.0.0
           bus info: scsi@0:0.0.0
           logical name: /dev/sda
           version: 0
           serial: QM00001
           size: 50GiB (53GB)
           *-volume:0
                description: EXT4 volume
                vendor: Linux
                physical id: 1
                bus info: scsi@0:0.0.0,1
                logical name: /dev/sda1
                logical name: /
                version: 1.0
                serial: 65d80188-ddfb-4f08-8018-a0c2e1da8af3
                size: 46GiB
                capacity: 46GiB
           *-volume:1
                description: Extended partition
                physical id: 2
                bus info: scsi@0:0.0.0,2
                logical name: /dev/sda2
                size: 4093MiB
                capacity: 4093MiB
  • 集群环境:3台主机节点
Cluster_Node1: Ubuntu 14.04 -- [IP_Addr]=192.168.1.124
Cluster_Node2: Ubuntu 14.04 -- [IP_Addr]=192.168.1.125
Cluster_Node3: Ubuntu 14.04 -- [IP_Addr]=192.168.1.104

二、部署方法

1. 下载Opendaylight Nitrogen

在每台主机节点上执行如下命令,下载tar.gz格式的压缩包到用户目录下的ODL_N子目录:

odl@mpodl:~/ODL_N$ wget -P . https://nexus.opendaylight.org/content/repositories/public/org/opendaylight/integration/karaf/0.7.2/karaf-0.7.2.tar.gz

或在官网下载,然后通过xftp软件从本地上传到服务器的Opendaylight运行环境中。

官网地址:https://www.opendaylight.org/technical-community/getting-started-for-developers/downloads-and-documentation

2. 安装Opendaylight Nitrogen

在每台主机节点上执行如下命令,解压压缩包到ODL_N子目录:

odl@mpodl:~/ODL_N$ tar -zxvf karaf-0.7.2.tar.gz

解压完成,即认为Opendaylight Nitrogen安装成功。

3. 配置Opendaylight Nitrogen集群

在每台主机节点下执行如下命令,完成集群脚本配置工作:

命令格式:sudo bash ./karaf-0.7.2/bin/configure_cluster.sh [index] [seed_node_list]

其中,[index]为正整数,表示在[seed_node_list]中对应哪个主机节点上配置集群脚本。另外,[seed_node_list]是组成集群的各个主机节点对应的IP地址列表,以空格或逗号隔开。

因此,具体执行命令如下:

# Cluster_Node1: IP=192.168.1.124
odl@mpodl:~/ODL_N$ sudo bash ./karaf-0.7.2/bin/configure_cluster.sh 1 192.168.1.124 192.168.1.125 192.168.1.104

# Cluster_Node2: IP=192.168.1.125
odl@mpodl:~/ODL_N$ sudo bash ./karaf-0.7.2/bin/configure_cluster.sh 2 192.168.1.124 192.168.1.125 192.168.1.104

# Cluster_Node3: IP=192.168.1.104
odl@mpodl:~/ODL_N$ sudo bash ./karaf-0.7.2/bin/configure_cluster.sh 3 192.168.1.124 192.168.1.125 192.168.1.104

当执行完之后,结果如下所示:

# Cluster_Node1: IP=192.168.1.124
odl@mpodl:~/ODL_N$ sudo bash ./karaf-0.7.2/bin/configure_cluster.sh 1 192.168.1.124 192.168.1.125 192.168.1.104
################################################
##             Configure Cluster              ##
################################################
Configuring unique name in akka.conf
Configuring hostname in akka.conf
Configuring data and rpc seed nodes in akka.conf
modules = [

    {
        name = "inventory"
        namespace = "urn:opendaylight:inventory"
        shard-strategy = "module"
    },
    {
        name = "topology"
        namespace = "urn:TBD:params:xml:ns:yang:network-topology"
        shard-strategy = "module"
    },
    {
        name = "toaster"
        namespace = "http://netconfcentral.org/ns/toaster"
        shard-strategy = "module"
    }
]
Configuring replication type in module-shards.conf
################################################
##   NOTE: Manually restart controller to     ##
##         apply configuration.               ##
################################################

备注:

(1)执行如上命令后,会在karaf-0.7.2/configuration目录下生成initial子目录,结果如下所示:

odl@mpodl:~/ODL_N$ ls ./karaf-0.7.2/configuration/
context.xml  factory  initial  logback.xml  tomcat-logging.properties  tomcat-server.xml

odl@mpodl:~/ODL_N$ ls ./karaf-0.7.2/configuration/initial/
akka.conf  modules.conf  module-shards.conf

可见,集群配置脚本在initial子目录下生成了akka.confmodules.confmodule-shards.conf三个配置文件。

(2)查看第一台Ubuntu主机节点的akka.conf文件内容,具体如下所示:

odl@mpodl:~/ODL_N$ cat ./karaf-0.7.2/configuration/initial/akka.conf 

odl-cluster-data {
  akka {
    remote {
      artery {
        enabled = off
        canonical.hostname = "192.168.1.124"   # 本机IP地址
        canonical.port = 2550
      }
      netty.tcp {
        hostname = "192.168.1.124"             # 本机IP地址
        port = 2550
      }
      # when under load we might trip a false positive on the failure detector
      # transport-failure-detector {
        # heartbeat-interval = 4 s
        # acceptable-heartbeat-pause = 16s
      # }
    }

    cluster {
      # Remove ".tcp" when using artery.
      # 集群节点列表
      seed-nodes = ["akka.tcp://opendaylight-cluster-data@192.168.1.124:2550",
                "akka.tcp://opendaylight-cluster-data@192.168.1.125:2550",
                "akka.tcp://opendaylight-cluster-data@192.168.1.104:2550"]

      roles = ["member-1"]

    }

    persistence {
      # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
      # modifying the following two properties. The directory location specified may be a relative or absolute path. 
      # The relative path is always relative to KARAF_HOME.

      # snapshot-store.local.dir = "target/snapshots"
      # journal.leveldb.dir = "target/journal"

      journal {
        leveldb {
          # Set native = off to use a Java-only implementation of leveldb.
          # Note that the Java-only version is not currently considered by Akka to be production quality.

          # native = off
        }
      }
    }
  }
}

同样地,第二台和第三台Ubuntu主机节点的akka.conf文件内容相似,只是对应的IP地址不同。

(3)查看第一台Ubuntu主机节点的module-shards.conf文件内容,具体如下所示:

odl@mpodl:~/ODL_N$ cat ./karaf-0.7.2/configuration/initial/module-shards.conf 
module-shards = [
    {
        name = "default"
        shards = [
            {
                name = "default"
                replicas = ["member-1",
                "member-2",
                "member-3"]
            }
        ]
    },
    {
        name = "inventory"
        shards = [
            {
                name="inventory"
                replicas = ["member-1",
                "member-2",
                "member-3"]
            }
        ]
    },
    {
        name = "topology"
        shards = [
            {
                name="topology"
                replicas = ["member-1",
                "member-2",
                "member-3"]
            }
        ]
    },
    {
        name = "toaster"
        shards = [
            {
                name="toaster"
                replicas = ["member-1",
                "member-2",
                "member-3"]
            }
        ]
    }
]

同样地,第二台和第三台Ubuntu主机节点的module-shards.conf文件内容完全相同。

4. 启动Opendaylight Nitrogen集群

在每台主机节点下执行如下命令,完成Opendaylight节点启动工作:

odl@mpodl:~/ODL_N$ ./karaf-0.7.2/bin/karaf 
karaf: JAVA_HOME not set; results may vary
Apache Karaf starting up. Press Enter to open the shell now...
100% [========================================================================]
Karaf started in 8s. Bundle stats: 208 active, 209 total

    ________                       ________                .__  .__       .__     __       
    \_____  \ ______   ____   ____ \______ \ _____  ___.__.|  | |__| ____ |  |___/  |_     
     /   |   \\____ \_/ __ \ /    \ |    |  \\__  \<   |  ||  | |  |/ ___\|  |  \   __\    
    /    |    \  |_> >  ___/|   |  \|    `   \/ __ \\___  ||  |_|  / /_/  >   Y  \  |      
    \_______  /   __/ \___  >___|  /_______  (____  / ____||____/__\___  /|___|  /__|      
            \/|__|        \/     \/        \/     \/\/            /_____/      \/          


Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown OpenDaylight.

opendaylight-user@root>

然后,执行命令feature:list -i检查odl-mdsal-clustering是否处于已安装状态。如果没有安装,则执行命令feature:install odl-mdsal-clustering完成对应Feature的安装。

opendaylight-user@root>feature:list -i
Name                            | Version | Required | State   | Repository                      | Description
----------------------------------------------------------------------------------------------------------------------------------------------------
odl-mdsal-broker                | 1.6.2   |          | Started | odl-mdsal-1.6.2                 | odl-mdsal-broker
odl-mdsal-clustering            | 1.6.2   | x        | Started | odl-mdsal-clustering            | odl-mdsal-clustering

5. 检查Opendaylight Nitrogen集群是否建立

在每台主机节点下执行如下命令,获取主机节点的角色信息(Leader/Follower):

opendaylight-user@root> ld | grep clustering

于是,在主机节点一,输出如下:

2018-03-21 13:22:45,688 | INFO  | d-dispatcher-125 | Shard                            | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-prefix-configuration-shard-config (Candidate): Starting new election term 21
2018-03-21 13:22:45,741 | INFO  | d-dispatcher-125 | Shard                            | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-prefix-configuration-shard-config (Candidate) :- Switching from behavior Candidate to Leader, election term: 21
2018-03-21 13:22:45,742 | INFO  | ult-dispatcher-5 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-1-shard-prefix-configuration-shard-config , received role change from Candidate to Leader
2018-03-21 13:22:45,821 | INFO  | d-dispatcher-121 | Shard                            | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-prefix-configuration-shard-operational (Candidate): Starting new election term 21
2018-03-21 13:22:45,858 | INFO  | d-dispatcher-125 | Shard                            | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-prefix-configuration-shard-operational (Candidate) :- Switching from behavior Candidate to Leader, election term: 21
2018-03-21 13:22:45,858 | INFO  | ult-dispatcher-5 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-1-shard-prefix-configuration-shard-operational , received role change from Candidate to Leader
2018-03-21 13:22:45,872 | INFO  | d-dispatcher-125 | EntityOwnershipShard             | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-entity-ownership-operational (Candidate): Starting new election term 21
2018-03-21 13:22:45,912 | INFO  | d-dispatcher-145 | EntityOwnershipShard             | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | member-1-shard-entity-ownership-operational (Candidate) :- Switching from behavior Candidate to Leader, election term: 21
2018-03-21 13:22:45,921 | INFO  | lt-dispatcher-21 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-1-shard-entity-ownership-operational , received role change from Candidate to Leader

在主机节点二,输出如下:

2018-03-21 13:22:36,286 | INFO  | ult-dispatcher-4 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-prefix-configuration-shard-config , received role change from null to Follower
2018-03-21 13:22:36,287 | INFO  | ult-dispatcher-4 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-prefix-configuration-shard-operational , received role change from null to Follower
2018-03-21 13:22:36,287 | INFO  | ult-dispatcher-4 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-prefix-configuration-shard-config , registered listener akka://opendaylight-cluster-data/user/shardmanager-config
2018-03-21 13:22:36,287 | INFO  | ult-dispatcher-4 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-prefix-configuration-shard-operational , registered listener akka://opendaylight-cluster-data/user/shardmanager-operational
2018-03-21 13:22:36,305 | INFO  | rd-dispatcher-32 | EntityOwnershipShard             | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | Starting recovery for member-2-shard-entity-ownership-operational with journal batch size 1
2018-03-21 13:22:36,313 | INFO  | rd-dispatcher-38 | EntityOwnershipShard             | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | Recovery completed - Switching actor to Follower - Persistence Id =  member-2-shard-entity-ownership-operational Last index in log = -1, snapshotIndex = -1, snapshotTerm = -1, journal-size = 0
2018-03-21 13:22:36,317 | INFO  | ult-dispatcher-2 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-2-shard-entity-ownership-operational , received role change from null to Follower

在主机节点三,输出如下:

2018-03-21 13:22:39,418 | INFO  | ult-dispatcher-6 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-prefix-configuration-shard-operational , received role change from null to Follower
2018-03-21 13:22:39,418 | INFO  | ult-dispatcher-6 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-prefix-configuration-shard-config , received role change from null to Follower
2018-03-21 13:22:39,418 | INFO  | ult-dispatcher-6 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-prefix-configuration-shard-operational , registered listener akka://opendaylight-cluster-data/user/shardmanager-operational
2018-03-21 13:22:39,418 | INFO  | ult-dispatcher-6 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-prefix-configuration-shard-config , registered listener akka://opendaylight-cluster-data/user/shardmanager-config
2018-03-21 13:22:39,466 | INFO  | rd-dispatcher-23 | EntityOwnershipShard             | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | Starting recovery for member-3-shard-entity-ownership-operational with journal batch size 1
2018-03-21 13:22:39,470 | INFO  | rd-dispatcher-23 | EntityOwnershipShard             | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | Recovery completed - Switching actor to Follower - Persistence Id =  member-3-shard-entity-ownership-operational Last index in log = -1, snapshotIndex = -1, snapshotTerm = -1, journal-size = 0
2018-03-21 13:22:39,473 | INFO  | lt-dispatcher-31 | RoleChangeNotifier               | 120 - org.opendaylight.controller.sal-clustering-commons - 1.6.2 | RoleChangeNotifier for member-3-shard-entity-ownership-operational , received role change from null to Follower

可以看出,第一台Ubuntu主机节点成为Leader,其它两台Ubuntu主机节点成为 Follower ,集群配置成功。但是,从日志也可以看出,对于不同的Shard,存在不同的集群关系。

三、小结

本文详细介绍了Opendaylight Nitrogen集群的搭建指南,后续碰到问题将继续补充。

网友评论

登录后评论
0/500
评论
adoryn
+ 关注