目录
- 4.1. Hypertable 安装
-
- 4.1.1. Hypertable standalone 单机安装
- 4.1.2. Hypertable on HDFS(hadoop) 安装
- 4.1.3. MapR
- 4.1.4. Ceph
- 4.1.5. 检验安装
- 4.2. Code examples
-
- 4.2.1. PHP
- 4.3. HQL
-
- 4.3.1. namespace 命名空间管理
- 4.3.2. Table 表
- 4.4. FAQ
-
- 4.4.1. 切换 DFS Broker
http://hypertable.org/
4.1. Hypertable 安装
4.1.1. Hypertable standalone 单机安装
过程 4.1. Hypertable standalone 安装过程
-
安装 Hypertable 软件包
# cd /usr/local/src/ # wget http://cdn.hypertable.com/packages/0.9.7.0/hypertable-0.9.7.0-linux-x86_64.rpm
-
安装 Hypertable， 我个人比较喜欢用yum localinstall他会解决软件之间的依赖关系
# yum localinstall hypertable-0.9.7.0-linux-x86_64.rpm
相关的软件会自动安装
Dependencies Resolved ==================================================================================================================== Package Arch Version Repository Size ==================================================================================================================== Installing: hypertable x86_64 0.9.7.0-1 /hypertable-0.9.7.0-linux-x86_64 288 M Installing for dependencies: mailcap noarch 2.1.31-2.el6 base 27 k perl-Bit-Vector x86_64 7.1-2.el6 base 169 k perl-Carp-Clan noarch 6.03-2.el6 base 25 k perl-Compress-Raw-Zlib x86_64 1:2.020-127.el6 base 68 k perl-Compress-Zlib x86_64 2.020-127.el6 base 43 k perl-HTML-Parser x86_64 3.64-2.el6 base 109 k perl-HTML-Tagset noarch 3.20-4.el6 base 17 k perl-IO-Compress-Base x86_64 2.020-127.el6 base 67 k perl-IO-Compress-Zlib x86_64 2.020-127.el6 base 134 k perl-IO-String noarch 1.08-9.el6 base 15 k perl-URI noarch 1.40-2.el6 base 117 k perl-libwww-perl noarch 5.833-2.el6 base 387 k Transaction Summary ==================================================================================================================== Install 13 Package(s)
-
Hypertable 默认安装在 /opt/hypertable/0.9.7.0
备份配置文件，
# cd /opt/hypertable/0.9.7.0/conf # cp hypertable.cfg hypertable.cfg.original
-
FHS-IZE 安装
# bin/fhsize.sh Setting up /var/opt/hypertable Setting up /etc/opt/hypertable fshize /opt/hypertable/0.9.7.0: success
-
设计 "CURRENT" 连接
# cd /opt/hypertable # ln -s 0.9.7.0 current
-
安装 notification-hook.sh 脚本.
# cp conf/notification-hook.tmpl conf/notification-hook.sh # chmod o+x conf/notification-hook.sh
测试 notification-hook.sh脚本 .
/opt/hypertable/current/conf/notification-hook.sh "Test Message" "This is a test."
-
启动 hypertable
# /opt/hypertable/current/bin/start-all-servers.sh local DFS broker: available file descriptors: 1024 Started DFS Broker (local) Started Hyperspace Started Hypertable.Master /proc/sys/vm/swappiness = 60 Started Hypertable.RangeServer Started ThriftBroker
# /opt/hypertable/current/bin/ht shell Welcome to the hypertable command interpreter. For information about Hypertable, visit http://hypertable.com Type 'help' for a list of commands, or 'help shell' for a list of shell meta commands. hypertable>
-
测试安装是否有效
# /opt/hypertable/current/bin/ht shell Welcome to the hypertable command interpreter. For information about Hypertable, visit http://hypertable.com Type 'help' for a list of commands, or 'help shell' for a list of shell meta commands. hypertable> help USE ................ Sets the current namespace CREATE NAMESPACE ... Creates a new namespace DROP NAMESPACE ..... Removes a namespace EXISTS TABLE ....... Check if table exists CREATE TABLE ....... Creates a table DELETE ............. Deletes all or part of a row from a table DESCRIBE TABLE ..... Displays a table's schema DROP TABLE ......... Removes a table RENAME TABLE ....... Renames a table DUMP TABLE ......... Create efficient backup file ALTER TABLE ........ Add/remove column family from existing table INSERT ............. Inserts data into a table LOAD DATA INFILE ... Loads data from a TSV input file into a table SELECT ............. Selects (and display) cells from a table SHOW CREATE TABLE .. Displays CREATE TABLE command used to create table SHOW TABLES ........ Displays only the list of tables in the current namespace GET LISTING ........ Displays the list of tables and namespace in the current namespace Statements must be terminated with ';'. For more information on a specific statement, type 'help <statement>', where <statement> is from the preceding list. hypertable>quit
-
停止 hypertable
运行下列命令停止 Hypertable
$ /opt/hypertable/current/bin/stop-servers.sh
4.1.2. Hypertable on HDFS(hadoop) 安装
Hadoop - HDFS 安装指南
过程 4.2. Hypertable on HDFS
-
创建工作目录
$ hadoop fs -mkdir /hypertable $ hadoop fs -chmod 777 /hypertable
-
安装 Java 运行环境
yum install java-1.7.0-openjdk yum localinstall http://ftp.cuhk.edu.hk/pub/packages/apache.org/hadoop/common/hadoop-1.1.2/hadoop-1.1.2-1.x86_64.rpm
-
修改 jrun bug
cp /opt/hypertable/current/bin/jrun /opt/hypertable/current/bin/jrun.old vim /opt/hypertable/current/bin/jrun #HT_JAR=`ls -1 /opt/hypertable/doug/current/lib/java/*.jar | grep "hypertable-[^-]*.jar" | awk 'BEGIN {FS="/"} {print $NF}'` HT_JAR=`ls -1 /opt/hypertable/current/lib/java/*.jar | grep "hypertable-[^-]*.jar" | awk 'BEGIN {FS="/"} {print $NF}'`export JAVA_HOME=/usr export HADOOP_HOME=/usr export HYPERTABLE_HOME=/opt/hypertable/current
-
hypertable.cfg
# cat conf/hypertable.cfg # # hypertable.cfg # # HDFS Broker #HdfsBroker.Hadoop.ConfDir=/etc/hadoop/conf HdfsBroker.Hadoop.ConfDir=/etc/hadoop # Ceph Broker CephBroker.MonAddr=192.168.6.2:6789 # Local Broker DfsBroker.Local.Root=fs/local # DFS Broker - for clients DfsBroker.Port=38030 # Hyperspace Hyperspace.Replica.Host=localhost Hyperspace.Replica.Port=38040 Hyperspace.Replica.Dir=hyperspace # Hypertable.Master #Hypertable.Master.Host=localhost Hypertable.Master.Port=38050 # Hypertable.RangeServer Hypertable.RangeServer.Port=38060 Hyperspace.KeepAlive.Interval=30000 Hyperspace.Lease.Interval=1000000 Hyperspace.GracePeriod=200000 # ThriftBroker ThriftBroker.Port=38080
Hadoop 配置文件 /etc/hadoop/core-site.xml
# cat /etc/hadoop/core-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.default.name</name> <value>hdfs://namenode.example.com:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/var/tmp/hadoop</value> </property> </configuration>Hadoop 配置文件 /etc/hadoop/hdfs-site.xml
# cat /etc/hadoop/hdfs-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.name.dir</name> <value>/var/hadoop/name1</value> <description> </description> </property> <property> <name>dfs.data.dir</name> <value>/var/hadoop/hdfs/data1</value> <description> </description> </property> <property> <name>dfs.replication</name> <value>2</value> </property> </configuration> -
启动 dfsbroker
# /opt/hypertable/current/bin/set-hadoop-distro.sh cdh4 Hypertable successfully configured for Hadoop cdh4
# /opt/hypertable/current/bin/start-dfsbroker.sh hadoop DFS broker: available file descriptors: 1024 Started DFS Broker (hadoop)
查看启动日志
# tail -f /opt/hypertable/current/log/DfsBroker.hadoop.log log4j:WARN No appenders could be found for logger (org.apache.hadoop.conf.Configuration). log4j:WARN Please initialize the log4j system properly. HdfsBroker.dfs.client.read.shortcircuit=false HdfsBroker.dfs.replication=2 HdfsBroker.Server.fs.default.name=hdfs://namenode.example.com:9000 Apr 23, 2013 6:43:18 PM org.hypertable.AsyncComm.IOHandler DeliverEvent INFO: [/192.168.6.25:53556 ; Tue Apr 23 18:43:18 HKT 2013] Connection Established Apr 23, 2013 6:43:18 PM org.hypertable.DfsBroker.hadoop.ConnectionHandler handle INFO: [/192.168.6.25:53556 ; Tue Apr 23 18:43:18 HKT 2013] Disconnect - COMM broken connection : Closing all open handles from /192.168.6.25:53556 Closed 0 input streams and 0 output streams for client connection /192.168.6.25:53556
4.1.3. MapR
4.1.4. Ceph
修改 CephBroker.MonAddr 对应的服务器与端口号即可
# cat hypertable.cfg # # hypertable.cfg # # HDFS Broker HdfsBroker.Hadoop.ConfDir=/etc/hadoop/conf # Ceph Broker CephBroker.MonAddr=192.168.6.2:6789 # Local Broker DfsBroker.Local.Root=fs/local # DFS Broker - for clients DfsBroker.Port=38030 # Hyperspace Hyperspace.Replica.Host=localhost Hyperspace.Replica.Port=38040 Hyperspace.Replica.Dir=hyperspace # Hypertable.Master Hypertable.Master.Port=38050 # Hypertable.RangeServer Hypertable.RangeServer.Port=38060 Hyperspace.KeepAlive.Interval=30000 Hyperspace.Lease.Interval=1000000 Hyperspace.GracePeriod=200000 # ThriftBroker ThriftBroker.Port=38080
启动 dfsbroker
# /opt/hypertable/current/bin/start-dfsbroker.sh ceph
4.1.5. 检验安装
创建一个表
# echo "USE '/'; CREATE TABLE foo ( c1, c2 ); GET LISTING;" \ > | /opt/hypertable/current/bin/ht shell --batch foo sys (namespace) tmp (namespace)
插入一些数据
# echo "USE '/'; INSERT INTO foo VALUES('001', 'c1', 'very'), \
> ('000', 'c1', 'Hypertable'), ('001', 'c2', 'easy'), ('000', 'c2', 'is');" \
> | /opt/hypertable/current/bin/ht shell --batch
查询数据
# echo "USE '/'; SELECT * FROM foo;" \ > | /opt/hypertable/current/bin/ht shell --batch 000 c1 Hypertable 000 c2 is 001 c1 very 001 c2 easy
如果你想清楚所有表运行下面命令
$ /opt/hypertable/current/bin/stop-servers.sh $ /opt/hypertable/current/bin/clean-database.sh
原文出处：Netkiller 系列 手札
本文作者：陈景峯
转载请与作者联系，同时请务必标明文章原始出处和作者信息及本声明。
