pig笔记

  1. 云栖社区>
  2. 博客>
  3. 正文

pig笔记

xiaohei.info 2015-03-26 22:03:21 浏览754
展开阅读全文
版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/qq1010885678/article/details/44657011
1.安装Pig
将pig添加到环境变量当中

2.pig使用
首先将数据库中的数据导入到HDFS上
sqoop import --connect jdbc:mysql://192.168.1.10:3306/cloud --username root --


password JChubby123  --table trade_detail --target-dir '/sqoop/td'
sqoop import --connect jdbc:mysql://192.168.1.10:3306/cloud --username root --


password JChubby123  --table user_info --target-dir '/sqoop/ui'
 

td = load '/sqoop/td' using PigStorage(',') as (id:long, account:chararray, 


income:double, expenses:double, time:chararray);
ui = load '/sqoop/ui' using PigStorage(',') as (id:long, account:chararray, 


name:chararray, age:int);

td1 = foreach td generate account, income, expenses, income-expenses as surplus;

td2 = group td1 by account;

td3 = foreach td2 generate group as account, SUM(td1.income) as income, SUM


(td1.expenses) as expenses, SUM(td1.surplus) as surplus;

tu = join td3 by account, ui by account;

result = foreach tu generate td3::account as account, ui::name, td3::income, 


td3::expenses, td3::surplus;

store result into '/result' using PigStorage(',');

网友评论

登录后评论
0/500
评论
xiaohei.info
+ 关注