Issue for loading Hive table using PIG

Question

I want to use PIG join 3 tables then load into Hive table. before store the data, I can show the data in screen using dump. I use below command to store the data, it failed.

store jnr_mas_ins INTO 'table_name' USING org.apache.hive.hcatalog.pig.HCatStorer();

the error is:

2017-01-25 10:58:48,487 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1002: Unable to store alias jnr_mas_ins
Details at logfile: /export/home/pig_1485359450881.log

I searched this site, somebody suggested to use below command:

store jnr_mas_ins INTO 'hdfs://fullpath' USING org.apache.hive.hcatalog.pig.HCatStorer();


001 : Error setting output information. Cause : NoSuchObjectException(message:default.hdfs://fullpath table not found)
Details at logfile: /export/home/pig_1485359450881.log

would you please help to resolve this issue, thanks in advance below is script I used for populating the table.

account_master_ALL = LOAD 'ACCT' USING org.apache.hive.hcatalog.pig.HCatLoader(); account_master_fil = filter account_master_ALL by croft_source_sys_cd == ''ABC''; account_master = foreach account_master_fil generate acct_master_key, (trans_date is null? '-999': trans_date) as trans_date, (acct_data_1 is null? '-999': acct_data_1) as acct_data_1,(acct_data_2 is null? '-999': acct_data_2) as acct_data_2, (acct_data_3 is null? '-999': acct_data_3) as acct_data_3, (acct_data_4 is null? '-999': acct_data_4) as acct_data_4, (acct_data_5 is null? '-999': acct_data_5) as acct_data_5, acct_m_ref_key, croft_source_sys_cd; acct_detail_all = LOAD 'acct_detail_tb' USING org.apache.hive.hcatalog.pig.HCatLoader(); acct_detail = foreach acct_detail_all generate c_number, b_number, a_number, c_coded, a_type, c_digit, s_number, r_type, r_cd, s_qty, s_b_cost, trade_qty, t_cost, p_date, load_date; jnr_all = join acct_detail by (a_number, b_number, c_coded, a_type, c_number, c_digit), account_master by (acct_data_1, trans_date, acct_data_2, acct_data_3, acct_data_4, acct_data_5); ins_master_all = LOAD 'ins_acct' USING org.apache.hive.hcatalog.pig.HCatLoader(); jnr_mas_ins_all = join jnr_all by (acct_detail::s_number), ins_master_all by (acct_data_1); jnr_mas_ins = foreach jnr_mas_ins_all generate acct_master_key as acct_master_key, ins_master_key as ins_master_key, 'ABC' as code_dev, c_number as c_number, b_number as b_number, a_number as src_a_number, c_coded as c_coded, a_type as a_type, c_digit as c_digit, s_number as s_number, r_type as r_type, r_cd as r_cd, s_qty as s_qty, s_b_cost as s_b_cost, trade_qty as trade_qty, t_cost as t_cost, p_date as p_date, CurrentTime() as load_date, 'PIG_USER' as create_userid, CurrentTime() as create_date, CurrentTime() as f_date; store jnr_mas_ins INTO 'tartet_table' USING org.apache.hive.hcatalog.pig.HCatStorer();

thanks inquisitive_mind, I just updated the first stream by adding my script on it. please take a look. thanks. — ken Han

VK_217 VK_217 · Accepted Answer · 2017-01-25T17:12:30

Looks like you are missing the relation jnr_mas_ins_all.Where is this coming from?jnr_mas_ins will be emtpy since there is no relation jnr_mas_ins_all.It should be ins_master_all?

ins_master_all = LOAD 'ins_acct' USING org.apache.hive.hcatalog.pig.HCatLoader();
jnr_mas_ins = foreach jnr_mas_ins_all generate acct_master_key as acct_master_key, ins_master_key as ins_master_key, 'ABC' as code_dev, c_number as c_number, b_number as b_number, a_number as src_a_number, c_coded as c_coded, a_type as a_type, c_digit as c_digit, s_number as s_number, r_type as r_type, r_cd as r_cd, s_qty as s_qty, s_b_cost as s_b_cost, trade_qty as trade_qty, t_cost as t_cost, p_date as p_date, CurrentTime() as load_date, 'PIG_USER' as create_userid, CurrentTime() as create_date, CurrentTime() as f_date;

Issue for loading Hive table using PIG

1 Answers