hive array, map, struct use

Keywords: hive

Links to the original text: https://my.oschina.net/zipu888/blog/549644

Structs: Data within structs can be accessed through DOT (.). For example, the type of column c in the table is STRUCT{a INT; b INT}, and we can access domain a through c.a.
Maps (K-V pairs): Access to a specified domain can be done by ["designated domain name"]. For example, a Map M contains a kv pair of group-"gid, and the value of GID can be obtained by M['group'].
Arrays: The data in array is of the same type. For example, if the elements in array A ['a','b','c'], then the value of A[1] is'b'.

Struct use

Building tables:

hive> create table student_test(id INT, info struct<name:STRING, age:INT>)
    > ROW FORMAT DELIMITED FIELDS TERMINATED BY ','                       
    > COLLECTION ITEMS TERMINATED BY ':';                                 
OK
Time taken: 0.446 seconds
'FIELDS TERMINATED BY': A separator between fields
''COLLECTION ITEMS TERMINATED BY': A separator for each item in a field
Import data:
$ cat test5.txt 
1,zhou:30
2,yan:30
3,chen:20
4,li:80
hive> LOAD DATA LOCAL INPATH '/home/work/data/test5.txt' INTO TABLE student_test;
Copying data from file:/home/work/data/test5.txt
Copying file: file:/home/work/data/test5.txt
Loading data to table default.student_test
OK
Time taken: 0.35 seconds
Enquiries:
hive> select info.age from student_test;
Total MapReduce jobs = 1
......
Total MapReduce CPU Time Spent: 490 msec
OK
30
30
20
80
Time taken: 21.677 seconds

Array use
Building tables:
hive> create table class_test(name string, student_id_list array<INT>)
    > ROW FORMAT DELIMITED                                            
    > FIELDS TERMINATED BY ','                                        
    > COLLECTION ITEMS TERMINATED BY ':';                             
OK
Time taken: 0.099 seconds
Import data:
$ cat test6.txt 
034,1:2:3:4
035,5:6
036,7:8:9:10
hive>  LOAD DATA LOCAL INPATH '/home/work/data/test6.txt' INTO TABLE class_test ;
Copying data from file:/home/work/data/test6.txt
Copying file: file:/home/work/data/test6.txt
Loading data to table default.class_test
OK
Time taken: 0.198 seconds
Enquiries:
hive> select student_id_list[3] from class_test;
Total MapReduce jobs = 1
......
Total MapReduce CPU Time Spent: 480 msec
OK
4
NULL
10
Time taken: 21.574 seconds

Map use
Building tables:
hive> create table employee(id string, perf map<string, int>)     
    > ROW FORMAT DELIMITED                                        
    > FIELDS TERMINATED BY '\t'                              
    > COLLECTION ITEMS TERMINATED BY ','                     
    > MAP KEYS TERMINATED BY ':';                                  
OK
Time taken: 0.144 seconds
'MAP KEYS TERMINATED BY': key value separator

Import data:
$ cat test7.txt 
1       job:80,team:60,person:70
2       job:60,team:80
3       job:90,team:70,person:100
hive>  LOAD DATA LOCAL INPATH '/home/work/data/test7.txt' INTO TABLE employee;
Enquiries:
hive> select perf['person'] from employee;
Total MapReduce jobs = 1
......
Total MapReduce CPU Time Spent: 460 msec
OK
70
NULL
100
Time taken: 20.902 seconds
hive> select perf['person'] from employee where perf['person'] is not null;   
Total MapReduce jobs = 1
.......
Total MapReduce CPU Time Spent: 610 msec
OK
70
100
Time taken: 21.989 seconds
hive> 
 

Reproduced in: https://my.oschina.net/zipu888/blog/549644

Posted by mortimerdude on Wed, 02 Oct 2019 17:08:18 -0700