hive sql之 視窗函式。

一、row_number 函式

區別於其它排序函式，相當於多了乙個排序的列。

資料準備：

name money

ming 12

yang 23

ming 35

ming 54

yang 43

1、按照其中money 進行排序並增加一行。

select name,money,

row_number
() over (order by money) from lei_test;
結果：ming 12
1yang 23
2ming 35
3yang 43
4ming 54
5

select name,money ,

row_number()
over
(partition by name order by money) from lei_test;
ming 12
1ming 35
2ming 54
3yang 23
1yang 43
2

select name,money,tb from (select name,money,

row_number
() over (partition by name order by money)
) tb from lei_test ) where tb =1;
結果：ming 12
1yang 23
1

1、rank() 函式和row_number 函式類似就是他會跳過成績相同的資料（如果 按照 name partiton並按照money進行order by 的化他相同的money 就會 row_number 會相同）

例如：

select name、money,

rank
() over (partition by name order by money) from lei_test;
結果為：
ming 12
1ming 12
1ming 35
3ming 54
4yang 23
1yang 43
2

dense_rank 函式和row_number函式類似。（區別：相同成績會並列，序列排序）

select name,money,

dense_rank()
over
(partition by name order by money) from lei_test;
結果ming 12
1ming 12
1ming 35
2ming 54
3yang 23
1yang 43
2

select name,money cume_dist

() over (partition by name order by money) from lei_test;
結果為：
ming 12
0.5ming 12
0.5ming 35
0.75
ming 54
1.0yang 23
0.5yang 43
1.0

select name,money,

percent_rank()
over
(partition by name order by money) from lei_test;
結果為ming 12
0.0ming 12
0.0ming 35
0.6666666666666666
ming 54
1.0yang 23
0.0yang 43
1.0

select name,money ,

ntile(2
) over (partition by name order by money) from lei_test;
結果為：
ming 12
1ming 12
1ming 35
2ming 54
2yang 23
1yang 43
2

1、leag(col，n,default)用於統計視窗內往上的n行值（上面可能只是給個 排序行數 這個返回的值）

select name,money,lead（money,

1）over (partition by name order by money) from lei_test;
返回結果：
ming 12
12ming 12
35ming 35
54ming 54 null
yang 23
43yang 43 null

select name,money,lag(money,1)

over(partition by name order by money)

from lei_test;

2、得到排序後的第乙個值

select name,money first_value

(money) over (partition by name ordre by money) from lei_test;
返回結果為：
ming 12
12ming 12
12ming 35
12ming 54
12yang 23
23yang 43
23

select name.money,

last_value
(money) over (partition by name order by money) from lei_test:
返回的結果為
ming 12
12ming 12
12ming 35
35ming 54
54yang 23
23yang 43
43

Hive之視窗函式

對一定視窗期內的資料進行聚合。示例 select sum a.pv over partition by cookieid order by create time rows between 3 preceding and current row as pv1,sum a.pv over partit...

hive視窗函式 Hive sql視窗函式原始碼分析

在了解了視窗函式實現原理 spark hive中視窗函式實現原理覆盤 和 sparksql比hivesql優化的點 視窗函式 之後，今天又擼了一遍hive sql 中視窗函式的原始碼實現，寫個筆記記錄一下。簡單來說，視窗查詢有兩個步驟 將記錄分割成多個分割槽 然後在各個分割槽上呼叫視窗函式。傳統的 ...

Scala之視窗函式排序

scala視窗函式這排序rank，重複排序和不重複排序 1 引入包 import org.apache.spark.sql.expressions.window import spark.implicits.2.建立測試的df，可以直接貼上測試。3.選擇按name分組，按score排序，且倒序。va...