Loading...

Spark SQL - Windowing Functions - Overview

3,819 views 23________

Let us understand Functions related to aggregations, ranking and windowing functions.

To get access to complete course which contains videos, reference material, code, labs and support, - you need to purchase https://kaizen.itversity.com/shop/all...

We use the functions in SELECT clause.
Specification: function() OVER (PARTITION BY column [ORDER BY
column])
PARTITION BY is used to group the data based on a column.
ORDER BY is used to sort the data based on a column.
Example: rank() OVER (PARTITION BY department_id ORDER BY
salary DESC)
Aggregations – sum, avg, min, max etc
Ranking – rank, dense_rank, row_number etc
Windowing – lead, lag etc
Window have APIs such as PARTITION BY, ORDER BY
For aggregations, we can define the group by using PARTITION
BY
For ranking or windowing, we need to use PARTITION BY and then
ORDER BY. PARTITION BY is to group the data and ORDER BY is to
sort the data to assign rank.

Connect with me or follow me at
  / durga0gadiraju  
  / itversity  
https://github.com/dgadiraju
   / itversityin  
  / itversity  

コメント