Raja's Data Engineering
133: Databricks Certification: Data Engineer Associate - PII Comment
Raja's Data Engineering
132: DataBricks Learning: System Variable _SQLDF
Raja's Data Engineering
131. Databricks | Pyspark| Built-in Function: ZIP_WITH
Raja's Data Engineering
130. Databricks | Pyspark| Delta Lake: Change Data Feed
Raja's Data Engineering
129. Databricks | Pyspark| Delta Lake: Deletion Vectors
Raja's Data Engineering
128. Databricks | Pyspark| Built-In Function: TRANSFORM
Raja's Data Engineering
127. Databricks | Pyspark| SQL Coding Interview:LeetCode-1045: Customers Who Bought All Products
Raja's Data Engineering
126. Databricks | Pyspark | Downloading Files from Databricks DBFS Location
Raja's Data Engineering
125. Databricks | Pyspark| Delta Live Table: Data Quality Check - Expect
Raja's Data Engineering
124. Databricks | Pyspark| Delta Live Table: Datasets - Tables and Views
Raja's Data Engineering
123. Databricks | Pyspark| Delta Live Table: Declarative VS Procedural
Raja's Data Engineering
122. Databricks | Pyspark| Delta Live Table: Introduction
Raja's Data Engineering
121. Databricks | Pyspark| AutoLoader: Incremental Data Load
Raja's Data Engineering
120. Databricks | Pyspark| SQL Coding Interview: Employees Earning More Than Department Avg Salary
Raja's Data Engineering
119. Databricks | Pyspark| Spark SQL: Except Columns in Select Clause
Raja's Data Engineering
118. Databricks | PySpark| SQL Coding Interview: Employees Earning More than Managers
Raja's Data Engineering
117. Databricks | Pyspark| SQL Coding Interview: Total Grand Slam Titles Winner
Raja's Data Engineering
116. Databricks | Pyspark| Query Dataframe Using Spark SQL
Raja's Data Engineering
115. Databricks | Pyspark| SQL Coding Interview: Number of Calls and Total Duration
Raja's Data Engineering
114. Databricks | Pyspark| Performance Optimization: Re-order Columns in Delta Table
Raja's Data Engineering
113. Databricks | PySpark| Spark Reader: Skip Specific Range of Records While Reading CSV File
Raja's Data Engineering
112. Databricks | Pyspark| Spark Reader: Skip First N Records While Reading CSV File
Raja's Data Engineering
111. Databricks | Pyspark| SQL Coding Interview: Exchange Seats of Students
Raja's Data Engineering
110. Databricks | Pyspark| Spark Reader: Reading Fixed Length Text File
Raja's Data Engineering
109. Databricks | Pyspark| Coding Interview Question: Pyspark and Spark SQL
Raja's Data Engineering
108. Databricks | Pyspark| Window Function: First and Last
Raja's Data Engineering
107. Databricks | Pyspark| Transformation: Subtract vs ExceptAll
Raja's Data Engineering
106.Databricks|Pyspark|Automation|Real Time Project:DataType Issue When Writing to Azure Synapse/SQL
Raja's Data Engineering
105. Databricks | Pyspark |Pyspark Development: Spark/Databricks Interview Question Series - V
Raja's Data Engineering
104. Databricks | Pyspark |Pyspark Development: Spark/Databricks Interview Question Series - IV
Raja's Data Engineering
103. Databricks | Pyspark |Delta Lake: Spark/Databricks Interview Question Series - III
Raja's Data Engineering
102. Databricks | Pyspark |Performance Optimization: Spark/Databricks Interview Question Series - II
Raja's Data Engineering
101. Databricks | Pyspark |Core/Architecture: Spark/Databricks Interview Question Series - I
Raja's Data Engineering
100. Databricks | Pyspark | Spark Architecture: Internals of Partition Creation Demystified
Raja's Data Engineering
99. Databricks | Pyspark | Real Time Use Case: Generate Test Data - Array_Repeat()
Raja's Data Engineering
98. Databricks | Pyspark | Interview Question: Pyspark VS Pandas
Raja's Data Engineering
97. Databricks | Pyspark | Data Security: Enforcing Column Level Encryption
Raja's Data Engineering
96. Databricks | Pyspark | Real Time Scenario | Schema Comparison
Raja's Data Engineering
95. Databricks | Pyspark | Schema | Different Methods of Schema Definition
Raja's Data Engineering
94. Databricks | Pyspark | Interview Question | Schema Definition: Struct Type vs Map Type
Raja's Data Engineering
93. Databricks | Pyspark | Interview Question | Schema Definition: Struct Type vs Struct Field
Raja's Data Engineering
92. Databricks | Pyspark | Interview Question | Performance Optimization: Select vs WithColumn
Raja's Data Engineering
91. Databricks | Pyspark | Interview Question |Handlining Duplicate Data: DropDuplicates vs Distinct
Raja's Data Engineering
90. Databricks | Pyspark | Interview Question: Read Excel File with Multiple Sheets
Raja's Data Engineering
89. Databricks | Pyspark | Notebook Scheduling through Event Based Trigger using Azure Data Factory
Raja's Data Engineering
88. Databricks |Pyspark |Notebook Scheduling through Schedule Based Trigger using Azure Data Factory
Raja's Data Engineering
87. Databricks | Pyspark | Real Time Project: ETL Pipeline Integrating ADF, ASQL, ADLS, Key Vault
Raja's Data Engineering
86. Databricks | Pyspark | Notebook Activity in Azure Data Factory with Output Parameter
Raja's Data Engineering
85. Databricks | Pyspark | Notebook Activity in Azure Data Factory with Input Parameter
Raja's Data Engineering
84. Databricks | Pyspark | Azure Data Factory + Azure Databricks: Execute Notebook Via ADF
Raja's Data Engineering
83. Databricks | Pyspark | Databricks Workflows: Job Scheduling
Raja's Data Engineering
82. Databricks | Pyspark | Databricks Secret Scopes: Azure Key Vault Backed Secrets
Raja's Data Engineering
81. Databricks | Pyspark | Workspace Object Access Control
Raja's Data Engineering
80. Databricks | Pyspark | Tips: Write Dataframe into Single File with Specific File Name
Raja's Data Engineering
79. Databricks | Pyspark | Split Array Elements into Separate Columns
Raja's Data Engineering
78. Databricks | Pyspark | Performance Optimization: Delta Cache
Raja's Data Engineering
77. Databricks | Pyspark | Create_map(): Convert Dataframe Columns to Dictionary (Map Type)
Raja's Data Engineering
76. Databricks|Pyspark:Interview Question|Scenario Based|Max Over () Get Max value of Duplicate Data
Raja's Data Engineering
75. Databricks | Pyspark | Performance Optimization - Bucketing
Raja's Data Engineering
74. Databricks | Pyspark | Interview Question: Sort-Merge Join (SMJ)
Raja's Data Engineering
73. Databricks | Pyspark | UDF to Check if Folder Exists
Raja's Data Engineering
72. Databricks | Pyspark | Interview Question: Explain Plan
Raja's Data Engineering
71. Databricks | Pyspark | Window Functions: Lead and Lag
Raja's Data Engineering
67. Databricks | Pypark | Delta: Schema Evolution - MergeSchema
Raja's Data Engineering
66. Databricks | Pyspark | Delta: Z-Order Command
Raja's Data Engineering
65. Databricks | Pyspark | Delta Lake: Vacuum Command
Raja's Data Engineering
64. Databricks | Pyspark | Delta Lake: Optimize Command - File Compaction
Raja's Data Engineering
63. Databricks | Pyspark| Delta Lake: Restore Command
Raja's Data Engineering
62. Databricks | Pyspark | Delta Lake: Time Travel
Raja's Data Engineering
61. Databricks | Pyspark | Delta Lake : Slowly Changing Dimension (SCD Type2)
Raja's Data Engineering
58. Databricks | Pyspark | Delta Lake : Update Delta Table
Raja's Data Engineering
57. Databricks| Pyspark| Delta Lake: Different Approaches to Delete Data from Delta Table
Raja's Data Engineering
56. Databricks| Pyspark | Delta Lake: Different Approaches to Insert Data Into Delta Table
Raja's Data Engineering
55. Databricks| Pyspark| Delta Lake: Delta Table Instance
Raja's Data Engineering
53. Databricks| Pyspark| Delta Lake: Solution Architecture
Raja's Data Engineering
52. Databricks| Pyspark| Delta Lake Architecture: Internal Working Mechanism
Raja's Data Engineering
51. Databricks | Pyspark | Delta Lake: Introduction to Delta Lake
Raja's Data Engineering
54. Databricks | Delta Lake| Pyspark: Create Delta Table Using Various Methods
Raja's Data Engineering
70. Databricks| Pyspark| Input_File_Name: Identify Input File Name of Corrupt Record
Raja's Data Engineering
69. Databricks | Spark | Pyspark | Data Skewness| Interview Question: SPARK_PARTITION_ID
Raja's Data Engineering
38. Databricks | Pyspark | Interview Question | Compression Methods: Snappy vs Gzip
Raja's Data Engineering
50. Databricks | Pyspark: Greatest vs Least vs Max vs Min
Raja's Data Engineering
37. Databricks | Pyspark: Dataframe Checkpoint
Raja's Data Engineering
68. Databricks | Pyspark | Dataframe InsertInto Delta Table
Raja's Data Engineering
36. Databricks: Autoscaling | Optimized Autoscaling
Raja's Data Engineering
49. Databricks & Spark: Interview Question(Scenario Based) - How many spark jobs get created?
Raja's Data Engineering
35. Databricks & Spark: Interview Question - Shuffle Partition
Raja's Data Engineering
19. Databricks & Pyspark: Real Time ETL Pipeline Azure SQL to ADLS
Raja's Data Engineering
17. Databricks & Pyspark: Azure Data Lake Storage Integration with Databricks
Raja's Data Engineering
20. Databricks & Pyspark: Azure Key Vault Integration
Raja's Data Engineering
18. Databricks & Pyspark: Ingest Data from Azure SQL Database
Raja's Data Engineering
60. Databricks & Pyspark: Delta Lake Audit Log Table with Operation Metrics
Raja's Data Engineering
59. Databricks Pyspark:Slowly Changing Dimension|SCD Type1| Merge using Pyspark and Spark SQL
Raja's Data Engineering
48. Databricks - Pyspark: Find Top or Bottom N Rows per Group
Raja's Data Engineering
34. Databricks - Spark: Data Skew Optimization
Raja's Data Engineering
47. Databricks | Spark | Pyspark | Null Count of Each Column in Dataframe
Raja's Data Engineering
46. Databricks | Spark | Pyspark | Number of Records per Partition in Dataframe
Raja's Data Engineering
16. Databricks | Spark | Pyspark | Bad Records Handling | Permissive;DropMalformed;FailFast
Raja's Data Engineering
33. Databricks | Spark | Pyspark | UDF
Raja's Data Engineering
45. Databricks | Spark | Pyspark | PartitionBy