MP4 | Video: h264, 1280x720 | Audio: AAC, 44.1 KHz, 2 Ch

Genre: eLearning | Language: English | Duration: 35 Lectures ( 1h 48m ) | Size: 1.1 GB

This course is based on real world problems in PySpark, surrounding Data Cleaning, Descriptive statistics, Classification and Regression Modeling. The first segment introduces descriptive statistics in PySpark and computing fundamental measures such as mean, standard deviation and generating an extended statistical summary. The second segment is based on cleaning the data in PySpark, working with null values,  redundant data and imputing the null values. The third segment is about Predictive modeling with PySpark using Gradient Boosted Trees Regression The fourth and fifth segments  are based on applying classification techniques in PySpark. The fourth Segment introduces the application of Spark XGB Classifier for a classification problem and the fifth segment is about using a deep learning model for text sentiment classification. The sixth segment is about time series analytics and modeling using PySpark and Prophet The seventh segment introduces  Spark SQL for data querying and analysis. These segments also include advanced visualization techniques through Seaborn and Plotly libraries including  Box plots to understand the distribution of the data and assessment of outliers, Count plots to understand balance in the proportion of data, Bar chart to represent feature importance as part of the Gradient Boosted Trees Regression Model, Word Cloud for text analytics and analyzing time series data to extract seasonality and trend components. Each of these segments, has a Google Colab notebook included aligning with the lecture.