Spark Practices Chapter 3

Here in this part of the series, we will cover some basic PySpark operations.

here we'll see how to :

Create a Spark Session

Create a DataFrame

Read a CSV file

Filter on records

Calculate Average

See list of columns and DataTypes