Spark Data Frames and Data Sets - Getting Started --Create Data Frames Using Files

Spark support multiple file formats out of the box. We will just go through overview for now and get into details at later point in time.

different file formats.
* Supported file formats
* Text File Format – csv and text
* parquet
* orc
* json
* avro (require plugin)
* All the file formats except text files typically store metadata along
with data. Hence when we create Data Frame out of the special
file formats, they typically inherit the schema.
* Let us create Data Frame out of JSON data and process it using
both Data Frame Operations as well as Spark SQL.

Connect with me or follow me at