Convert a CSV to AVRO or Parquet using Hive & Interview Questions related to Big Data File Format

Prerequisite:
---------------------
Row based & Column based formats | Demystifying RC Format in Big Data

Hive Queries:
----------------------
CREATE DATABASE IF NOT EXISTS firstdemo;

`Id` int,
`SEPAL_LENGTH` double,
`SEPAL_WIDTH` double,
`PETAL_LENGTH` double,
`PETAL_WIDTH` double,
`CLASS_NAME` string
)
WITH SERDEPROPERTIES (
) LOCATION 's3://ingestionhivetesting/landing/'

`Id` int,
`SEPAL_LENGTH` double,
`SEPAL_WIDTH` double,
`PETAL_LENGTH` double,
`PETAL_WIDTH` double,
`CLASS_NAME` string
)
STORED AS AVRO
LOCATION 's3://ingestionhivetesting/avronative/';

`Id` int,
`SEPAL_LENGTH` double,
`SEPAL_WIDTH` double,
`PETAL_LENGTH` double,
`PETAL_WIDTH` double,
`CLASS_NAME` string
)
STORED AS PARQUET
LOCATION 's3://ingestionhivetesting/parquetnative/';

Check this playlist for more AWS Projects in Big Data domain:

[ravikreddy]

Can you explain more about ROW FORMAT SERDE? what exactly it does?

[dhiatarchoun]

hello, plz i develop a REST API in Talend for ESB, i have difficulty calling it in IDE