Read Parquet Data from External Stage & write in Permanent Snowflake Table using Snowflake Snowpark

This video explains how to read data from Parquet files in Snowpark Python in-depth.

Prerequisite:
---------------------------
Automatic Schema Detection in Snowflake

Documentation:
-------------------------------

Dataset:
-----------------

Snowflake SQL Queries:
-------------------------------------------
drop database if exists ramu;
--Create Database
create database if not exists ramu;
--use the database
use ramu;

--stage creation
create or replace stage ramu.PUBLIC.snow_simple url="s3://{}/{}/"
credentials=(aws_key_id=''
aws_secret_key='');
--check the data present in S3
list @ramu.PUBLIC.snow_simple;
--Execute the query
select * from parquet_overwrite;

create or replace table video1 (id int,sepal_length number(10,4),sepal_width number(10,4),petal_length number(10,4) , petal_width number(10,4),class_name varchar(200));

select * from video1;

create or replace table video2 (id int,sepal_length number(10,4),sepal_width number(10,4),petal_length number(10,4) ,dummy_column number(10,4), petal_width number(10,4),class_name varchar(200));

select * from video2;

select count(*) from video2;

Code:
-------------------
#setup the connection
connection_parameters = {"account":"",
"user":"",
"password": "",
"role":"ACCOUNTADMIN",
"warehouse":"COMPUTE_WH",
"database":"RAMU",
"schema":"PUBLIC"
}
withColumnRenamed('"sepal_length"','sepal_length') \
.withColumnRenamed('"sepal_width"','sepal_width')\
.withColumnRenamed('"petal_length"','petal_length')\
.withColumnRenamed('"petal_width"','petal_width')\
.withColumnRenamed('"class_name"','class_name')

Check this playlist for more Data Engineering related videos:

Apache Kafka form scratch

Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation--

🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY YOUTUBE CHANNEL