Complete Snowflake Tutorial & Hands on Guide- Zero To Hero 
Data Engineering Simplified 
↗ [Version:2023-06-03]
Snowflake Introduction & History
Episode-01 is a 20-min long video where you will learn about Snowflake history and why it has come into existence. It explores the evolution of the Cloud Data Platform and how Snowflake fits into modern data architecture. The video highlights the value of a Cloud Data Platform and emphasizes the concept of having one platform, one copy of data, and multiple workloads.
● Watch: Snowflake History & Co-founders
● Watch: Evolution of Cloud Data Platform
● Watch:Modern Data Architecture & Snowflake
● Watch:The Value of Cloud Data Platform
● Watch:One platform, one copy of data & many workload
Snowflake Free Trial Edition
Episode-02 is a 20+ min long video and covers the Snowflake free trial registration process in detail. When we work for an organization, we get a predefined setup and our exposure is very limited to explore this tool and that's why it becomes so important for us to do a self registration process and understand how it really works and what all matters when selecting the underlying cloud providers, regions, edition and pricing policies ● Watch: Snowflake Feature Matrix
● Watch: Snowflake Pricing
● Watch: 05:36 Snowflake Free Registration
● Watch: Welcome Snowflake Worksheet, Tabs & Roles ● Watch: MFA Multi Factor Authentication
Snowflake 3 Layered Architecture
Episode-03 is a 30 + min long video covering the Snowflake architecture & design approach in detail. In general, the snowflake is known as a cloud-native platform that supports storage & computes decoupling but how it achieve this is now known to us and this video chapter helps you understand this concept in detail..
● Watch: Traditional Data Warehouse Architecture
● Watch: Snowflake 3 Layer Unique Architecture
● Watch: Decode Snowflake Storage Layer
● Watch: Decode Snowflake Compute Layer
● Watch: Decode Snowflake Cloud Service Layer
● Watch: Lifecycle of a query in Snowflake World
● Watch: Developer's day - Snowflake At a Glance
Snowflake Legacy WebUI
Episode-04 This tutorial on the Snowflake Classic Legacy WebUI covers every feature of the Snowflake Web Interface. The tutorial aims to help data developers and analysts use the tool effectively by
● Watch: General Overview & Tabs
● Watch: Top left/right menus
● Watch: Navigation Tree
● Watch: Query Pan
● Watch: Query History
● Watch: Account Tab
Snowsight - Modern WebUI
Episode-05 This video tutorial covers every function and feature of the Snowsight, including worksheet, folder, filter, sharing, dashboard, visualization, and many more features, described in detail with a hands-on guided tour. Snowsight's dashboard is compared with Tableau dashboard and PowerBI dashboard due to its powerful features and supported sharing capabilities within the organization. The Snowsight preview app brings productivity to the developer community and makes overall navigation smooth and sleek.
● Watch: Why and Why Not Snowsight
● Watch: Snowsight quick feature review
● Watch: Live Demo - How to access
● Watch: Live Demo - Feather Map (All Menus)
● Watch: Live Demo - Profile
● Watch: Live Demo - Worksheet
● Watch: Live Demo - Worksheet Formatting
● Watch: Live Demo - Worksheet Filter
● Watch: Live Demo - Dashboard
● Watch: Live Demo - Dashboard Filter
● Watch: Live Demo - Data
● Watch: Live Demo - Compute
● Watch: Live Demo - Accounts
● Watch: Live Demo - Classic Console & Switch Account
Snowflake Unique Features
Episode-06 Covers building ETL Workflow in snowflake is not an easy task and if you have to build an end to end ETL workflow (or ETL workflow) we need to use pipe, stream and task components along with external stages and orchestrate all the activities. This episode is a comprehensive 60 min practical guide with handson exercise to demonstrate you how it works.
● Watch: What so special about snowflake
● Watch: Traditional ETL/ELT Challenges
● Watch: Feature 1 - Cloud Provide Support
● Watch: Feature 2 - Unlimited Storage & Compute ● Watch: Feature 3 - Data Platform as Service
● Watch: Feature 4 - Unique 3 Layer Architecture
● Watch: Feature 5 - Virtue Warehouse (compute)
● Watch: Feature 6 - Support for semi structure data ● Watch: Feature 7 - Snowflake Time Travel
● Watch: Feature 8 - Snowflake Zero Copy Clone
● Watch: Feature 17 - Snowflake Partner Connect
● Watch: Feature 18 - Snowflake Data Marketplace
Container Hierarchy Concept
Episode-07 This chapter focuses on the hierarchy concept of databases, schemas, tables, and containers. It provides hands-on exercises for creating tables with different types, constraints, and data loading methods, including standard and external tables and
insert statements, create as select, insert as select, and copy commands. The tutorial also covers topics such as not null constraints, case sensitivity with table names, describing tables, primary and unique key constraints, and querying external stages in Snowflake. The video includes a sub-topic tree map and concludes with a quick recap of the main points and a thank-you note. ● Watch: Snowflake Container Hierarchy
● Watch: Container - Database - Schema - Table
● Watch: Database, Schema, Table Creation & Describe ● Watch: Snowflake Table DDL & Data Types
● Watch: Desc Table & get_ddl()
● Watch: Fully Qualified name for table
● Watch: Snowflake Table DDL & string/varchar datatype ● Watch: Load data via SnowSQL & Put Command
New/Different Objects In Snowflake
Episode-08 This chapter focuses on the new objects introduced by Snowflake that data developers and leads should know. The video explains how knowing the purpose and power of these objects can save time and optimize cost and solution. The video covers live demos of Sequence Objects, File Format Objects, Stage Objects, Integration Objects, Pipe Objects, Stream Objects, Task Objects, and Resource monitor.
● Watch: What so special about snowflake new objects ● Watch: All About Sequence Objects
● Watch: All About File Format Objects
● Watch: All About Stage Objects
● Watch: All About Integration Objects
● Watch: All About Pipe Objects
● Watch: All About Stream Objects
● Watch: All About Task Objects
● Watch: All About Resource monitor
Fast, Bulk & Batch Data Ingestion
Episode-09 This tutorial helps data developers understand bulk data loading and data ingestion concepts in Snowflake, with different stages and file formats to support various use cases. The video covers user stage, table stages, internal stages, query stages, file formats,
and the copy command. It explains how to perform file validation, optimize compression and warehouse size, and copy semi-structured
● Watch: List stage objects before copying
● Watch: Loading Data from stage to table
● Watch: Copy & Copy History
● Watch: Role of file format in copy command
● Watch: Query stage data without loading
● Watch: Stage file validation during copy command ● Watch: Role of VWH size & compression during data loading
Continuous Data Loading
Episode-10 This guide covers continuous data loading and data ingestion in Snowflake. The tutorial provides a hands-on approach to help data developers ingest streaming and micro-batch data using Snowpipe, a SQL object designed to support streaming data, delta data, and CDC data into Snowflake. The tutorial covers Snowpipe's architecture, components, internal stages, and Python program, as well as monitoring and pausing/resuming pipes. The video guide includes live examples to explain Snowpipe's most important features. This tutorial is a comprehensive guide to continuous data loading and data ingestion in Snowflake.
● Watch: What so special about pipe
● Watch: Snowpipe Auto Ingest Process
● Watch: Introduction - Snowpipe Hands-on
● Watch: SQL Constructs (Snowpipe Hands-on)
● Watch: Python Code for Auto Ingest (Snowpipe Hands-on) ● Watch: Key Pair Association (Snowpipe Hands-on) ● Watch: Monitoring Pipe (Snowpipe Hands-on)
● Watch: View Pipe in Snowsight (Snowpipe Hands-on) ● Watch: Pause & Resume Pipe (Snowpipe Hands-on)
External Table Snowflake
Episode-11 This tutorial covers how to work with external tables in Snowflake . The video guide provides a visual and hands-on approach to demonstrate how to connect to a cloud data lake and view different data sets such as CSV, Parquet, ORC, Avro, JSON, and XML. The tutorial answers questions such as what external tables are in Snowflake, how to create them, and how to map them with external stages like AWS S3. It also covers auto-ingestion via external tables, working with Parquet files, and more. This tutorial offers a comprehensive guide to working with external tables in Snowflake.
● Watch: Why External Table
● Watch: Create External Table
● Watch: External Table & External Stage Mapping
● Watch: External Table & Partition
● Watch: External Table & Auto Ingest
● Watch: External Table Functions & Table Functions ● Watch: External Table & Semistructured Data (Parquet & ORC) ● Watch: External Table & SnowPipe
providing a visual guide to the structure and layout of the web UI. The video covers every tab along with its relationship with user roles and when it works and doesn't work. The tutorial covers topics such as navigation tree, query history, and account tab. The video also includes a quiz time section and ends with a thank you note and disclaimer.
● Watch: Legacy WebUI Introduction
● Watch: Feature 9 - Continuous Data Loading ● Watch: Feature 10 - Support for ANSI SQL + Extended SQL ● Watch: Feature 11 - Snowflake Micropartition/ Clustering ● Watch: Feature 12 - Snowflake Data Security & Encryption ● Watch: Feature 13 - Snowflake RBACC & DAC ● Watch: Feature 14 - Data Sharing & Reader's Account ● Watch: Feature 15 - Data Replication & Failover ● Watch: Feature 16 - Snowflake Connectors & Drivers
data. The tutorial provides a hands-on visual guide with live examples and covers essential features.
● Watch: What so special about snowflake new objects ● Watch: Why to subscribe this channel
● Watch: Stage Concepts and type of stages
● Watch: User Stage & list them
● Watch: Table Stage & list them
Snowflake End To End ETL Flow
Episode-12 This chapter offers a comprehensive guide on working with Snowflake virtual warehouse beyond SQL syntax. It covers topics such as standard and multi-cluster virtual warehouses, enabling auto-resume and auto-suspend features, scaling policies, setting up a resource monitor, and validating virtual
Complete Snowflake Tutorial & Hands on Guide- Zero To Hero Data Engineering Simplified ↗
warehouse workload. The video also explains the challenges of fixed capacity compute models and Snowflake's compute sizing and pricing model. Additionally, it discusses the Virtual Warehouse SQL construct and how it works with SnowSight.
● Watch: Snowflake virtual warehouse & beyond
● Watch: Challenges with fixed capacity compute model ● Watch: Snowflake Compute Sizing and Pricing Model ● Watch: Virtual Warehouse SQL Construct
● Watch: Multi Cluster Auto Scaling
● Watch: How to calculate virtual warehouse workload ● Watch: Virtual Warehouse Resource Monitor
● Watch: Virtual Warehouse & SnowSight
Snowflake Micro Partition Concept
Episode-13 This tutorial covers the concept of micro partitions used by Snowflake to make its cloud data warehouse fast and cost-effective. It covers various aspects of micro partitions, including how they work, their appearance, and the number of micro partitions created by Snowflake. It also explains micro partition clustering depth and overlap. SQL commands for micro partition and cluster key are also provided. Finally, the tutorial discusses the relationship between micro partitions, query profiles, clustering depth, and cluster overlap.
● Watch: So secretive about Micro Partition
● Watch: List, Range, Hash Partitions Vs Micro Partition ● Watch: How Does the Snowflake Micro Partition Work? ● Watch: What does a micro partition look like?
● Watch: How many micro partitions are created? ● Watch: Clustering depth and clustering overlap? ● Watch: Micro partition summary and benefit
● Watch: SQL for Micro partition & cluster key
● Watch: Micro Partition & Query Profile
● Watch: Micro Partition - Clustering Depth/Overlap
Time Travel Feature In Snowflake
Episode-14 Snowflake's Time Travel Feature is a powerful utility that can save time and effort in managing complex data projects. Time travel offers more than just the ability to view past data, it opens up new possibilities for designing and operating data projects. The 30-minute episode introduces time travel and answers questions such as what time travel is and how it works conceptually, whether all editions of Snowflake support time travel, whether time travel and fail-safe are the same, how time travel works for transient and temporary tables, whether time travel is expensive, whether time travel and data retention parameters are the same, what is extended SQL for time travel, and whether time travel is applicable only for tables or for databases and schemas as well.
● Watch: What problem does time-travel solve?
● Watch: What is time travel?
● Watch: Difference Between Time Travel & Fails Safe? ● Watch: Time Travel for Transient/Temporary Tables? ● Watch: How to enable time travel?
● Watch: Drop & Undrop Table, Database, Schema SQLs? ● Watch: Clone (Create Table) with Time Travel SQL. ● Watch: Create Table As Select Using CTAS with Time Travel
● Watch: Table Storage Cost for Time Travel Feature
Snowflake Zero Copy Clone
Episode-15 Snowflake's Clone Feature is a useful and versatile feature that can greatly benefit development teams who rely on data operation teams. Cloning can be applied not only to tables, but also to schemas, databases, and other objects within schemas. The 34-minute episode introduces cloning (also known as zero copy clone) and answers questions such as whether clone and zero copy clone are the same, whether cloning costs anything and if so, how much and when, how to clone an object, what objects are not cloneable, what happens if a pipe is cloned, and whether internal stages or external tables can be cloned.
● Watch: What you need to know about clones?
● Watch: What is cloning in snowflakes?
● Watch: Schema level cloneable objects.
● Watch: List of direct cloneable objects?
● Watch: Clone Feature Hands-on
● Watch: Table cloning & Understand Zero Copy Clone ● Watch: How Transient & Temporary Table Cloning Works ● Watch: Object Dependency with Cloning
● Watch: How External Table Cloning Works
● Watch: How File Format & Sequences Cloning Works ● Watch: How Pipe Objects Cloning Works
● Watch: How Stream Objects Cloning Works
● Watch: How Task Objects Cloning Works
● Watch: How A large database Cloning Works
Snowflake Data Sharing
Episode-16 Snowflake Data Sharing is a game-changing feature that enables secure data sharing and supports many previously impossible use cases. The 25-minute episode provides an introduction to the feature and helps answer questions such as whether a Snowflake account is necessary for consumption, whether account sharing is possible in the same cloud and region, who pays for data and compute for direct share and reader accounts, whether only permanent tables can be shared, and whether changes made by providers are immediately reflected at the consumer side.
● Watch: What you don't know about data sharing? ● Watch: Cloneable vs Data Sharing
● Watch: Account to account direct sharing
● Watch: Cross cloud & cross region data sharing
● Watch: Data sharing using snowflake reader account ● Watch: Data sharing hands-on section
● Watch: Scenario-1 Share only database
● Watch: Scenario-2 Share table and views
● Watch: Scenario-3 Share via reader's account
● Watch: Scenario-4 Cross Cloud & Cross Region
Change Data Capture - Streams
Episode-17 Snowflake Stream & Change Data Capture is a powerful feature that simplifies CDC architecture. The 27-minute episode introduces the feature and elaborates on stream objects,
answering questions such as how Snowflake Stream works, how it tracks changes, what offsets are, and whether it is expensive. The episode also addresses how long Stream holds the CDC dataset.
● Watch: What you don't know about snowflake streams? ● Watch: What is stream and how does it work?
● Watch: Type of stream in snowflake
● Watch: Can I create multiple stream on a source table ● Watch: Can I clone a stream
● Watch: What is offset and how it works in stream? ● Watch: Snowflake Stream Hands on.
● Watch: Stream on standard table
● Watch: Append Only Stream
● Watch: Stream on transient table
● Watch: Data Retention Period for Time Travel & Stream. ● Watch: Scenario-5 How to consume stream data ● Watch: Scenario-6 Stream on external table
Snowflake Task & Task Tree
Episode-18 Snowflake Tasks and Task Tree are two essential components in Snowflake that allow you to automate your SQL scripts and workflows. By utilizing these features, you can achieve significant automation and efficiently move data from the ingestion to consumption layer. These components are native to Snowflake and function effectively when designed correctly.
● Watch: What you don't know about snowflake tasks? ● Watch: What is a snowflake task and how does it work? ● Watch: Why to subscribe this channel
● Watch: Task DDL and SQL Construct
● Watch: Task Scheduling (Non-Cron)
● Watch: Task Scheduling Using Cron Keyword
● Watch: Serverless Task
● Watch: Snowflake Task Tree (Workflow & Orchestration) ● Watch: Snowflake Task & Stored Procedure
Snowflake End To End ETL Flow
Episode-19 Covers building ETL Workflow in snowflake is not an easy task and if you have to build an end to end ETL workflow (or ETL workflow) we need to use pipe, stream and task components along with external stages and orchestrate all the activities. This episode is a comprehensive 60 min practical guide with handson exercise to demonstrate you how it works. ● Watch: Customer Order Use Case for Snowflake
● Watch: How many objects are needed for this ETL ● Watch: Practice it together
● Watch: Part-1: Creating Landing Layer DDLs
● Watch: Part-2: Creating Curated Layer DDLs
● Watch: Part-3: Creating Consumption Layer DDLs ● Watch: Part-4: Stages & Pipes for Landing Layers ● Watch: Part-5: Stream & Tasks for Landing Layers ● Watch: Part-6: Stream & Tasks for Curated Layers ● Watch: Part-7: Data Loading via WebUI (Delta Load)
● Watch: Delta Load via S3 Bucket
Snowflake End To End ETL Flow Episode-19 Covers building ETL Workflow in snowflake is
not an easy task and if you have to build end to end ETL workflow (or ETL workflow) we need to use pipe, stream and task components along with external stages and orchestrate all the activities. This episode is a comprehensive 60 min practical guide with handson exercise to demonstrate you how it works. Work With Snowflake ODBC Driver
Episode-24.2 The Snowflake ODBC driver is a popular tool for BI tools like PowerBI or Tableau, but using it to bring data from Snowflake to MS Excel and apply transformations is not as widely known. This guide is designed to help analysts who are skilled at analyzing data in Excel but not familiar with programming or BI tools, to fetch data into their worksheets. The guide covers installation and usage of the Snowflake ODBC driver in MS Excel, along with answers to common questions, such as setup, configuration parameters, fetching data, and transforming data with Power Query. It also provides information on configuring the driver in Linux or Mac OS, programming language compatibility, downloading the driver, sample connection strings, and the latest version.
Snowflake Python Connector
Episode-24.3 This tutorial covers the installation, configuration, and authentication approaches of the Snowflake Python connector. It also offers hands-on experience with running DDL and DML operations, getting data from Snowflake into Pandas data frames, managing asynchronous queries, and loading data to stage location via the Snowflake Python API.
● Watch: Python Version Requirement for Snowflake Connector ● Watch:Install Snowflake Python Connector on Windows ● Watch:Snowflake Python Connector Example
● Watch: Python Connector Authentication Key Pair & MFA ● Watch:Snowflake Python Connector DDL & DML Example ● Watch:Snowflake Python Connector Execute Many Example ● Watch: Python Connector Asynchronous Query Example ● Watch: Python Connector Pandas Dataframe Example ● Watch:Snowflake Python Connector Data Loading Example
PowerBI & Snowflake Integration Episode-25.1 To Be Published Shortly.
Tableau & Snowflake Integration Episode-25.2 To Be Published Shortly.
No comments:
Post a Comment