Category: Snowflake

Overview of Time Travel feature in Snowflake

Time Travel in Snowflake

Snowflake Time Travel enables us to perform the following tasks:● Accessing data that has been changed or deleted at any point within a retention period.● Restoring tables, schemas, and databases that have been dropped.● Create Clones of tables, schemas and databases at or before specific points in the past.These tasks can be performed until the retention period, after which the data is moved into Snowflake Fail-safe. Time Travel SQL Extensions:In Snowflake Time Travel to access or clone historical data we use the Time Travel SQL Extensions AT or BEFORE clause which can be specified in SELECT statements and CREATE … CLONE commands. To pinpoint the exact historical data which you wish to access we use the below parameters with the SQL Extensions:TIMESTAMPOFFSET (time difference in seconds from the present time)STATEMENT (identifier for statement, e.g. query ID) Restoring Objects:A dropped table or schema or database that has not yet been purged from the system (i.e. not yet moved into Snowflake Fail-safe) can be restored using the UNDROP command. Calling UNDROP restores the object to its most recent state before the object was dropped. Enabling and Disabling Time Travel:Time travel is automatically enabled in Snowflake with 1-day data retention period by default […]

Read More +

Snowflake Insights Webcast: Leave Your Legacy Data Problems Behind

Snowflake Insights Webcast

Snowflake Insights Webcast: Leave Your Legacy Data Problems Behind No images? Click here Leave Your Legacy Data Problems Behind With Snowflake’s Modern Cloud Data Platform Webcast: Wednesday, Aug 12, 10 AM PT Reserve my Seat   Presenter: Eric Pouliot     Snowflake delivers a serverless, modern cloud data platform, optimized for data engineering, data lakes, data warehouses, data applications, data science, and data exchange. Built from the ground up in the cloud, Snowflake provides limitless possibilities for businesses of all sizes. This Precisio presentation will highlight High Level Overview Key Benefits Basic Pricing Getting Started Together, DataTerrain and Precisio provide enterprise-class, cloud-ready data engineering, business intelligence, analytics, and data science solutions.  Reserve my Seat DataTerrain Inc 228 Hamilton Ave, Palo Alto, CA 94301, USA Preferences  |  Unsubscribe  

Read More +

HOW TO SHARE DATA IN SNOWFLAKE

Azure Snowflake

Snowflake enables sharing of data through named Snowflake objects called shares which supports data sharing by sharing tables, secure views, and secure UDFs in our Snowflake database (Data Provider) with other Snowflake accounts (Data Consumer). If the data consumer does not have a Snowflake account then we can create a reader account through which we can share data without requiring the consumer to become a Snowflake customer. All database objects shared between accounts are read-only so data consumers cannot make any change to the data. In secure data sharing, data is not actually copied or transferred between accounts; it means that shared data does not take up any storage in a consumer account. The only charges to consumers are for the compute resources used to query the shared data. In the case of a reader account, the provider will be charged for the compute resources used by users in the reader account. Data shares and reader accounts must be located in the same region of the same Cloud Provider (AWS, Azure or GCP). Data consumers cannot re-share the shared data with other accounts. In reader accounts data consumers are allowed to create new users. To start sharing data we need […]

Read More +

Overview of Continuous and Automated Data loading in Snowflake using Snowpipe

Snowpipe is Snowflake's continuous data ingestion service

Snowpipe is Snowflake’s continuous data ingestion service. Snowpipe loads data from files as soon as they are available in a staging area based on the COPY statement defined in a referenced pipe. The COPY statement identifies the source location (e.g. AWS S3 Bucket) of the data files and a target table to be used by Snowpipe to load data into Snowflake table. Below mechanisms are available for detecting and loading the staged files: Automating Continuous Data Loading Using Cloud Messaging Calling Snowpipe REST Endpoints to Load Data Automating Continuous Data Loading Using Cloud Messaging – Automated data loads uses event notifications (e.g. AWS SQS) for cloud storage to inform Snowpipe of the arrival of new data files to load. Snowpipe then copies the files into a queue, from which they are loaded into the target table in a continuous, serverless fashion based on parameters defined in the pipe.Calling Snowpipe REST Endpoints to Load Data – Your client application calls a public REST endpoint with a list of data filenames and a referenced pipe name. If new data files matching the list are discovered in the stage, they are queued for loading. Snowflake-provided compute resources load data from the queue into […]

Read More +

HOW TO CHANGE TRACKING USING TABLE STREAMS IN SNOWFLAKE

Streams in Snowflake

A stream is a Snowflake object that provides change data capture (CDC) capabilities to track the changes in a table. It records changes made to a table, including information about inserts, updates and deletes as well as metadata about each change. This process is referred to as change data capture (CDC). Types Of Streams: Standard table stream tracks all DML changes to the source table, including inserts, updates, and deletes (including table truncates). Append-only table stream tracks row inserts only. Update and Delete operations (including table truncates) are not recorded. An append-only stream returns the appended rows only and therefore can be a better choice than a standard stream for extract, load, transform (ELT) and similar scenarios that depend exclusively on row inserts. Append-only table streams can be combined with Snowflake tasks for continuous ELT workflows to process recently changed table rows. STALE Streams: A stream becomes stale if its point of offset gets positioned at a point earlier than the table’s data retention period. In order to avoid staleness we should ensure that data from the stream is consumed regularly. Consuming stream data advances the stream offset point as a result of which it avoids stream becoming stale. ENABLE […]

Read More +

An overview Snowflake Analytic Datawarehouse

Snowflade as a Saas

Snowflake is an analytic data warehouse provided as Software-as-a-Service (SaaS). It is not a packaged software offering that can be installed by a user.Snowflake manages all aspects of software installation and updates.Snowflake runs completely on cloud infrastructure. It cannot be run on private cloud infrastructures (on-premises or hosted).All components of Snowflake’s service (other than an optional command line client), run in a public cloud infrastructure. It uses virtual compute instances for its compute needs and a storage service for persistent storage of data. Snowflake provides a data warehouse that is faster, easier to use, and far more flexible than traditional data warehouse offerings.Snowflake’s data warehouse is not built on an existing database or “big data” software platform such as Hadoop. It uses a new SQL database engine with a unique architecture designed for the cloud. To the user, Snowflake has many similarities to other enterprise data warehouses, but also has additional functionality and unique capabilities. Snowflake’s data warehouse is a true SaaS offering. More specifically: ● There is no hardware (virtual or physical) for you to select, install, configure, or manage. ● There is no software for you to install, configure, or manage. ● On-going maintenance, management, and tuning is […]

Read More +