Snowflake Sink Connector

The Snowflake Sink Connector allows you to continuously store the data from your Kafka Topics to Snowflake.

In this guide, we will walk you through creating a Snowflake Sink Connector.

Get Started

Create a Kafka Cluster

If you do not have a Kafka cluster and/or topic already, follow these steps to create one.

Prepare the Snowflake Environment

From the snowflake console, the following configurations need to be obtained:

1. snowflake.url.name
2. snowflake.user.name
3. snowflake.private.key
4. snowflake.database.name
5. snowflake.schema.name

If you already have these and configured the required roles and keys for the database and the user, you can skip to the Create The Connector section.

For more detailed configurations see the snowflake connector documentation

snowflake.url.name

snowflake.url.name can be found at the home page of the snowflake app. Click on the account identifier and navigate to copy account URL as shown below.

A URL similar to https://mn93536.eu-central-1.snowflakecomputing.com will be copied. We need to append port 443 while passing it to the connector. At the end snowflake.url.name will look like the following.

https://mn93536.eu-central-1.snowflakecomputing.com:443

snowflake.user.name

snowflake.user.name can be seen on the profile view. To open the profile view, go to the top left and click on the profile as shown below.

snowflake.private.key

snowflake.private.key will be generated by you locally. A pair of private and public keys need to be generated. public.key will be set to the user on the snowflake and the private key will be set to the connector as snowflake.private.key.

See the following document to learn how to generate the keys and set the public key to snowflake.

snowflake.database.name & snowflake.schema.name

From the snowflake app, create a database and a schema. To be able to use this schema and connector we need to create and assign a custom role to the database and the schema. You can follow this document to see how to do it.

Make sure that the script described in the document above is running on the desired database and schema by selecting them at the top of the script as follows:

Now, everything should be ready on the snowflake side. We can move on the creating the connector.

Create the Connector

Go to the Connectors tab, and create your first connector by clicking the New Connector button.

Choose your connector as Snowflake Connector

Enter the required properties.

The advanced screen is for any other configuration that the selected connector supports. At the top of this screen, you can find a link to related documentation. We can proceed with what we have and click the Connect button directly.

Congratulations! You have created a Snowflake Sink Connector.

As you put data into your selected topics, the data will be written into Snowflake. You should see the data in the snowflake app as follows: