• Stars
    star
    253
  • Rank 160,776 (Top 4 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created over 3 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Snowflake Snowpark Python API

Snowflake Snowpark Python API

Build and Test codecov PyPi License Apache-2.0 Codestyle Black

The Snowpark library provides intuitive APIs for querying and processing data in a data pipeline. Using this library, you can build applications that process data in Snowflake without having to move data to the system where your application code runs.

Source code | Developer guide | API reference | Product documentation | Samples

Getting started

Have your Snowflake account ready

If you don't have a Snowflake account yet, you can sign up for a 30-day free trial account.

Create a Python virtual environment

You can use miniconda, anaconda, or virtualenv to create a Python 3.8, 3.9 or 3.10 virtual environment.

To have the best experience when using it with UDFs, creating a local conda environment with the Snowflake channel is recommended.

Install the library to the Python virtual environment

pip install snowflake-snowpark-python

Optionally, you need to install pandas in the same environment if you want to use pandas-related features:

pip install "snowflake-snowpark-python[pandas]"

Create a session and use the APIs

from snowflake.snowpark import Session

connection_parameters = {
  "account": "<your snowflake account>",
  "user": "<your snowflake user>",
  "password": "<your snowflake password>",
  "role": "<snowflake user role>",
  "warehouse": "<snowflake warehouse>",
  "database": "<snowflake database>",
  "schema": "<snowflake schema>"
}

session = Session.builder.configs(connection_parameters).create()
df = session.create_dataframe([[1, 2], [3, 4]], schema=["a", "b"])
df = df.filter(df.a > 1)
df.show()
pandas_df = df.to_pandas()  # this requires pandas installed in the Python environment
result = df.collect()

Samples

The Developer Guide and API references have basic sample code. Snowflake-Labs has more curated demos.

Logging

Configure logging level for snowflake.snowpark for Snowpark Python API logs. Snowpark uses the Snowflake Python Connector. So you may also want to configure the logging level for snowflake.connector when the error is in the Python Connector. For instance,

import logging
for logger_name in ('snowflake.snowpark', 'snowflake.connector'):
    logger = logging.getLogger(logger_name)
    logger.setLevel(logging.DEBUG)
    ch = logging.StreamHandler()
    ch.setLevel(logging.DEBUG)
    ch.setFormatter(logging.Formatter('%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))
    logger.addHandler(ch)

Contributing

Please refer to CONTRIBUTING.md.

More Repositories

1

snowflake-connector-python

Snowflake Connector for Python
Python
574
star
2

gosnowflake

Go Snowflake Driver
Go
291
star
3

snowflake-sqlalchemy

Snowflake SQLAlchemy
Python
231
star
4

spark-snowflake

Snowflake Data Source for Apache Spark.
Scala
211
star
5

SnowAlert

Security Analytics Using The Snowflake Data Warehouse
Python
181
star
6

snowflake-connector-net

Snowflake Connector for .NET
C#
175
star
7

snowflake-jdbc

Snowflake JDBC Driver
Java
172
star
8

snowflake-cli

Snowflake CLI is an open-source command-line tool explicitly designed for developer-centric workloads in addition to SQL operations.
Python
168
star
9

snowflake-kafka-connector

Snowflake Kafka Connector (Sink Connector)
Java
135
star
10

snowflake-connector-nodejs

NodeJS driver
JavaScript
119
star
11

polaris-catalog

Polaris Catalog is an open source catalog for Apache Iceberg
90
star
12

snowflake-ingest-java

Java SDK for the Snowflake Ingest Service -
Java
66
star
13

dplyr-snowflakedb

SnowflakeDB backend for dplyr
R
65
star
14

snowflake-ingest-python

A Python API for Asynchronously Loading Data into Snowflake DB -
Python
59
star
15

pdo_snowflake

PHP PDO driver for snowflake
C
58
star
16

snowflake-ml-python

Python
38
star
17

libsnowflakeclient

Snowflake Connector for C/C++
C
25
star
18

connectors-native-sdk

Snowflake Native SDK for Connectors
Java
25
star
19

native-apps-examples

Public-facing example applications built using the Snowflake Native App Framework
Python
22
star
20

snowpark-java-scala

Snowflake Snowpark Java & Scala API
Scala
18
star
21

snowflake-hive-metastore-connector

Java
13
star
22

snowflake-s3compat-api-test-suite

S3Compat API Test Suite
Java
12
star
23

snowflake-telemetry-python

Python
7
star
24

snowflake-rest-api-specs

Public rest api specs for Snowflake
4
star
25

snowflake-common

3
star
26

native-apps-templates

Official Snowflake CLI templates released for the Snowflake Native App Framework
Python
3
star
27

homebrew-snowflake-cli

Ruby
1
star
28

reusable-workflows

A reusable workflow that will be used by repositories to run semgrep on each PR
1
star
29

snowflake-hibernate

Snowflake SQL dialect definition for Hibernate ORM
Java
1
star