• Stars
    star
    516
  • Rank 82,712 (Top 2 %)
  • Language
    Python
  • License
    Other
  • Created over 4 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Google Drive API Python wrapper library. Maintained fork of PyDrive.

GHA Tests Conda-forge PyPI

PyDrive2

PyDrive2 is a wrapper library of google-api-python-client that simplifies many common Google Drive API V2 tasks. It is an actively maintained fork of https://pypi.python.org/pypi/PyDrive. By the authors and maintainers of the Git for Data - DVC project.

Project Info

Features of PyDrive2

  • Simplifies OAuth2.0 into just few lines with flexible settings.
  • Wraps Google Drive API V2 into classes of each resource to make your program more object-oriented.
  • Helps common operations else than API calls, such as content fetching and pagination control.
  • Provides fsspec filesystem implementation.

How to install

You can install PyDrive2 with regular pip command.

$ pip install PyDrive2

To install the current development version from GitHub, use:

$  pip install git+https://github.com/iterative/PyDrive2.git#egg=PyDrive2

OAuth made easy

Download client_secrets.json from Google API Console and OAuth2.0 is done in two lines. You can customize behavior of OAuth2 in one settings file settings.yaml.

from pydrive2.auth import GoogleAuth
from pydrive2.drive import GoogleDrive

gauth = GoogleAuth()
gauth.LocalWebserverAuth()

drive = GoogleDrive(gauth)

File management made easy

Upload/update the file with one method. PyDrive2 will do it in the most efficient way.

file1 = drive.CreateFile({'title': 'Hello.txt'})
file1.SetContentString('Hello')
file1.Upload() # Files.insert()

file1['title'] = 'HelloWorld.txt'  # Change title of the file
file1.Upload() # Files.patch()

content = file1.GetContentString()  # 'Hello'
file1.SetContentString(content+' World!')  # 'Hello World!'
file1.Upload() # Files.update()

file2 = drive.CreateFile()
file2.SetContentFile('hello.png')
file2.Upload()
print('Created file %s with mimeType %s' % (file2['title'],
file2['mimeType']))
# Created file hello.png with mimeType image/png

file3 = drive.CreateFile({'id': file2['id']})
print('Downloading file %s from Google Drive' % file3['title']) # 'hello.png'
file3.GetContentFile('world.png')  # Save Drive file as a local file

# or download Google Docs files in an export format provided.
# downloading a docs document as an html file:
docsfile.GetContentFile('test.html', mimetype='text/html')

File listing pagination made easy

PyDrive2 handles file listing pagination for you.

# Auto-iterate through all files that matches this query
file_list = drive.ListFile({'q': "'root' in parents"}).GetList()
for file1 in file_list:
    print('title: {}, id: {}'.format(file1['title'], file1['id']))

# Paginate file lists by specifying number of max results
for file_list in drive.ListFile({'maxResults': 10}):
    print('Received {} files from Files.list()'.format(len(file_list))) # <= 10
    for file1 in file_list:
        print('title: {}, id: {}'.format(file1['title'], file1['id']))

Fsspec filesystem

PyDrive2 provides easy way to work with your files through fsspec compatible GDriveFileSystem.

from pydrive2.fs import GDriveFileSystem

fs = GDriveFileSystem("root", client_id=my_id, client_secret=my_secret)

for root, dnames, fnames in fs.walk(""):
    ...

Concurrent access made easy

All API functions made to be thread-safe.

Contributors

Thanks to all our contributors!

https://contrib.rocks/image?repo=iterative/PyDrive2

More Repositories

1

dvc

๐Ÿฆ‰ ML Experiments and Data Management with Git
Python
13,036
star
2

cml

โ™พ๏ธ CML - Continuous Machine Learning | CI/CD for ML
JavaScript
3,907
star
3

mlem

๐Ÿถ A tool to package, serve, and deploy any ML model on any platform. Archived to be resurrected one day๐Ÿคž
Python
713
star
4

shtab

โ†”๏ธ Automagic shell tab completion for Python CLI applications
Python
328
star
5

dvc.org

๐Ÿ“– DVC website and documentation
TypeScript
320
star
6

terraform-provider-iterative

โ˜๏ธ Terraform plugin for machine learning workloads: spot instance recovery & auto-termination | AWS, GCP, Azure, Kubernetes
Go
287
star
7

vscode-dvc

Machine learning experiment tracking and data versioning with DVC extension for VS Code
TypeScript
176
star
8

example-get-started

Get started DVC project
Python
167
star
9

dvclive

๐Ÿ“ˆ Log and track ML metrics, parameters, models with Git and/or DVC
Python
150
star
10

gto

๐Ÿท๏ธ Git Tag Ops. Turn your Git repository into Artifact Registry or Model Registry.
Python
134
star
11

awesome-iterative-projects

A list of projects relying on Iterative.AI tools to achieve awesomeness
63
star
12

dataset-registry

Dataset registry DVC project
60
star
13

magnetic-tiles-defect

Demo Computer Vision Project
Jupyter Notebook
55
star
14

course-ds-base

Jupyter Notebook
39
star
15

example-get-started-experiments

Get started DVC project
Python
39
star
16

demo-bank-customer-churn

Demo DVC project training a classification model on tabular data
Jupyter Notebook
38
star
17

aita_dataset

AITA dataset based on r/AmItheAsshole/
Python
32
star
18

setup-dvc

DVC GitHub action
JavaScript
29
star
19

example_cml

Python
28
star
20

workshop-uncool-mlops

Accompanies the uncool MLOps workshop
Python
26
star
21

ldb-resources

Python
26
star
22

mlem.ai

โœจ Landing page for MLEM
TypeScript
26
star
23

setup-cml

GitHub Action for CML setup
TypeScript
21
star
24

cml_base_case

Python
21
star
25

example-repos-dev

Source code and generator scripts for example DVC projects
Python
21
star
26

cml_cloud_case

Python
20
star
27

dvc-bench

Benchmarks for DVC
Python
20
star
28

scmrepo

SCM wrapper and fsspec filesystem for Git for use in DVC.
Python
19
star
29

cml_dvc_case

Python
18
star
30

intellij-dvc

DVC integration plugin for Intellij IDEs including PyCharm, IntelliJ IDEA and CLion
Java
17
star
31

dvc-data

DVC's data management subsystem
Python
17
star
32

studio-support

โ“ DVC Studio Issues, Question, and Discussions
15
star
33

studio-selfhosted

This repository contains auxiliary installation code for self-hosting Studio
Shell
15
star
34

VSCode-DVC-Workshop

Workshop about DVC VSCode Extension
Jupyter Notebook
14
star
35

example-dvc-experiments

DVC Get Started Project with a focus on `dvc experiment` features.
HTML
13
star
36

pytest-servers

Create temporary directories on the various filesystems for testing
Python
12
star
37

py-template

Hypermodern Python Cookiecutter
Python
12
star
38

cml.dev

๐Ÿ”— CML website and documentation
TypeScript
12
star
39

dvc-objects

dvc objects - contains filesystem and object-db level abstractions to use in dvc and dvc-data
Python
10
star
40

dvcyaml-schema

Schema for dvc.yaml
Python
10
star
41

example-versioning

Data sets and ML models versioning example from DVC get started
Python
9
star
42

dvc-streamlit-components

Streamlit components for DVC
Python
9
star
43

blog

๐Ÿ“– DVC blog engine
TypeScript
8
star
44

cml_tensorboard_case

Python
8
star
45

priority-list

โ›๏ธ Make a dent in GitHub issue & PR backlogs across repositories
Python
7
star
46

dvc-task

Celery task queue used in DVC
Python
7
star
47

tpi

Python wrapper for terraform-provider-iterative
Python
7
star
48

example-gto

Get Started GTO Project
7
star
49

example-pokemon-classifier

Example project with a CNN to train a Pokรฉmon type classifier.
Python
7
star
50

dvc-render

Library for rendering DVC plots
Python
6
star
51

example-mlem-get-started

Get Started MLEM project
Python
6
star
52

dvc-studio-client

Client to interact with DVC Studio
Python
6
star
53

workshop-uncool-mlops-solution

Python
6
star
54

pytest-test-utils

Python
6
star
55

features

A collection of development container 'features' for machine learning and data science
Shell
6
star
56

dvc-s3

AWS S3 plugin for dvc
Python
6
star
57

gatsby-theme-iterative

A Gatsby theme for shared logic between all the websites from iterative.ai
JavaScript
6
star
58

stale-model-example

This is the repo for the Preventing Stale Models in Production blog post.
Jupyter Notebook
6
star
59

gto-action

โš™๏ธ GTO Github Action
Shell
6
star
60

morefs

A collection of self-contained fsspec-based filesystems
Python
6
star
61

llm-demo

Demo of using DVC with LangChain
Python
6
star
62

course-checkpoints-project

This is the project we use for the DVC educational course to demonstrate how checkpoints work.
HTML
6
star
63

dvc-checkpoints-mnist

Example of checkpoints in a DVC project training a simple convolutional neural net to classify MNIST data
Python
5
star
64

dvc-s3-repo

Maintain deb and rpm repositories on s3
Python
5
star
65

enhancement-proposals

5
star
66

evidently-dvc

Tutorial: Automate Data Validation and Model Monitoring Pipelines with DVC and Evidently
HTML
5
star
67

dvc-gs

Google Storage plugin for dvc
Python
4
star
68

link-check

A Node-based tool to verify if links are alive. Built to be used anywhere!
TypeScript
4
star
69

dvc-snap

dvc snap package
Shell
4
star
70

cml-runner-base-case

Python
4
star
71

pretrained-model-demo

Python
4
star
72

homebrew-dvc

Automatic updates for dvc homebrew package
Shell
4
star
73

sqltrie

SQL-based prefix tree implementation inspired by pygtrie and python-diskcache
Python
3
star
74

vscode-dvc-demo

Python
3
star
75

example_model_export_cml

Example on how to use CML to provision an AWS EC2 runner, train a model, and export the resulting model.
Python
3
star
76

cnn_tutorial

CNN tutorial for DVC
Python
3
star
77

cml-playground

Shell
3
star
78

dvc-learn-project

This is the project used in the DVC Learn Meetups and videos.
HTML
3
star
79

telemetry-python

Common library to send usage telemetry
Python
3
star
80

checkpoints-tutorial

This is the code used in the checkpoints tutorial.
Python
3
star
81

ldb

Python
3
star
82

chocolatey-dvc

Chocolatey package for dvc
PowerShell
3
star
83

blog-tpi-jupyter

Terraform Provider Iterative + Jupyter + TensorBoard + AWS/Azure/GCP/K8s
Jupyter Notebook
3
star
84

example-get-started-ssh-private-fixture

Frozen copy of the Get Started DVC project. Used in Studio tests to test SSH remote credentials.
Python
2
star
85

link-check.action

A GitHub Action driver for link-check, deployed via submodules.
JavaScript
2
star
86

dvc-gdrive

Google Drive plugin for DVC
Python
2
star
87

example-get-started-s3

Example get started (metrics and plots in S3)
Python
2
star
88

example-mlem

Example of using MLEM with DVC Pipeline
Python
2
star
89

katacoda-scenarios

Interactive Katacoda Scenarios
Shell
2
star
90

dvc-exe

Private repository for building and signing dvc for windows
Inno Setup
2
star
91

dvc-azure

Azure plugin for dvc
Python
2
star
92

dvc_action_example

Python
2
star
93

dvc-oss

Alibaba OSS plugin for dvc
Python
2
star
94

sagemaker-pipeline

An example project, showcasing a DVC pipeline using SageMaker SDK for data preparation and model training
Python
2
star
95

dvc-test

Integration tests for dvc
Python
2
star
96

vscode-dvc-pack

2
star
97

get-started-pipelines

DVC Get Started project with a focus on creating pipelines with `dvc stage`
Python
2
star
98

cookiecutter-dvc-plugin

A Cookiecutter template for dvc plugins
Python
2
star
99

testing-ldb

Aug 10th Hackathon
Python
2
star
100

gha-required-workflows

Repository containing Iterative Required workflows for GitHub Actions
1
star