• Stars
    star
    565
  • Rank 78,889 (Top 2 %)
  • Language
    Python
  • License
    Other
  • Created almost 5 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Google Drive API Python wrapper library. Maintained fork of PyDrive.

GHA Tests Conda-forge PyPI

PyDrive2

PyDrive2 is a wrapper library of google-api-python-client that simplifies many common Google Drive API V2 tasks. It is an actively maintained fork of https://pypi.python.org/pypi/PyDrive. By the authors and maintainers of the Git for Data - DVC project.

Project Info

Features of PyDrive2

  • Simplifies OAuth2.0 into just few lines with flexible settings.
  • Wraps Google Drive API V2 into classes of each resource to make your program more object-oriented.
  • Helps common operations else than API calls, such as content fetching and pagination control.
  • Provides fsspec filesystem implementation.

How to install

You can install PyDrive2 with regular pip command.

$ pip install PyDrive2

To install the current development version from GitHub, use:

$  pip install git+https://github.com/iterative/PyDrive2.git#egg=PyDrive2

OAuth made easy

Download client_secrets.json from Google API Console and OAuth2.0 is done in two lines. You can customize behavior of OAuth2 in one settings file settings.yaml.

from pydrive2.auth import GoogleAuth
from pydrive2.drive import GoogleDrive

gauth = GoogleAuth()
gauth.LocalWebserverAuth()

drive = GoogleDrive(gauth)

File management made easy

Upload/update the file with one method. PyDrive2 will do it in the most efficient way.

file1 = drive.CreateFile({'title': 'Hello.txt'})
file1.SetContentString('Hello')
file1.Upload() # Files.insert()

file1['title'] = 'HelloWorld.txt'  # Change title of the file
file1.Upload() # Files.patch()

content = file1.GetContentString()  # 'Hello'
file1.SetContentString(content+' World!')  # 'Hello World!'
file1.Upload() # Files.update()

file2 = drive.CreateFile()
file2.SetContentFile('hello.png')
file2.Upload()
print('Created file %s with mimeType %s' % (file2['title'],
file2['mimeType']))
# Created file hello.png with mimeType image/png

file3 = drive.CreateFile({'id': file2['id']})
print('Downloading file %s from Google Drive' % file3['title']) # 'hello.png'
file3.GetContentFile('world.png')  # Save Drive file as a local file

# or download Google Docs files in an export format provided.
# downloading a docs document as an html file:
docsfile.GetContentFile('test.html', mimetype='text/html')

File listing pagination made easy

PyDrive2 handles file listing pagination for you.

# Auto-iterate through all files that matches this query
file_list = drive.ListFile({'q': "'root' in parents"}).GetList()
for file1 in file_list:
    print('title: {}, id: {}'.format(file1['title'], file1['id']))

# Paginate file lists by specifying number of max results
for file_list in drive.ListFile({'maxResults': 10}):
    print('Received {} files from Files.list()'.format(len(file_list))) # <= 10
    for file1 in file_list:
        print('title: {}, id: {}'.format(file1['title'], file1['id']))

Fsspec filesystem

PyDrive2 provides easy way to work with your files through fsspec compatible GDriveFileSystem.

from pydrive2.fs import GDriveFileSystem

fs = GDriveFileSystem("root", client_id=my_id, client_secret=my_secret)

for root, dnames, fnames in fs.walk(""):
    ...

Concurrent access made easy

All API functions made to be thread-safe.

Contributors

Thanks to all our contributors!

https://contrib.rocks/image?repo=iterative/PyDrive2

More Repositories

1

dvc

🦉 ML Experiments and Data Management with Git
Python
13,294
star
2

cml

♾️ CML - Continuous Machine Learning | CI/CD for ML
JavaScript
3,996
star
3

datachain

AI-dataframe to enrich, transform and analyze data from cloud storages for ML training and LLM apps
Python
757
star
4

mlem

🐶 A tool to package, serve, and deploy any ML model on any platform. Archived to be resurrected one day🤞
Python
717
star
5

shtab

↔️ Automagic shell tab completion for Python CLI applications
Python
362
star
6

dvc.org

📖 DVC website and documentation
TypeScript
320
star
7

terraform-provider-iterative

☁️ Terraform plugin for machine learning workloads: spot instance recovery & auto-termination | AWS, GCP, Azure, Kubernetes
Go
288
star
8

vscode-dvc

Machine learning experiment tracking and data versioning with DVC extension for VS Code
TypeScript
187
star
9

example-get-started

Get started DVC project
Python
167
star
10

dvclive

📈 Log and track ML metrics, parameters, models with Git and/or DVC
Python
161
star
11

gto

🏷️ Git Tag Ops. Turn your Git repository into Artifact Registry or Model Registry.
Python
138
star
12

awesome-iterative-projects

A list of projects relying on Iterative.AI tools to achieve awesomeness
64
star
13

dataset-registry

Dataset registry DVC project
60
star
14

magnetic-tiles-defect

Demo Computer Vision Project
Jupyter Notebook
59
star
15

example-get-started-experiments

Get started DVC project
Python
45
star
16

course-ds-base

Jupyter Notebook
44
star
17

demo-bank-customer-churn

Demo DVC project training a classification model on tabular data
Jupyter Notebook
38
star
18

aita_dataset

AITA dataset based on r/AmItheAsshole/
Python
33
star
19

setup-dvc

DVC GitHub action
JavaScript
30
star
20

ldb-resources

Python
27
star
21

mlem.ai

✨ Landing page for MLEM
TypeScript
27
star
22

example_cml

Python
27
star
23

workshop-uncool-mlops

Accompanies the uncool MLOps workshop
Python
26
star
24

setup-cml

GitHub Action for CML setup
TypeScript
24
star
25

scmrepo

SCM wrapper and fsspec filesystem for Git for use in DVC.
Python
21
star
26

cml_base_case

Python
21
star
27

example-repos-dev

Source code and generator scripts for example DVC projects
Python
21
star
28

cml_cloud_case

Python
20
star
29

dvc-bench

Benchmarks for DVC
Shell
20
star
30

cml_dvc_case

Python
18
star
31

dvc-data

DVC's data management subsystem
Python
18
star
32

intellij-dvc

DVC integration plugin for Intellij IDEs including PyCharm, IntelliJ IDEA and CLion
Java
17
star
33

studio-support

❓ DVC Studio Issues, Question, and Discussions
16
star
34

pytest-servers

Create temporary directories on the various filesystems for testing
Python
15
star
35

studio-selfhosted

This repository contains auxiliary installation code for self-hosting Studio
Shell
14
star
36

py-template

Hypermodern Python Cookiecutter
Python
14
star
37

VSCode-DVC-Workshop

Workshop about DVC VSCode Extension
Jupyter Notebook
14
star
38

example-dvc-experiments

DVC Get Started Project with a focus on `dvc experiment` features.
HTML
13
star
39

cml.dev

🔗 CML website and documentation
TypeScript
12
star
40

dvcyaml-schema

Schema for dvc.yaml
Python
10
star
41

dvc-objects

dvc objects - contains filesystem and object-db level abstractions to use in dvc and dvc-data
Python
10
star
42

example-versioning

Data sets and ML models versioning example from DVC get started
Python
9
star
43

dvc-streamlit-components

Streamlit components for DVC
Python
9
star
44

morefs

A collection of self-contained fsspec-based filesystems
Python
9
star
45

priority-list

⛏️ Make a dent in GitHub issue & PR backlogs across repositories
Python
8
star
46

cml_tensorboard_case

Python
8
star
47

dvc-s3

AWS S3 plugin for dvc
Python
8
star
48

llm-demo

Demo of using DVC with LangChain
Python
8
star
49

example-mlem-get-started

Get Started MLEM project
Python
7
star
50

dvc-task

Celery task queue used in DVC
Python
7
star
51

tpi

Python wrapper for terraform-provider-iterative
Python
7
star
52

example-gto

Get Started GTO Project
7
star
53

example-pokemon-classifier

Example project with a CNN to train a Pokémon type classifier.
Python
7
star
54

dvc-render

Library for rendering DVC plots
Python
6
star
55

dvc-studio-client

Client to interact with DVC Studio
Python
6
star
56

pytest-test-utils

Python
6
star
57

workshop-uncool-mlops-solution

Python
6
star
58

gatsby-theme-iterative

A Gatsby theme for shared logic between all the websites from iterative.ai
JavaScript
6
star
59

stale-model-example

This is the repo for the Preventing Stale Models in Production blog post.
Jupyter Notebook
6
star
60

gto-action

⚙️ GTO Github Action
Shell
6
star
61

evidently-dvc

Tutorial: Automate Data Validation and Model Monitoring Pipelines with DVC and Evidently
HTML
6
star
62

features

A collection of development container 'features' for machine learning and data science
Shell
6
star
63

course-checkpoints-project

This is the project we use for the DVC educational course to demonstrate how checkpoints work.
HTML
6
star
64

sqltrie

SQL-based prefix tree implementation inspired by pygtrie and python-diskcache
Python
5
star
65

dvc-checkpoints-mnist

Example of checkpoints in a DVC project training a simple convolutional neural net to classify MNIST data
Python
5
star
66

dvc-s3-repo

Maintain deb and rpm repositories on s3
Python
5
star
67

enhancement-proposals

5
star
68

dvc-gs

Google Storage plugin for dvc
Python
4
star
69

link-check

A Node-based tool to verify if links are alive. Built to be used anywhere!
TypeScript
4
star
70

dvc-snap

dvc snap package
Shell
4
star
71

sagemaker-pipeline

An example project, showcasing a DVC pipeline using SageMaker SDK for data preparation and model training
Python
4
star
72

cml-runner-base-case

Python
4
star
73

cml-playground

Shell
4
star
74

pretrained-model-demo

Python
4
star
75

homebrew-dvc

Automatic updates for dvc homebrew package
Shell
4
star
76

vscode-dvc-demo

Python
3
star
77

example_model_export_cml

Example on how to use CML to provision an AWS EC2 runner, train a model, and export the resulting model.
Python
3
star
78

dvc-azure

Azure plugin for dvc
Python
3
star
79

cnn_tutorial

CNN tutorial for DVC
Python
3
star
80

telemetry-python

Common library to send usage telemetry
Python
3
star
81

dvc-learn-project

This is the project used in the DVC Learn Meetups and videos.
HTML
3
star
82

checkpoints-tutorial

This is the code used in the checkpoints tutorial.
Python
3
star
83

ldb

Python
3
star
84

chocolatey-dvc

Chocolatey package for dvc
PowerShell
3
star
85

blog-tpi-jupyter

Terraform Provider Iterative + Jupyter + TensorBoard + AWS/Azure/GCP/K8s
Jupyter Notebook
3
star
86

dvc-gdrive

Google Drive plugin for DVC
Python
2
star
87

link-check.action

A GitHub Action driver for link-check, deployed via submodules.
JavaScript
2
star
88

katacoda-scenarios

Interactive Katacoda Scenarios
Shell
2
star
89

dvc-exe

Private repository for building and signing dvc for windows
Inno Setup
2
star
90

jameson-metrics

Metrics examples
Python
2
star
91

example-mlem

Example of using MLEM with DVC Pipeline
Python
2
star
92

example-get-started-s3

Example get started (metrics and plots in S3)
Python
2
star
93

dvc_action_example

Python
2
star
94

dvc-oss

Alibaba OSS plugin for dvc
Python
2
star
95

dvc-test

Integration tests for dvc
Python
2
star
96

reflink-copy

Python wrapper for `reflink_copy` Rust library
Python
2
star
97

vscode-dvc-pack

2
star
98

dvclive-exp-tracking

Example repo to show how to start tracking experiments in DVC by adding DVCLive to your Python code.
Jupyter Notebook
2
star
99

cookiecutter-dvc-plugin

A Cookiecutter template for dvc plugins
Python
2
star
100

testing-ldb

Aug 10th Hackathon
Python
2
star