• Stars
    star
    476
  • Rank 88,850 (Top 2 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created over 6 years ago
  • Updated 24 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Machine Learning on Google Cloud Platform

Machine Learning on Google Cloud Platform

Guides to bringing your code from various Machine Learning frameworks to Google Cloud Platform.

The goal is to present recipes and practices that will help you spend less time wrangling with the various interfaces and more time exploring your datasets, building your models, and in general solving the problems you really care about.


Blog posts

  1. Genomic ancestry inference with deep learning - Ancestry inference on Google Cloud Platform using the 1000 Genomes dataset

  2. Running TensorFlow inference workloads at scale with TensorRT 5 and NVIDIA T4 GPUs - Creating a demo of ML inference using Tesla T4, TensorFlow, TensorRT, Load balancing and Auto-scale.

  3. NVIDIA’s RAPIDS joins our set of Deep Learning VM images for faster data science - Google Cloud’s set of Deep Learning Virtual Machine (VM) images, which enable the one-click setup machine learning-focused development environments. But some data scientists still use a combination of pandas, Dask, scikit-learn, and Spark on traditional CPU-based instances. If you’d like to speed up your end-to-end pipeline through scale, Google Cloud’s Deep Learning VMs now include an experimental image with RAPIDS, NVIDIA’s open source and Python-based GPU-accelerated data processing and machine learning libraries that are a key part of NVIDIA’s larger collection of CUDA-X AI accelerated software. CUDA-X AI is the collection of NVIDIA's GPU acceleration libraries to accelerate deep learning, machine learning, and data analysis.

  4. Inferring Machine Learning Models from Google Cloud Functions - Introduction to Inferring AI Platform models from Google Cloud Function endpoints.

  5. NVIDIA Achieves Breakthroughs in Language Understanding to Enable Real-Time Conversational AI - BERT Notebook in AI Hub and AI Platform Notebooks


TensorFlow

  1. Estimators - A guide to the Estimator interface.

scikit-learn

  1. scikit-learn on GCE - Train a simple model with scikit-learn on a Google Compute Engine

  2. Model serve - Serve model with Google App Engine and Cloud Endpoints.

  3. Hyperparameter search - Hyperparameter search on a Google Kubernetes Engine cluster from a Jupyter notebook.


Google Compute Engine

  1. Compute Engine survival training - Introduces a framework for running resilient training jobs on Google Compute Engine.

  2. Compute Engine burst training - A guide to using powerful VMs to quickly and cheaply perform computationally intensive training jobs. (The example training job in this guide uses xgboost as well as scikit-learn.)


Google Cloud Functions

  1. Google Cloud Functions + AI Platform Example - Example endpoints to infer AI Platform models.

Example Zoo

Collections of examples adapted to be runnable on AI Platform.

  1. tensorflow-probability examples.

  2. tensorflow-models examples.

Google Machine Learning Repositories

If you’re looking for our guides on how to do Machine Learning on Google Cloud Platform (GCP) using other services, please checkout our other repositories:

  • AI Platform samples, which has guides on how to bring your code from various ML frameworks to Google Cloud AI Platform using different products such as AI Platform Training, Prediction, Notebooks and AI Hub.
  • Keras Idiomatic Programmer This repository contains content produced by Google Cloud AI Developer Relations for machine learning and artificial intelligence. The content covers a wide spectrum from educational, training, and research, covering from novices, junior/intermediate to advanced.
  • Professional Services, common solutions and tools developed by Google Cloud's Professional Services team.

More Repositories

1

microservices-demo

Sample cloud-first application with 10 microservices showcasing Kubernetes, Istio, and gRPC.
Go
15,783
star
2

terraformer

CLI tool to generate terraform files from existing infrastructure (reverse Terraform). Infrastructure to Code
Go
11,610
star
3

training-data-analyst

Labs and demos for courses for GCP Training (http://cloud.google.com/training).
Jupyter Notebook
7,479
star
4

python-docs-samples

Code samples used on cloud.google.com
Jupyter Notebook
6,985
star
5

generative-ai

Sample code and notebooks for Generative AI on Google Cloud
Jupyter Notebook
5,282
star
6

golang-samples

Sample apps and code written for Google Cloud in the Go programming language.
Go
4,136
star
7

nodejs-docs-samples

Node.js samples for Google Cloud Platform products.
JavaScript
2,762
star
8

tensorflow-without-a-phd

A crash course in six episodes for software developers who want to become machine learning practitioners.
Jupyter Notebook
2,735
star
9

professional-services

Common solutions and tools developed by Google Cloud's Professional Services team. This repository and its contents are not an officially supported Google product.
Python
2,730
star
10

gcsfuse

A user-space file system for interacting with Google Cloud Storage
Go
1,977
star
11

community

Java
1,908
star
12

PerfKitBenchmarker

PerfKit Benchmarker (PKB) contains a set of benchmarks to measure and compare cloud offerings. The benchmarks use default settings to reflect what most users will see. PerfKit Benchmarker is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.
Python
1,855
star
13

java-docs-samples

Java and Kotlin Code samples used on cloud.google.com
Java
1,610
star
14

ml-design-patterns

Source code accompanying O'Reilly book: Machine Learning Design Patterns
Jupyter Notebook
1,600
star
15

continuous-deployment-on-kubernetes

Get up and running with Jenkins on Google Kubernetes Engine
Shell
1,582
star
16

cloudml-samples

Cloud ML Engine repo. Please visit the new Vertex AI samples repo at https://github.com/GoogleCloudPlatform/vertex-ai-samples
Python
1,507
star
17

asl-ml-immersion

This repos contains notebooks for the Advanced Solutions Lab: ML Immersion
Jupyter Notebook
1,469
star
18

localllm

Python
1,449
star
19

cloud-builders

Builder images and examples commonly used for Google Cloud Build
Go
1,354
star
20

cloud-foundation-fabric

End-to-end modular samples and landing zones toolkit for Terraform on GCP.
HCL
1,343
star
21

vertex-ai-samples

Sample code and notebooks for Vertex AI, the end-to-end machine learning platform on Google Cloud
Jupyter Notebook
1,331
star
22

cloud-builders-community

Community-contributed images for Google Cloud Build
Go
1,233
star
23

data-science-on-gcp

Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017
Jupyter Notebook
1,230
star
24

berglas

A tool for managing secrets on Google Cloud
Go
1,223
star
25

cloud-sql-proxy

A utility for connecting securely to your Cloud SQL instances
Go
1,218
star
26

kubernetes-engine-samples

Sample applications for Google Kubernetes Engine (GKE)
HCL
1,178
star
27

functions-framework-nodejs

FaaS (Function as a service) framework for writing portable Node.js functions
TypeScript
1,162
star
28

cloud-vision

Sample code for Google Cloud Vision
Python
1,093
star
29

DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
Java
1,078
star
30

bigquery-utils

Useful scripts, udfs, views, and other utilities for migration and data warehouse operations in BigQuery.
Java
1,030
star
31

php-docs-samples

A collection of samples that demonstrate how to call Google Cloud services from PHP.
PHP
944
star
32

buildpacks

Builders and buildpacks designed to run on Google Cloud's container platforms
Go
937
star
33

deploymentmanager-samples

Deployment Manager samples and templates.
Jinja
928
star
34

bank-of-anthos

Retail banking sample application showcasing Kubernetes and Google Cloud
Java
926
star
35

cloud-foundation-toolkit

The Cloud Foundation toolkit provides GCP best practices as code.
Go
916
star
36

flask-talisman

HTTP security headers for Flask
Python
896
star
37

DataflowJavaSDK

Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.
857
star
38

gsutil

A command line tool for interacting with cloud storage services.
Python
857
star
39

k8s-config-connector

GCP Config Connector, a Kubernetes add-on for managing GCP resources
Go
826
star
40

nodejs-getting-started

A tutorial for creating a complete application using Node.js on Google Cloud Platform
JavaScript
800
star
41

keras-idiomatic-programmer

Books, Presentations, Workshops, Notebook Labs, and Model Zoo for Software Engineers and Data Scientists wanting to learn the TF.Keras Machine Learning framework
Jupyter Notebook
797
star
42

gcr-cleaner

Delete untagged image refs in Google Container Registry or Artifact Registry
Go
795
star
43

metacontroller

Lightweight Kubernetes controllers as a service
Go
790
star
44

getting-started-python

Code samples for using Python on Google Cloud Platform
Python
756
star
45

magic-modules

Add Google Cloud Platform support to Terraform
HTML
753
star
46

awesome-google-cloud

A curated list of awesome stuff for Google Cloud.
742
star
47

mlops-on-gcp

Jupyter Notebook
728
star
48

dotnet-docs-samples

.NET code samples used on https://cloud.google.com
C#
719
star
49

click-to-deploy

Source for Google Click to Deploy solutions listed on Google Cloud Marketplace.
Ruby
709
star
50

cloud-sdk-docker

Google Cloud CLI Docker Image - Docker Image containing the gcloud CLI and its bundled components.
Dockerfile
697
star
51

iap-desktop

IAP Desktop is a Windows application that provides zero-trust Remote Desktop and SSH access to Linux and Windows VMs on Google Cloud.
C#
687
star
52

tf-estimator-tutorials

This repository includes tutorials on how to use the TensorFlow estimator APIs to perform various ML tasks, in a systematic and standardised way
Jupyter Notebook
671
star
53

functions-framework-python

FaaS (Function as a service) framework for writing portable Python functions
Python
670
star
54

flink-on-k8s-operator

[DEPRECATED] Kubernetes operator for managing the lifecycle of Apache Flink and Beam applications.
Go
659
star
55

terraform-google-examples

Collection of examples for using Terraform with Google Cloud Platform.
HCL
573
star
56

functions-framework-dart

FaaS (Function as a service) framework for writing portable Dart functions
Dart
531
star
57

cloud-run-button

Let anyone deploy your GitHub repos to Google Cloud Run with a single click
Go
520
star
58

govanityurls

Use a custom domain in your Go import path
Go
513
star
59

bigquery-oreilly-book

Source code accompanying: BigQuery: The Definitive Guide by Lakshmanan & Tigani to be published by O'Reilly Media
Jupyter Notebook
499
star
60

getting-started-java

Java
478
star
61

ipython-soccer-predictions

Sample iPython notebook with soccer predictions
Jupyter Notebook
473
star
62

covid-19-open-data

Datasets of daily time-series data related to COVID-19 for over 20,000 distinct locations around the world.
Python
470
star
63

ai-platform-samples

Official Repo for Google Cloud AI Platform. Find samples for Vertex AI, Google Cloud's new unified ML platform at: https://github.com/GoogleCloudPlatform/vertex-ai-samples
Jupyter Notebook
453
star
64

practical-ml-vision-book

Jupyter Notebook
441
star
65

gradle-appengine-templates

Freemarker based templates that build with the gradle-appengine-plugin
439
star
66

distributed-load-testing-using-kubernetes

Distributed load testing using Kubernetes on Google Container Engine
Smarty
438
star
67

terraform-validator

Terraform Validator is not an officially supported Google product; it is a library for conversion of Terraform plan data to CAI Assets. If you have been using terraform-validator directly in the past, we recommend migrating to `gcloud beta terraform vet`.
Go
436
star
68

hackathon-toolkit

GCP Hackathon Toolkit
HTML
434
star
69

monitoring-dashboard-samples

TypeScript
428
star
70

nodejs-docker

The Node.js Docker image used by Google App Engine Flexible.
TypeScript
406
star
71

cloud-ops-sandbox

Cloud Operations Sandbox is an open source collection of tools that helps practitioners to learn O11y and R9y practices from Google and apply them using Cloud Operations suite of tools.
HCL
398
star
72

cloud-code-vscode

Cloud Code for Visual Studio Code: Issues, Documentation and more
392
star
73

k8s-stackdriver

Go
390
star
74

professional-services-data-validator

Utility to compare data between homogeneous or heterogeneous environments to ensure source and target tables match
Python
375
star
75

cloud-code-samples

Code templates to make working with Kubernetes feel like editing and debugging local code.
Java
374
star
76

require-so-slow

`require`s taking too much time? Profile 'em.
TypeScript
373
star
77

functions-framework-go

FaaS (Function as a service) framework for writing portable Go functions
Go
373
star
78

k8s-multicluster-ingress

kubemci: Command line tool to configure L7 load balancers using multiple kubernetes clusters
Go
372
star
79

compute-image-packages

Packages for Google Compute Engine Linux images.
Python
370
star
80

healthcare

Python
367
star
81

android-docs-samples

Java
365
star
82

stackdriver-errors-js

Client-side JavaScript exception reporting library for Cloud Error Reporting
JavaScript
358
star
83

google-cloud-iot-arduino

Google Cloud IOT Example on ESP8266
C++
340
star
84

istio-samples

Istio demos and sample applications for GCP
Shell
331
star
85

ios-docs-samples

iOS samples that demonstrate APIs and services of Google Cloud Platform.
Swift
325
star
86

mlops-with-vertex-ai

An end-to-end example of MLOps on Google Cloud using TensorFlow, TFX, and Vertex AI
Jupyter Notebook
317
star
87

cloud-code-intellij

Plugin to support the Google Cloud Platform in IntelliJ IDEA - Docs and Issues Repository
315
star
88

gcping

The source for the CLI and web app at gcping.com
Go
303
star
89

spring-cloud-gcp

New home for Spring Cloud GCP development starting with version 2.0.
Java
299
star
90

airflow-operator

Kubernetes custom controller and CRDs to managing Airflow
Go
296
star
91

security-analytics

Community Security Analytics provides a set of community-driven audit & threat queries for Google Cloud
Python
289
star
92

elixir-samples

A collection of samples on using Elixir with Google Cloud Platform.
Elixir
289
star
93

gke-networking-recipes

Shell
286
star
94

datalab-samples

Jupyter Notebook
281
star
95

compute-archlinux-image-builder

A tool to build a Arch Linux Image for GCE
Shell
280
star
96

solutions-terraform-cloudbuild-gitops

HCL
276
star
97

kotlin-samples

Kotlin
276
star
98

gcpdiag

gcpdiag is a command-line diagnostics tool for GCP customers.
Python
268
star
99

PerfKitExplorer

PerfKit Explorer is a dashboarding and performance analysis tool built with Google technologies and easily extensible. PerfKit Explorer is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.
JavaScript
268
star
100

kube-jenkins-imager

Shell
261
star