GoogleCloudPlatform/ml-design-patterns

Stars
1,600
Rank 29,247 (Top 0.6 %)
Language
Jupyter Notebook
License
Apache License 2.0
Created over 4 years ago
Updated over 1 year ago

GoogleCloudPlatform/ml-design-patterns

GoogleCloudPlatform

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Source code accompanying O'Reilly book: Machine Learning Design Patterns

This is not an official Google product

ml-design-patterns

Source code accompanying O'Reilly book:
Title: Machine Learning Design Patterns
Authors: Valliappa (Lak) Lakshmanan, Sara Robinson, Michael Munn

https://www.oreilly.com/library/view/machine-learning-design/9781098115777/

Buy from O'Reilly
Buy from Amazon

We will update this repo with source code as we write each chapter. Stay tuned!

Chapters

Preface
The Need for ML Design Patterns
Data representation design patterns
- #1 Hashed Feature
- #2 Embedding
- #3 Feature Cross
- #4 Multimodal Input
Problem representation design patterns
- #5 Reframing
- #6 Multilabel
- #7 Ensemble
- #8 Cascade
- #9 Neutral Class
- #10 Rebalancing
Patterns that modify model training
- #11 Useful overfitting
- #12 Checkpoints
- #13 Transfer Learning
- #14 Distribution Strategy
- #15 Hyperparameter Tuning
Resilience patterns
- #16 Stateless Serving Function
- #17 Batch Serving
- #18 Continuous Model Evaluation
- #19 Two Phase Predictions
- #20 Keyed Predictions
Reproducibility patterns
- #21 Transform
- #22 Repeatable Sampling
- #23 Bridged Schema
- #24 Windowed Inference
- #25 Workflow Pipeline
- #26 Feature Store
- #27 Model Versioning
Responsible AI
- #28 Heuristic benchmark
- #29 Explainable Predictions
- #30 Fairness Lens
Summary

microservices-demo

Sample cloud-first application with 10 microservices showcasing Kubernetes, Istio, and gRPC.

terraformer

CLI tool to generate terraform files from existing infrastructure (reverse Terraform). Infrastructure to Code

training-data-analyst

Labs and demos for courses for GCP Training (http://cloud.google.com/training).

Jupyter Notebook

python-docs-samples

Code samples used on cloud.google.com

Jupyter Notebook

generative-ai

Sample code and notebooks for Generative AI on Google Cloud, with Gemini on Vertex AI

Jupyter Notebook

golang-samples

Sample apps and code written for Google Cloud in the Go programming language.

professional-services

Common solutions and tools developed by Google Cloud's Professional Services team. This repository and its contents are not an officially supported Google product.

nodejs-docs-samples

Node.js samples for Google Cloud Platform products.

tensorflow-without-a-phd

A crash course in six episodes for software developers who want to become machine learning practitioners.

Jupyter Notebook

gcsfuse

A user-space file system for interacting with Google Cloud Storage

community

PerfKitBenchmarker

PerfKit Benchmarker (PKB) contains a set of benchmarks to measure and compare cloud offerings. The benchmarks use default settings to reflect what most users will see. PerfKit Benchmarker is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.

asl-ml-immersion

This repos contains notebooks for the Advanced Solutions Lab: ML Immersion

Jupyter Notebook

vertex-ai-samples

Notebooks, code samples, sample apps, and other resources that demonstrate how to use, develop and manage machine learning and generative AI workflows using Google Cloud Vertex AI.

Jupyter Notebook

java-docs-samples

Java and Kotlin Code samples used on cloud.google.com

continuous-deployment-on-kubernetes

Get up and running with Jenkins on Google Kubernetes Engine

cloudml-samples

Cloud ML Engine repo. Please visit the new Vertex AI samples repo at https://github.com/GoogleCloudPlatform/vertex-ai-samples

cloud-foundation-fabric

End-to-end modular samples and landing zones toolkit for Terraform on GCP.

localllm

cloud-builders

Builder images and examples commonly used for Google Cloud Build

cloud-sql-proxy

A utility for connecting securely to your Cloud SQL instances

cloud-builders-community

Community-contributed images for Google Cloud Build

berglas

A tool for managing secrets on Google Cloud

data-science-on-gcp

Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017

Jupyter Notebook

kubernetes-engine-samples

Sample applications for Google Kubernetes Engine (GKE)

functions-framework-nodejs

FaaS (Function as a service) framework for writing portable Node.js functions

DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks

bigquery-utils

Useful scripts, udfs, views, and other utilities for migration and data warehouse operations in BigQuery.

cloud-vision

Sample code for Google Cloud Vision

bank-of-anthos

Retail banking sample application showcasing Kubernetes and Google Cloud

buildpacks

Builders and buildpacks designed to run on Google Cloud's container platforms

php-docs-samples

A collection of samples that demonstrate how to call Google Cloud services from PHP.

cloud-foundation-toolkit

The Cloud Foundation toolkit provides GCP best practices as code.

deploymentmanager-samples

Deployment Manager samples and templates.

flask-talisman

HTTP security headers for Flask

k8s-config-connector

GCP Config Connector, a Kubernetes add-on for managing GCP resources

gsutil

A command line tool for interacting with cloud storage services.

DataflowJavaSDK

Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.

nodejs-getting-started

A tutorial for creating a complete application using Node.js on Google Cloud Platform

magic-modules

Add Google Cloud Platform support to Terraform

gcr-cleaner

Delete untagged image refs in Google Container Registry or Artifact Registry

keras-idiomatic-programmer

Books, Presentations, Workshops, Notebook Labs, and Model Zoo for Software Engineers and Data Scientists wanting to learn the TF.Keras Machine Learning framework

Jupyter Notebook

metacontroller

Lightweight Kubernetes controllers as a service

awesome-google-cloud

A curated list of awesome stuff for Google Cloud.

mlops-on-gcp

Jupyter Notebook

getting-started-python

Code samples for using Python on Google Cloud Platform

dotnet-docs-samples

.NET code samples used on https://cloud.google.com

click-to-deploy

Source for Google Click to Deploy solutions listed on Google Cloud Marketplace.

iap-desktop

IAP Desktop is a Windows application that provides zero-trust Remote Desktop and SSH access to Linux and Windows VMs on Google Cloud.

cloud-sdk-docker

Google Cloud CLI Docker Image - Docker Image containing the gcloud CLI and its bundled components.

tf-estimator-tutorials

This repository includes tutorials on how to use the TensorFlow estimator APIs to perform various ML tasks, in a systematic and standardised way

Jupyter Notebook

functions-framework-python

FaaS (Function as a service) framework for writing portable Python functions

flink-on-k8s-operator

[DEPRECATED] Kubernetes operator for managing the lifecycle of Apache Flink and Beam applications.

terraform-google-examples

Collection of examples for using Terraform with Google Cloud Platform.

functions-framework-dart

FaaS (Function as a service) framework for writing portable Dart functions

cloud-run-button

Let anyone deploy your GitHub repos to Google Cloud Run with a single click

bigquery-oreilly-book

Source code accompanying: BigQuery: The Definitive Guide by Lakshmanan & Tigani to be published by O'Reilly Media

Jupyter Notebook

govanityurls

Use a custom domain in your Go import path

ml-on-gcp

Machine Learning on Google Cloud Platform

practical-ml-vision-book

Jupyter Notebook

getting-started-java

ipython-soccer-predictions

Sample iPython notebook with soccer predictions

Jupyter Notebook

monitoring-dashboard-samples

Google Cloud Monitoring Dashboard Samples

covid-19-open-data

Datasets of daily time-series data related to COVID-19 for over 20,000 distinct locations around the world.

ai-platform-samples

Official Repo for Google Cloud AI Platform. Find samples for Vertex AI, Google Cloud's new unified ML platform at: https://github.com/GoogleCloudPlatform/vertex-ai-samples

Jupyter Notebook

hackathon-toolkit

GCP Hackathon Toolkit

gradle-appengine-templates

Freemarker based templates that build with the gradle-appengine-plugin

distributed-load-testing-using-kubernetes

Distributed load testing using Kubernetes on Google Container Engine

terraform-validator

Terraform Validator is not an officially supported Google product; it is a library for conversion of Terraform plan data to CAI Assets. If you have been using terraform-validator directly in the past, we recommend migrating to `gcloud beta terraform vet`.

cloud-code-vscode

Cloud Code for Visual Studio Code: Issues, Documentation and more

nodejs-docker

The Node.js Docker image used by Google App Engine Flexible.

cloud-ops-sandbox

Cloud Operations Sandbox is an open source collection of tools that helps practitioners to learn O11y and R9y practices from Google and apply them using Cloud Operations suite of tools.

professional-services-data-validator

Utility to compare data between homogeneous or heterogeneous environments to ensure source and target tables match

k8s-stackdriver

cloud-code-samples

Code templates to make working with Kubernetes feel like editing and debugging local code.

healthcare

require-so-slow

`require`s taking too much time? Profile 'em.

functions-framework-go

FaaS (Function as a service) framework for writing portable Go functions

k8s-multicluster-ingress

kubemci: Command line tool to configure L7 load balancers using multiple kubernetes clusters

compute-image-packages

Packages for Google Compute Engine Linux images.

android-docs-samples

stackdriver-errors-js

Client-side JavaScript exception reporting library for Cloud Error Reporting

applied-ai-engineering-samples

This repository compiles code samples and notebooks demonstrating how to use Generative AI on Google Cloud Vertex AI.

Jupyter Notebook

mlops-with-vertex-ai

An end-to-end example of MLOps on Google Cloud using TensorFlow, TFX, and Vertex AI

Jupyter Notebook

google-cloud-iot-arduino

Google Cloud IOT Example on ESP8266

istio-samples

Istio demos and sample applications for GCP

ios-docs-samples

iOS samples that demonstrate APIs and services of Google Cloud Platform.

cloud-code-intellij

Plugin to support the Google Cloud Platform in IntelliJ IDEA - Docs and Issues Repository

security-analytics

Community Security Analytics provides a set of community-driven audit & threat queries for Google Cloud

gke-networking-recipes

gcping

The source for the CLI and web app at gcping.com

solutions-terraform-cloudbuild-gitops

spring-cloud-gcp

New home for Spring Cloud GCP development starting with version 2.0.

airflow-operator

Kubernetes custom controller and CRDs to managing Airflow

genai-for-marketing

Showcasing Google Cloud's generative AI for marketing scenarios via application frontend, backend, and detailed, step-by-step guidance for setting up and utilizing generative AI tools, including examples of their use in crafting marketing materials like blog posts and social media content, nl2sql analysis, and campaign personalization.

Jupyter Notebook

elixir-samples

A collection of samples on using Elixir with Google Cloud Platform.

gcpdiag

gcpdiag is a command-line diagnostics tool for GCP customers.

kotlin-samples

compute-archlinux-image-builder

A tool to build a Arch Linux Image for GCE

datalab-samples

Jupyter Notebook