• Stars
    star
    376
  • Rank 113,810 (Top 3 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 2 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Directly Connecting Python to LLMs via Strongly-Typed Functions, Dataclasses, Interfaces & Generic Types

llm-strategy

Release Build status codecov Commit activity License

Implementing the Strategy Pattern using LLMs.

Also, please see https://blog.blackhc.net/2022/12/llm_software_engineering/ for a wider perspective on why this could be important in the future.

This package adds a decorator llm_strategy that connects to an LLM (such as OpenAI’s GPT-3) and uses the LLM to "implement" abstract methods in interface classes. It does this by forwarding requests to the LLM and converting the responses back to Python data using Python's @dataclasses.

It uses the doc strings, type annotations, and method/function names as prompts for the LLM, and can automatically convert the results back into Python types (currently only supporting @dataclasses). It can also extract a data schema to send to the LLM for interpretation. While the llm-strategy package still relies on some Python code, it has the potential to reduce the need for this code in the future by using additional, cheaper LLMs to automate the parsing of structured data.

Example

from dataclasses import dataclass
from llm_strategy import llm_strategy
from langchain.llms import OpenAI


@llm_strategy(OpenAI(max_tokens=256))
@dataclass
class Customer:
    key: str
    first_name: str
    last_name: str
    birthdate: str
    address: str

    @property
    def age(self) -> int:
        """Return the current age of the customer.

        This is a computed property based on `birthdate` and the current year (2022).
        """

        raise NotImplementedError()


@dataclass
class CustomerDatabase:
    customers: list[Customer]

    def find_customer_key(self, query: str) -> list[str]:
        """Find the keys of the customers that match a natural language query best (sorted by closeness to the match).

        We support semantic queries instead of SQL, so we can search for things like
        "the customer that was born in 1990".

        Args:
            query: Natural language query

        Returns:
            The index of the best matching customer in the database.
        """
        raise NotImplementedError()

    def load(self):
        """Load the customer database from a file."""
        raise NotImplementedError()

    def store(self):
        """Store the customer database to a file."""
        raise NotImplementedError()


@llm_strategy(OpenAI(max_tokens=1024))
@dataclass
class MockCustomerDatabase(CustomerDatabase):
    def load(self):
        self.customers = self.create_mock_customers(10)

    def store(self):
        pass

    @staticmethod
    def create_mock_customers(num_customers: int = 1) -> list[Customer]:
        """
        Create mock customers with believable data (our customers are world citizens).
        """
        raise NotImplementedError()

See examples/customer_database_search.py for a full example.

Customer Database Viewer

Searching for a Customer

Searching for a Customer

Getting started with contributing

Clone the repository first. Then, install the environment and the pre-commit hooks with

make install

The CI/CD pipeline will be triggered when you open a pull request, merge to main, or when you create a new release.

To finalize the set-up for publishing to PyPi or Artifactory, see here. For activating the automatic documentation with MkDocs, see here. To enable the code coverage reports, see here.

Releasing a new version

  • Create an API Token on Pypi.
  • Add the API Token to your projects secrets with the name PYPI_TOKEN by visiting this page.
  • Create a new release on Github. Create a new tag in the form *.*.*.

For more details, see here.


Repository initiated with fpgmaas/cookiecutter-poetry.

More Repositories

1

tfpyth

Putting TensorFlow back in PyTorch, back in TensorFlow (differentiable TensorFlow PyTorch adapters).
Python
641
star
2

toma

Helps you write algorithms in PyTorch that adapt to the available (CUDA) memory
Python
354
star
3

BatchBALD

Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning.
Python
219
star
4

dart_repl

Proof of concept REPL shell for Dart
Dart
81
star
5

batchbald_redux

Reusable BatchBALD implementation
Jupyter Notebook
71
star
6

mdp

Make it easy to specify simple MDPs that are compatible with the OpenAI Gym.
Python
37
star
7

mnist_by_zip

Compression algorithms (like the well-known zip file compression) can be used for machine learning purposes, specifically for classifying hand-written digits (MNIST)
Jupyter Notebook
35
star
8

llmtracer

Trace LLM calls (and others) and visualize them in WandB, as interactive SVG or using a streaming local webapp
Python
12
star
9

player_of_jeopardy

ChatGPT can solve Jeopardy! clues really well!
Python
10
star
10

chatplayground

Chat Playground for LLMs
Python
9
star
11

ddu_dirty_mnist

Dirty-MNIST dataset introduced in "Deterministic Neural Networks with Inductive Biases Capture Epistemic and Aleatoric Uncertainty" (https://arxiv.org/abs/2102.11582)
Jupyter Notebook
7
star
12

pbt

Jupyter notebooks to play around with population based training, as described in https://arxiv.org/abs/1711.09846
Jupyter Notebook
7
star
13

blackboard-pagi

Python
7
star
14

2302.08981

Jupyter Notebook
5
star
15

batch_pong_poc

Instead of running one environment at a time or one per thread, run everything in batch using numpy on a single core.
Jupyter Notebook
4
star
16

hello-slurm

Shell
4
star
17

pytorch_datadiet

Python
3
star
18

laaos

Logs as append-only source.
Python
3
star
19

implicit_lambda

This package adds support for implicit lambdas, so you can write `map(_ + 5, a_list)` instead of `map(lambda x: x + 5, a_list)`.
Python
3
star
20

dlb_chapter2

TeX
1
star
21

WML

Whitespace Markup Language
C++
1
star
22

2020_ebm_presentation

A presentation about EBMs and Hopfield networks @ OATML
HTML
1
star
23

2202.01851

Repository for 'A Note on "Assessing Generalization of SGD via Disagreement"'
Jupyter Notebook
1
star
24

algo_fairness

Jupyter Notebook
1
star
25

2208.00549

Unifying Approaches in Data Subset Selection - Experiments
Jupyter Notebook
1
star