• Stars
    star
    892
  • Rank 51,172 (Top 2 %)
  • Language
    Python
  • License
    MIT License
  • Created over 1 year ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

⛓️ Serving LangChain LLM apps and agents automagically with FastApi. LLMops

Langcorn

LangCorn is an API server that enables you to serve LangChain models and pipelines with ease, leveraging the power of FastAPI for a robust and efficient experience.

GitHub Contributors GitHub Last Commit GitHub Issues GitHub Pull Requests Github License

Features

  • Easy deployment of LangChain models and pipelines
  • Ready to use auth functionality
  • High-performance FastAPI framework for serving requests
  • Scalable and robust solution for language processing applications
  • Supports custom pipelines and processing
  • Well-documented RESTful API endpoints
  • Asynchronous processing for faster response times

📦 Installation

To get started with LangCorn, simply install the package using pip:

pip install langcorn

⛓️ Quick Start

Example LLM chain ex1.py

import os

from langchain import LLMMathChain, OpenAI

os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "sk-********")

llm = OpenAI(temperature=0)
chain = LLMMathChain(llm=llm, verbose=True)

Run your LangCorn FastAPI server:

langcorn server examples.ex1:chain


[INFO] 2023-04-18 14:34:56.32 | api:create_service:75 | Creating service
[INFO] 2023-04-18 14:34:57.51 | api:create_service:85 | lang_app='examples.ex1:chain':LLMChain(['product'])
[INFO] 2023-04-18 14:34:57.51 | api:create_service:104 | Serving
[INFO] 2023-04-18 14:34:57.51 | api:create_service:106 | Endpoint: /docs
[INFO] 2023-04-18 14:34:57.51 | api:create_service:106 | Endpoint: /examples.ex1/run
INFO:     Started server process [27843]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8718 (Press CTRL+C to quit)

or as an alternative

python -m langcorn server examples.ex1:chain

Run multiple chains

python -m langcorn server examples.ex1:chain examples.ex2:chain


[INFO] 2023-04-18 14:35:21.11 | api:create_service:75 | Creating service
[INFO] 2023-04-18 14:35:21.82 | api:create_service:85 | lang_app='examples.ex1:chain':LLMChain(['product'])
[INFO] 2023-04-18 14:35:21.82 | api:create_service:85 | lang_app='examples.ex2:chain':SimpleSequentialChain(['input'])
[INFO] 2023-04-18 14:35:21.82 | api:create_service:104 | Serving
[INFO] 2023-04-18 14:35:21.82 | api:create_service:106 | Endpoint: /docs
[INFO] 2023-04-18 14:35:21.82 | api:create_service:106 | Endpoint: /examples.ex1/run
[INFO] 2023-04-18 14:35:21.82 | api:create_service:106 | Endpoint: /examples.ex2/run
INFO:     Started server process [27863]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8718 (Press CTRL+C to quit)

Import the necessary packages and create your FastAPI app:

from fastapi import FastAPI
from langcorn import create_service

app:FastAPI = create_service("examples.ex1:chain")

Multiple chains

from fastapi import FastAPI
from langcorn import create_service

app:FastAPI = create_service("examples.ex2:chain", "examples.ex1:chain")

or

from fastapi import FastAPI
from langcorn import create_service

app: FastAPI = create_service(
    "examples.ex1:chain",
    "examples.ex2:chain",
    "examples.ex3:chain",
    "examples.ex4:sequential_chain",
    "examples.ex5:conversation",
    "examples.ex6:conversation_with_summary",
    "examples.ex7_agent:agent",
)

Run your LangCorn FastAPI server:

uvicorn main:app --host 0.0.0.0 --port 8000

Now, your LangChain models and pipelines are accessible via the LangCorn API server.

Docs

Automatically served FastAPI doc Live example hosted on vercel.

Auth

It possible to add a static api token auth by specifying auth_token

python langcorn server examples.ex1:chain examples.ex2:chain --auth_token=api-secret-value

or

app:FastAPI = create_service("examples.ex1:chain", auth_token="api-secret-value")

Custom API KEYs

POST http://0.0.0.0:3000/examples.ex6/run
X-LLM-API-KEY: sk-******
Content-Type: application/json

Handling memory

{
  "history": "string",
  "input": "What is brain?",
  "memory": [
    {
      "type": "human",
      "data": {
        "content": "What is memory?",
        "additional_kwargs": {}
      }
    },
    {
      "type": "ai",
      "data": {
        "content": " Memory is the ability of the brain to store, retain, and recall information. It is the capacity to remember past experiences, facts, and events. It is also the ability to learn and remember new information.",
        "additional_kwargs": {}
      }
    }
  ]
}

Response:

{
  "output": " The brain is an organ in the human body that is responsible for controlling thought, memory, emotion, and behavior. It is composed of billions of neurons that communicate with each other through electrical and chemical signals. It is the most complex organ in the body and is responsible for all of our conscious and unconscious actions.",
  "error": "",
  "memory": [
    {
      "type": "human",
      "data": {
        "content": "What is memory?",
        "additional_kwargs": {}
      }
    },
    {
      "type": "ai",
      "data": {
        "content": " Memory is the ability of the brain to store, retain, and recall information. It is the capacity to remember past experiences, facts, and events. It is also the ability to learn and remember new information.",
        "additional_kwargs": {}
      }
    },
    {
      "type": "human",
      "data": {
        "content": "What is brain?",
        "additional_kwargs": {}
      }
    },
    {
      "type": "ai",
      "data": {
        "content": " The brain is an organ in the human body that is responsible for controlling thought, memory, emotion, and behavior. It is composed of billions of neurons that communicate with each other through electrical and chemical signals. It is the most complex organ in the body and is responsible for all of our conscious and unconscious actions.",
        "additional_kwargs": {}
      }
    }
  ]
}

Documentation

For more detailed information on how to use LangCorn, including advanced features and customization options, please refer to the official documentation.

👋 Contributing

Contributions to LangCorn are welcome! If you'd like to contribute, please follow these steps:

  • Fork the repository on GitHub
  • Create a new branch for your changes
  • Commit your changes to the new branch
  • Push your changes to the forked repository
  • Open a pull request to the main LangCorn repository

Before contributing, please read the contributing guidelines.

License

LangCorn is released under the MIT License.

More Repositories

1

Alfred-collection

A collection of all known Alfred3 workflows
Go
947
star
2

agentic_security

Agentic LLM Vulnerability Scanner
Python
606
star
3

hacker-slides

A small UI for building presentation slides from markdown markup
Go
348
star
4

flask-graphql-example

Example GraphQL application with Flask, pypy/python3 and MongoDB
Python
55
star
5

vector_lake

S3 vector database for LLM Agents and RAG.
Python
28
star
6

firex

Firex is a library for automatically generating command line interfaces (CLIs) from elixir module
Elixir
25
star
7

quick.py

Property-based testing library for Python
Python
16
star
8

validex

Simplifies the retrieval, extraction, and training of structured data from various unstructured sources.
Python
15
star
9

q-learner

Open ai gym q-learning for SpaceInvaders
Python
8
star
10

broccoli

A dependency injection package
Python
8
star
11

khromecat

Google chrome, chromecast cli player
Go
4
star
12

afk

A command-line tool called afk which gives you a simple reminder when you get back.
Python
4
star
13

graphitex

Graphite client for Elixir
Elixir
4
star
14

cloud_ips

Cloud IP ranges / CIDR blocks used by all cloud service providers GCP/AWS/DO/Oracle/Azure
Python
4
star
15

mix-install

Elixir
2
star
16

baguette

Python
2
star
17

toxic_proxy

An asyncio tcp proxy for network resilience testing
Python
2
star
18

docker-pyflame

Docker container for pyflame (a ptracing profiler for python) based on official python images
Perl
2
star
19

hayaku

Python
2
star
20

ex_form

Simplistic api wrapper for Typeform api without abusing macro/dsl
Elixir
2
star
21

coorl

A coooool curl command from net/http
Go
1
star
22

docker-alfred

Alfred3 workflow for docker
1
star