# grab the code.
git clone https://github.com/shawwn/openai-server
cd openai-server

# install dependencies.
pip3 install -r requirements.txt

# grab a gpt-2 model.
python3 download_model.py 117M # or 345M, 774M, 1558M

# start the server.
MODELS=117M bash prod.sh

# in a new terminal, ask for a completion.
bash 002_test_completion.sh 'Hello there. My name is'

$ OPENAI_API_BASE=http://localhost:9000 openai api completions.create -e davinci -p 'Hello, world' -t 0.8 -M 16 -n 4
===== Completion 0 =====
Hello, world. It seems like a good idea to make a living. The fact that it
===== Completion 1 =====
Hello, world. This is not the first time you're seeing the same thing at any given
===== Completion 2 =====
Hello, world, please do my best to continue the development of Monad and its conforming
===== Completion 3 =====
Hello, world controlled enemy.

"Be careful. We have come across a near total

$ bash 003_completions.sh 'Yo dawg, we implemented OpenAI API'
Yo dawg, we implemented OpenAI API. Now, we have the ability to connect to Signal, a cryptographic data store.

We can now make this secure by using new kid on the block chain, OpenAI.

OpenAI is the new block chain protocol for the internet. This is a major milestone. As the internet becomes more open and open for everybody, it is important for us to have a robust, high-quality blockchain. It is also important that we never create an untraceable chain. The blockchain is the only way to guarantee that everyone has the same access to the network.

We are an open consortium and we believe that the blockchain is the bridge between the internet and the rest of the world. We're committed to this project. We believe that the blockchain is a bridge between the internet and
^C

$ curl 'http://localhost:9000/v1/engines/117M/completions?prompt=Hello,%20my%20name%20is&max_tokens=32&n=4&temperature=0.9&echo=true'
{
  "choices": [
    {
      "finish-reason": "length",
      "index": 0,
      "logprobs": null,
      "text": "Hello, my name is Loium Chazz, and I have been far from satisfied with your departure. But I will, at least by some chance, give you permission to decide for"
    },
    {
      "finish-reason": "length",
      "index": 1,
      "logprobs": null,
      "text": "Hello, my name is Tim and my name is Jodie. Yours, Tom.\n\nTim: Oh hello, my name is Tim.\n\nJB: Where?'"
    },
    {
      "finish-reason": "length",
      "index": 2,
      "logprobs": null,
      "text": "Hello, my name is Rosen Sylvan. That's right, Buck Paoli, who was a member of the Board of Governors for George W. Bush in the 2009 Democratic primary\u2014"
    },
    {
      "finish-reason": "length",
      "index": 3,
      "logprobs": null,
      "text": "Hello, my name is Nick Martens, I am an English-speaking Canadian, University of Toronto, Mississauga, Canada. I work in a computer software company located in Canada."
    }
  ],
  "created": 1601701785.777768,
  "id": "cmpl-3qN8kwW1Ya7_qxWz4h8wuIzN",
  "model": "117M",
  "object": "text_completion"
}

$ T=0.8 M=32 bash 002_test_completion.sh 'Hello, my name is'
Hello, my name is Plato and, like many of you, I am very happy with the pre-release.

The primary goal of the pre-release was to provide

bash 002_test_completion.sh 'Hello there. My name is'

T=0.8 M=32 bash 002_test_completion.sh 'Hello there. My name is'

T=0.8 M=32 bash 002_test_completion.sh README.md

# grab the code.
git clone https://github.com/shawwn/openai-server
cd openai-server

# install dependencies.
pip3 install -r requirements.txt

# grab all models (requires ~8GB of disk space; if low, just download 117M, which only requires 550MB)
python3 download_model.py 117M
python3 download_model.py 345M
python3 download_model.py 774M
python3 download_model.py 1558M

# then, do *one* of the following:

# ...serve one specific model:
MODELS=117M bash prod.sh

# ...or serve multiple models:
MODELS=1558M,117M bash prod.sh

# ...or serve all models you've downloaded (the default):
bash prod.sh

PORT=8080 bash prod.sh

curl 'http://localhost:9000/v1/engines/1558M/completions?prompt=Hello,%20my%20name%20is&max_tokens=32&n=4&temperature=0.4&frequency_penalty=0.9&echo=true'

while true; do MODELS=117M bash prod.sh ; sleep 20 ; done