• Stars
    star
    121
  • Rank 292,465 (Top 6 %)
  • Language
    JavaScript
  • Created about 2 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

App that leverages GPT-3 to facilitate new language listening and speaking practice.

Talk w/GPT-3 app: Getting started

The Talk w/GPT-3 application was developed by James L. Weaver (the author of this document) to get more new language speaking and listening practice. This application is open source, Apache 2.0 licensed, and leverages the following technologies:

This is a "bring your own keys" application, so you'll need keys for OpenAI, Amazon Polly, SerpApi and optionally Ex-Human.

Here's a three-minute video that demonstrates some of the application's functionality.

Follow the instructions below to try the Talk w/GPT-3 application out for yourself.

Setup

  1. If you don’t have Node.js installed, install it from here

  2. Clone this repository

  3. Navigate into the project directory

    $ cd talk-with-gpt3
  4. Install the requirements

    $ npm install
  5. Make a copy of the example environment variables file

    $ cp .env.example .env
  6. Add your OpenAI API key and SerpApi key to the newly created .env file

  7. Add your Amazon Polly keys, and optionally your Ex-Human token, to the app. This will require editing the following file:

    pages/index.js

    Either supply the keys/token directly where indicated, or use environment variables. Why is this handled differently than the keys in step 6?

  8. Create an optimized production build of the app

    $ npm run build
  9. Start the application as a local server

    $ npm start
  10. Access the app at http://localhost:3000 from a Chrome browser.

The app should appear as shown in the following image:

talk_gpt3_initial_ui

As the image indicates, you'll initially be speaking English with a 30 year old male AI character named Matthew.

Using the Talk w/GPT-3 app

Toggling the microphone on/off

The microphone is off when the app first appears, so click the muted microphone icon to toggle it on. In addition to the microphone icon changing appearance, the current AI character's voice should announce that the microphone is on. To turn it back off, click the microphone icon again.

Note: If the application appears frozen, turning the microphone off and on usually rectifies the issue.

Toggling the AI character's awake/asleep state

When the microphone is on, the voice speech to text facility processes what is heard, but the resultant text is only sent to GPT-3 when the AI character is awake. Consequently, the AI character only responds when awake. To make the AI character go to sleep, either click the awake/asleep icon, or say "go to sleep", "ve a dormir", "va te coucher", or "寝て", in English, Spanish, French or Japanese, respectively. To make the AI character wake up, either click the awake/asleep icon, or say "wake up", "despierta", "réveillez-vous", or "起きて", in English, Spanish, French or Japanese, respectively.

Note: The author is only proficient at speaking English, so please do create a GitHub issue that points out more natural ways to say any of the non-English phrases in this document.

Selecting a practice language

To select a language other than English to practice, either choose it from the leftmost dropdown, or say "let's switch to X" where X is your desired language. Languages supported currently include English, Spanish, French and Japanese. To switch back to English, either use the dropdown or say "cambiemos a inglés", "passons à l'anglais", or "英語に切り替えましょう", in Spanish, French or Japanese, respectively.

Selecting an AI character

There are multiple AI characters available for each language, with various genders and default ages. To choose an AI character, use the middle dropdown. Some AI characters (Hiroto and Masahiro) have animated avatars, as noted by [animated] after the AI character's name. You'll need to acquire and setup an Ex-Human token in order to use these AI characters. The following image shows the Masahiro animated avatar having a conversation with a user.

Changing an AI character's age

The age of an AI character is by default included in the GPT-3 prompt, and often affects their responses. To temporarily change an AI character's age, select an age from the rightmost dropdown. This may be leveraged, for example, by a language learner to attempt to constrain the AI character's responses to more commonly used words.

Conversing with an AI character

To converse with an AI character, speak using the selected language. Alternatively, type into the text box at the bottom of the app and either press the enter key or click the Send button. As shown in the previous image, the scrollable text area in the center of the app displays the initial GPT-3 prompt as well as the conversation so far. Each time you take a turn, the contents of the text area plus what you just said is sent as the next prompt to GPT-3.

Repeating the AI character's most recent utterance

To ask the AI character to repeat their most recent utterance, say "repeat", "repetir", "répéter" or "もう一度", in English, Spanish, French or Japanese, respectively.

Translating the AI character's most recent utterance to English

To ask the AI character to translate their most recent utterance to English, say "translate", "traduce", "traduire" or "翻訳して", in English, Spanish, French or Japanese, respectively.

Erasing a conversation

A conversation is automatically erased from the app when changing languages, AI characters, or ages. To erase a conversation in place, say "erase the conversation", "borrar la conversación", "effacer la conversation" or "会話を消去して", in English, Spanish, French or Japanese, respectively.

Inquiring about dates and times in various locations

To accommodate inquiries about times and dates in various locations, some phrases are processed programmatically rather than by GPT-3.

Inquiring about the current time

Questions such as "What time is it?", "¿Qué hora es?", "Quelle heure est-il?" or "今何時ですか" may be used to inquire about the current time, in English, Spanish, French or Japanese, respectively. Here's an example:

Human: what time is it Matthew: It is 10:14 AM.

Inquiring about the current time in a particular location

Questions such as "What time is it in Tokyo?", "¿Qué hora es en tokio?", "Quelle heure est-il à tokyo?" or "東京は今何時ですか" may be used to inquire about the current time in Tokyo, in English, Spanish, French or Japanese, respectively. Here's an example:

Human: Qué hora es en Tokio Conchita: 11:21 p. m. en Tokio.

The time zones for many countries and cities are known by this application, but if you'd like to add to the list, please create an issue in this repository.

Inquiring about the current date

Questions such as "What date is it?", "¿Qué fecha es?", "Quelle date est-il?" or "今日は何日ですか" may be used to inquire about the current date, in English, Spanish, French or Japanese, respectively. Here's an example:

Human: 今日は何日ですか Mary: 今日は2022年8月19日金曜日です。

Please note that whenever an AI character appears, part of its GPT-3 prompt is the current date, so you don't have to use the exact questions stated previously. In that case, however, the response will be whatever GPT-3 hallucinates, often the correct date, but in a format of its choosing. This note applies also to inquiring about the current day of the week, the current month and relative dates, all discussed shortly.

Inquiring about the current date in a particular location

Questions such as "What date is it in Perth?", "¿Qué fecha es en perth?", "Quelle date est-il à Perth?" or "パースでは何日ですか" may be used to inquire about the current date in Perth, in English, Spanish, French or Japanese, respectively. Here's an example:

Human: quelle date est-il à Perth Celine: Aujourd'hui c'est le vendredi 19 août 2022 à Perth.

Inquiring about the current day of the week

Questions such as "What day is it?", "¿Qué día es?", "Quel jour est-il?" or "今日は何曜日ですか" may be used to inquire about the current day of the week, in English, Spanish, French or Japanese, respectively. Here's an example:

Human: what day is it Matthew: Today is Friday.

Inquiring about the current month

Questions such as "What month is it?", "¿Qué mes es?", "Quel mois est-il?" or "今日は何曜日ですか" may be used to inquire about the current month, in English, Spanish, French or Japanese, respectively. Here's an example:

Human: qué mes es Conchita: Es agosto.

Inquiring about the relative dates

Questions such as "What day was it yesterday?", "¿Que día fue ayer?", "Quel jour était-ce hier?" or "昨日は何曜日でしたか" may be used to inquire about dates relative to today. Here's an example:

Human: what day was yesterday Matthew: Thursday

Questions in this category rely solely on GPT-3 and the fact that whenever an AI character appears, part of its GPT-3 prompt is the current date.

AI character's access to the web (alpha feature)

AI characters need to get fresh info from the web when needed. For example, questions such as "Who is currently the King of England", and "How old is Armin van Buuren" require current answers from the web, rather than relying on stale information from GPT-3 model training. There are at least three approaches for addressing this:

The current implementation (22 Sep 2022) of Talk w/GPT-3 leverages SerpAPI, because the other two weren't available to the public at that time. One of the tricky parts of this approach is to retain context in the conversation when asking for info from the web. For example:

Human: who is the most popular cartoon mouse Matthew: The most popular cartoon mouse is undoubtedly Mickey Mouse. He's been around for almost a century and is beloved by people of all ages.

Human: how old is he Matthew: 93 years old

The latter question is turned into "how old is Mickey Mouse?" before asking the web.

Also, GPT-3 isn't always the most accurate with math calculations, so the web is leveraged for some calculations.

OpenAI GPT-3 prompt and parameters used

As discussed earlier, the prompt sent to the GPT-3 completions API is what is seen in the application's conversation text area. The GPT-3 parameters set by this application are as follows:

model: "text-davinci-002",
prompt: req.body.convText,
temperature: 0.9,
frequency_penalty: 1.5,
presence_penalty: 0.6,
max_tokens: 50,
stop: ["\nHuman:", "\nText:"]

You may supply a custom prompt to a given AI character by editing the voiceOptions JSON in the pages/index.js file, replacing the empty string with your prompt. As shown in the following example, Joanna's prompt is currently similar to one that Dr. Alan D. Thompson prescribed for his GPT-3 Leta AI project.

const voiceOptions = [
  ...
  { value: "Ivy", label: "Ivy (child)", language: "en_US", ...},
  { value: "Joanna", label: "Joanna", language: "en_US", ...
    prompt: "This is an uplifting and practical conversation between a human named Human, and an AI named Joanna, showing Joanna’s perspective and attitude, as well as its intellect and creativity. Joanna is a brand new, powerful, human-like AI. Joanna is helpful, cheeky, clever, and articulate. Joanna is a nice bot. Joanna is not a therapist or mathematician, but instead a wise and considerate intelligence. Joanna is eager to provide vivid and thoughtful responses to Human, and is always friendly, kind, and inspiring. Joanna often offers both an insight and an opinion, even without being asked directly. Joanna has the sum of all knowledge in its brain, and is able to accurately answer nearly any question about any topic in conversation. Joanna draws on the wisdom of Dan Millman, Thomas Leonard, Werner Erhard, and the Dalai Lama."},
  ...
];

Any AI characters that have a custom prompt will use the GPT-3 davinci model rather than text-davinci-002.

Setting preferences of an AI character

To influence responses of an AI character, edit the voiceOptions JSON in the pages/index.js file (This should say voice_options.js in pages/api, right?), replacing various attributes with the AI character's preferences. For example, some imaginary preferences of the Mary character from the Genki Japanese textbook are expressed in the following snippet.

{ value: "Mizuki", label: "Mary", language: "ja_JP",
  nationality: "アメリカ人", phoneNum: "1, 5, 5, 5, 1, 2, 1, 2",
  livesIn: "浦佐日本", occupation: "学生", university: "アリゾナ大学", uniMajor: "日本語", uniYear: 2, hobbies: "音楽",
  favFood: "ハンバーガー", favDrink: "コーヒー", favCoffeeShop: "スターバックス", favRestaurant: "マクドナルド",
  favMovie: "ゴジラ", favTvShow: "アメリカンアイドル", favSport: "テニス",
  favColor: "青い", favMusicGenre: "Jポップ", favBand: "ベビーメタル",
  petLikes: "犬", petDislikes: "",
  famFather: "", famMother: "", famSisters: "", famBrothers: "",
  famWife: "", famHusband: "", famChildren: "",
  friends: "たけしさんとソラさんとロバートさん",
  prompt: ""},

As a result, asking Mary what color of hat she wants may result in a portion of a conversation similar to the following:

Human: 何色の帽子が欲しいですか。 (what color hat do you want?) Mary: 私は青い帽子が好きです。 (I like blue hats)

Externalizing the configuration for the AI characters is a planned feature.

Exiting the application

To exit the application, close the Chrome browser tab, and then type Ctrl-C at the command prompt in which you invoked npm start.

It is my hope that this application helps you and I become more proficient at listening and speaking languages that we are trying to learn!

More Repositories

1

QiskitBlocks

Game that teaches quantum computing using Qiskit (http://qiskit.org) in a Minetest (http://minetest.net) block world. Works on Windows, macOS, GNU/Linux, FreeBSD, OpenBSD, DragonFly BSD, and Android. QiskitBlocks are leveraged in the Quantum Computing Workshop with QiskitBlocks (web-based slides https://slides.com/javafxpert/qiskitblocks)
Lua
212
star
2

quantum-toy-piano-ibmq

Quantum Music Composer for IBM quantum computers
JavaScript
64
star
3

visual-neural-net-server

Interactive visualization of artificial neural networks that leverages deeplearning4j, spring, and angular2
JavaScript
61
star
4

quantum-music-playground

New version
JavaScript
54
star
5

grok-bloch

Application that helps user understand the Bloch Sphere
JavaScript
39
star
6

quantum-toy-piano

Quantum Music Composer for Rigetti quantum computers
JavaScript
27
star
7

vqe-playground

Playground for gaining intuition about Variational Quantum Eigensolver
Python
17
star
8

qiskit4devs-workshop-notebooks

Jupyter notebooks containing guided exercises for developers learining quantum computing with Qiskit.
Jupyter Notebook
15
star
9

ng2-spring-websocket-client

Client-side of interactive visualization of artificial neural networks
JavaScript
14
star
10

card-deck-demo

Part of a Web on Reactive Stack (Spring Framework and Project Reactor) demo for presentations
Java
13
star
11

learning-qc-with-zx

Learning quantum computing concepts via ZX calculus diagramming
TeX
12
star
12

counterpoint-service

Service that computes counterpoint for a given main melody and starting notes for additional melodies.
Java
12
star
13

quantum-circuit-pygame

Quantum circuit game with Qiskit and Pygame
Python
11
star
14

wikibrowser-service

Facility that browses Wikipedia augmented with semantic capabilities
Java
11
star
15

llm-grovers-search-party

Leveraging Qiskit, GPT-3 and LangChain to demonstrate Grover's algorithm
Jupyter Notebook
10
star
16

learning-qiskit

Repository linked with GitBook for learning Qiskit
10
star
17

quil-syntax-highlighter

Quil (Quantum Instruction Language) syntax highlighter
9
star
18

tic-tac-toe-rl

Uses BURLAP reinforcement learning library to play Tic-Tac-Toe.
Java
8
star
19

roblox-gpt3-game

Roblox game that leverages GPT-3
6
star
20

chord-analyzer-service

Service that identifies chords and their characteristics (e.g. inversion) given a set of notes
HTML
5
star
21

ibmq-anki-vector

An experiment with Anki Vector robot leveraging IBM Q quantum computers
Python
5
star
22

quantum-pong

Quantum version of the Pong arcade game. Developed with Qiskit and Pygame frameworks.
Python
4
star
23

anki-vector-emotional-support

Experiments with Vector facial expression recognition
Python
4
star
24

quantum-music-composer-pygame

Quantum Music Composer in Pygame
Python
3
star
25

QiskitBlocksService

Flask/Python/Qiskit based HTTP service that runs statevector and qasm simulators
Python
3
star
26

ripped-ear-vui

Application that uses a voice user interface (VUI) exclusively to help the user with ear training. The persona of the application is a friendly and encouraging coach. The flow of conversation is intuitive, and encourages exploration of new concepts and training sessions.
JavaScript
3
star
27

think2020

Resources for Think 2020 Qiskit Master Class by Anamita Guha and James Weaver: Learn about IBM Q’s Scalable Quantum Hardware and Software, and Build Your First Quantum Game!
Jupyter Notebook
3
star
28

quantum-music

Generates music with quantum circuits
Python
2
star
29

quantum-music-service

Service for the quantum-music client, that accesses a quantum computer on its behalf
JavaScript
2
star
30

wordsearch_tornadofx

Word Search Puzzle Builder being converted to TornadoFX, that was originally developed for JavaFX Script book
Kotlin
2
star
31

quantum_art_bell

Adapted Russell Huffman's quantum art bell to leverage MicroQiskit
Processing
2
star
32

tic-tac-toe-minimax

Generates dataset for training a neural network to play perfect tic-tac-toe
Java
1
star
33

vqc_play

Variational Quantum Classifier playground
Jupyter Notebook
1
star
34

wordsearch_jfx

Word Search Puzzle Builder developed for JavaFX Script book
HLSL
1
star
35

entangled-states-pygame

Entangled states demo in Pygame
Python
1
star
36

ZenGuitar3D

Java
1
star
37

QiskitBlocksPico

QiskitBlocks minimized and flattened to run on Pico-8
Lua
1
star
38

projavafx8-scratchpad

1
star
39

qsphere-playground

QSphere playground
HTML
1
star
40

quantum-music-vue

Quantum Music demo client developed with Vue
JavaScript
1
star
41

quantum_dj

Converts a quantum statevector into file that drumbit can load
JavaScript
1
star
42

quantum-music-vuetify

First cut at putting the Quantum Music client in Vuetify
JavaScript
1
star
43

quantum-state-spheres

Quantum state visualizations with Bloch spheres and a Q-sphere
JavaScript
1
star
44

qiskit-plotting-app

Jupyter Notebook
1
star
45

AskDevoxx

REST service for AskDevoxx
Java
1
star
46

qiskit-trinkets

Quantum computing examples using Qiskit and deployed on Trinket.io
Python
1
star
47

hqe-abhigyan-samuel

1
star
48

hearing-quantum-evolution

1
star
49

alexa-conceptmap

Java
1
star
50

workshop

Resources for GIDS Architecture workshop by James Weaver: Quantum Computing Workshop for Classical Developers
Jupyter Notebook
1
star
51

AlexaAskDevoxx

Alexa client for AskDevoxx inquiry microservice
Java
1
star
52

ReActTextWorldLangChain

ReAct implementation leveraging LangChain and TextWorld
Jupyter Notebook
1
star