• Stars
    star
    1,084
  • Rank 42,307 (Top 0.9 %)
  • Language
    TypeScript
  • License
    MIT License
  • Created over 1 year ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

The TypeScript library for building AI applications.

ModelFusion

Build AI applications, chatbots, and agents with JavaScript and TypeScript.

NPM Version MIT License Docs Discord Created by Lars Grammel

Introduction | Quick Install | Usage | Features | Integrations | Documentation | Examples | Contributing | modelfusion.dev

Note
ModelFusion is in its initial development phase. Until version 1.0 there may be breaking changes, because I am still exploring the API design. Feedback and suggestions are welcome.

Introduction

ModelFusion is a library for building AI apps, chatbots, and agents. It provides abstractions for AI models, vector indices, and tools.

  • Type inference and validation: ModelFusion uses TypeScript and Zod to infer types wherever possible and to validate model responses.
  • Flexibility and control: AI application development can be complex and unique to each project. With ModelFusion, you have complete control over the prompts and model settings, and you can access the raw responses from the models quickly to build what you need.
  • No chains and predefined prompts: Use the concepts provided by JavaScript (variables, functions, etc.) and explicit prompts to build applications you can easily understand and control. Not black magic.
  • Multimodal Support: Beyond just LLMs, ModelFusion encompasses a diverse array of models including text generation, text-to-speech, speech-to-text, and image generation, allowing you to build multifaceted AI applications with ease.
  • Integrated support features: Essential features like logging, retries, throttling, tracing, and error handling are built-in, helping you focus more on building your application.

Quick Install

npm install modelfusion

You need to install zod and a matching version of zod-to-json-schema (peer dependencies):

npm install zod zod-to-json-schema

Or use a template: ModelFusion terminal app starter

Usage Examples

You can provide API keys for the different integrations using environment variables (e.g., OPENAI_API_KEY) or pass them into the model constructors as options.

Generate Text

Generate text using a language model and a prompt. You can stream the text if it is supported by the model. You can use prompt formats to change the prompt format of a model.

generateText

const text = await generateText(
  new OpenAITextGenerationModel({ model: "text-davinci-003" }),
  "Write a short story about a robot learning to love:\n\n"
);

streamText

const textStream = await streamText(
  new OpenAIChatModel({
    model: "gpt-3.5-turbo",
    maxCompletionTokens: 1000,
  }),
  [
    OpenAIChatMessage.system("You are a story writer."),
    OpenAIChatMessage.user("Write a story about a robot learning to love"),
  ]
);

for await (const textFragment of textStream) {
  process.stdout.write(textFragment);
}

Prompt Format

Prompt format lets you use higher level prompt structures (such as instruction or chat prompts) for different models.

const text = await generateText(
  new LlamaCppTextGenerationModel({
    contextWindowSize: 4096, // Llama 2 context window size
    maxCompletionTokens: 1000,
  }).withPromptFormat(Llama2InstructionPromptFormat()),
  {
    system: "You are a story writer.",
    instruction: "Write a short story about a robot learning to love.",
  }
);
const textStream = await streamText(
  new OpenAIChatModel({
    model: "gpt-3.5-turbo",
  }).withPromptFormat(OpenAIChatChatPromptFormat()),
  [
    { system: "You are a celebrated poet." },
    { user: "Write a short story about a robot learning to love." },
    { ai: "Once upon a time, there was a robot who learned to love." },
    { user: "That's a great start!" },
  ]
);

Metadata and original responses

ModelFusion model functions return rich results that include the original response and metadata when you call .asFullResponse() before resolving the promise.

// access the full response and the metadata:
// the response type is specific to the model that's being used
const { output, response, metadata } = await generateText(
  new OpenAITextGenerationModel({
    model: "text-davinci-003",
    maxCompletionTokens: 1000,
    n: 2, // generate 2 completions
  }),
  "Write a short story about a robot learning to love:\n\n"
).asFullResponse();

for (const choice of response.choices) {
  console.log(choice.text);
}

console.log(`Duration: ${metadata.durationInMs}ms`);

Generate JSON

Generate JSON value that matches a schema.

const value = await generateJson(
  new OpenAIChatModel({
    model: "gpt-3.5-turbo",
    temperature: 0,
    maxCompletionTokens: 50,
  }),
  {
    name: "sentiment" as const,
    description: "Write the sentiment analysis",
    schema: z.object({
      sentiment: z
        .enum(["positive", "neutral", "negative"])
        .describe("Sentiment."),
    }),
  },
  OpenAIChatFunctionPrompt.forSchemaCurried([
    OpenAIChatMessage.system(
      "You are a sentiment evaluator. " +
        "Analyze the sentiment of the following product review:"
    ),
    OpenAIChatMessage.user(
      "After I opened the package, I was met by a very unpleasant smell " +
        "that did not disappear even after washing. Never again!"
    ),
  ])
);

Generate JSON or Text

Generate JSON (or text as a fallback) using a prompt and multiple schemas. It either matches one of the schemas or is text reponse.

const { schema, value, text } = await generateJsonOrText(
  new OpenAIChatModel({
    model: "gpt-3.5-turbo",
    maxCompletionTokens: 1000,
  }),
  [
    {
      name: "getCurrentWeather" as const, // mark 'as const' for type inference
      description: "Get the current weather in a given location",
      schema: z.object({
        location: z
          .string()
          .describe("The city and state, e.g. San Francisco, CA"),
        unit: z.enum(["celsius", "fahrenheit"]).optional(),
      }),
    },
    {
      name: "getContactInformation" as const,
      description: "Get the contact information for a given person",
      schema: z.object({
        name: z.string().describe("The name of the person"),
      }),
    },
  ],
  OpenAIChatFunctionPrompt.forSchemasCurried([OpenAIChatMessage.user(query)])
);

Tools

Tools are functions that can be executed by an AI model. They are useful for building chatbots and agents.

Create Tool

A tool is a function with a name, a description, and a schema for the input parameters.

const calculator = new Tool({
  name: "calculator",
  description: "Execute a calculation",

  inputSchema: z.object({
    a: z.number().describe("The first number."),
    b: z.number().describe("The second number."),
    operator: z.enum(["+", "-", "*", "/"]).describe("The operator."),
  }),

  execute: async ({ a, b, operator }) => {
    switch (operator) {
      case "+":
        return a + b;
      case "-":
        return a - b;
      case "*":
        return a * b;
      case "/":
        return a / b;
      default:
        throw new Error(`Unknown operator: ${operator}`);
    }
  },
});

useTool

The model determines the parameters for the tool from the prompt and then executes it.

const { tool, parameters, result } = await useTool(
  new OpenAIChatModel({ model: "gpt-3.5-turbo" }),
  calculator,
  OpenAIChatFunctionPrompt.forToolCurried([
    OpenAIChatMessage.user("What's fourteen times twelve?"),
  ])
);

useToolOrGenerateText

The model determines which tool to use and its parameters from the prompt and then executes it. Text is generated as a fallback.

const { tool, parameters, result, text } = await useToolOrGenerateText(
  new OpenAIChatModel({ model: "gpt-3.5-turbo" }),
  [calculator /* and other tools... */],
  OpenAIChatFunctionPrompt.forToolsCurried([
    OpenAIChatMessage.user("What's fourteen times twelve?"),
  ])
);

Transcribe Speech

Turn speech (audio) into text.

const transcription = await transcribe(
  new OpenAITranscriptionModel({ model: "whisper-1" }),
  {
    type: "mp3",
    data: await fs.promises.readFile("data/test.mp3"),
  }
);

Synthesize Speech

Turn text into speech (audio).

// `speech` is a Buffer with MP3 audio data
const speech = await synthesizeSpeech(
  new ElevenLabsSpeechSynthesisModel({
    voice: "ErXwobaYiN019PkySvjV",
  }),
  "Hello, World!"
);

Generate Image

Generate a base64-encoded image from a prompt.

const image = await generateImage(
  new OpenAIImageGenerationModel({ size: "512x512" }),
  "the wicked witch of the west in the style of early 19th century painting"
);

Embed Text

Create embeddings for text. Embeddings are vectors that represent the meaning of the text.

const embeddings = await embedTexts(
  new OpenAITextEmbeddingModel({ model: "text-embedding-ada-002" }),
  [
    "At first, Nox didn't know what to do with the pup.",
    "He keenly observed and absorbed everything around him, from the birds in the sky to the trees in the forest.",
  ]
);

Tokenize Text

Split text into tokens and reconstruct the text from tokens.

const tokenizer = new TikTokenTokenizer({ model: "gpt-4" });

const text = "At first, Nox didn't know what to do with the pup.";

const tokenCount = await countTokens(tokenizer, text);

const tokens = await tokenizer.tokenize(text);
const tokensAndTokenTexts = await tokenizer.tokenizeWithTexts(text);
const reconstructedText = await tokenizer.detokenize(tokens);

Upserting and Retrieving Text Chunks from Vector Indices

const texts = [
  "A rainbow is an optical phenomenon that can occur under certain meteorological conditions.",
  "It is caused by refraction, internal reflection and dispersion of light in water droplets resulting in a continuous spectrum of light appearing in the sky.",
  // ...
];

const vectorIndex = new MemoryVectorIndex<TextChunk>();
const embeddingModel = new OpenAITextEmbeddingModel({
  model: "text-embedding-ada-002",
});

// update an index - usually done as part of an ingestion process:
await upsertTextChunks({
  vectorIndex,
  embeddingModel,
  chunks: texts.map((text) => ({ text })),
});

// retrieve text chunks from the vector index - usually done at query time:
const { chunks } = await retrieveTextChunks(
  new SimilarTextChunksFromVectorIndexRetriever({
    vectorIndex,
    embeddingModel,
    maxResults: 3,
    similarityThreshold: 0.8,
  }),
  "rainbow and water droplets"
);

Features

Integrations

Model Providers

Text and JSON Generation

OpenAI Cohere Llama.cpp Hugging Face
Generate text โœ… โœ… โœ… โœ…
Stream text โœ… โœ… โœ…
Generate JSON chat models
Generate JSON or Text chat models
Embed text โœ… โœ… โœ… โœ…
Tokenize text full full basic

Image Generation

Speech Transcription

Speech Synthesis

Vector Indices

Observability

Prompt Formats

Use higher level prompts that are mapped into model specific prompt formats.

Prompt Format Instruction Prompt Chat Prompt
OpenAI Chat โœ… โœ…
Llama 2 โœ… โœ…
Alpaca โœ… โŒ
Vicuna โŒ โœ…
Generic Text โœ… โœ…

Documentation

More Examples

Basic Examples

Examples for the individual functions and objects.

Chatbot (Terminal)

Terminal app, chat, llama.cpp

Chatbot (Next.JS)

Next.js app, OpenAI GPT-3.5-turbo, streaming, abort handling

A web chat with an AI assistant, implemented as a Next.js app.

Chat with PDF

terminal app, PDF parsing, in memory vector indices, retrieval augmented generation, hypothetical document embedding

Ask questions about a PDF document and get answers from the document.

Image generator (Next.js)

Next.js app, Stability AI image generation

Create an 19th century painting image for your input.

Voice recording and transcription (Next.js)

Next.js app, OpenAI Whisper

Record audio with push-to-talk and transcribe it using Whisper, implemented as a Next.js app. The app shows a list of the transcriptions.

BabyAGI Agent

terminal app, agent, BabyAGI

TypeScript implementation of the BabyAGI classic and BabyBeeAGI.

Wikipedia Agent

terminal app, ReAct agent, GPT-4, OpenAI functions, tools

Get answers to questions from Wikipedia, e.g. "Who was born first, Einstein or Picasso?"

Middle school math agent

terminal app, agent, tools, GPT-4

Small agent that solves middle school math problems. It uses a calculator tool to solve the problems.

PDF to Tweet

terminal app, PDF parsing, recursive information extraction, in memory vector index, _style example retrieval, OpenAI GPT-4, cost calculation

Extracts information about a topic from a PDF and writes a tweet in your own style about it.

Contributing

Contributing Guide

Read the ModelFusion contributing guide to learn about the development process, how to propose bugfixes and improvements, and how to build and test your changes.

More Repositories

1

next.js

The React Framework
JavaScript
124,125
star
2

hyper

A terminal built on web technologies
TypeScript
43,163
star
3

swr

React Hooks for Data Fetching
TypeScript
30,147
star
4

turborepo

Build system optimized for JavaScriptย and TypeScript, written in Rust
Rust
25,810
star
5

pkg

Package your Node.js project into an executable
JavaScript
24,260
star
6

vercel

Develop. Preview. Ship.
TypeScript
12,555
star
7

commerce

Next.js Commerce
TypeScript
11,004
star
8

satori

Enlightened library to convert HTML and CSS to SVG
TypeScript
10,769
star
9

micro

Asynchronous HTTP microservices
TypeScript
10,525
star
10

ai

Build AI-powered applications with React, Svelte, Vue, and Solid
TypeScript
9,245
star
11

serve

Static file serving and directory listing
TypeScript
9,208
star
12

ncc

Compile a Node.js project into a single file. Supports TypeScript, binary addons, dynamic requires.
JavaScript
9,063
star
13

styled-jsx

Full CSS support for JSX without compromises
JavaScript
7,677
star
14

nextjs-subscription-payments

Clone, deploy, and fully customize a SaaS subscription application with Next.js.
TypeScript
6,135
star
15

ai-chatbot

A full-featured, hackable Next.js AI chatbot built by Vercel
TypeScript
5,969
star
16

platforms

A full-stack Next.js app with multi-tenancy and custom domain support. Built with Next.js App Router and the Vercel Domains API.
TypeScript
5,506
star
17

ms

Tiny millisecond conversion utility
TypeScript
4,912
star
18

og-image

Open Graph Image as a Service - generate cards for Twitter, Facebook, Slack, etc
TypeScript
4,051
star
19

next-learn

Learn Next.js Starter Code
TypeScript
3,578
star
20

release

Generate changelogs with a single command
JavaScript
3,558
star
21

examples

Enjoy our curated collection of examples and solutions. Use these patterns to build your own robust and scalable applications.
TypeScript
3,466
star
22

hazel

Lightweight update server for Electron apps
JavaScript
2,896
star
23

next-plugins

Official Next.js plugins
2,677
star
24

next-app-router-playground

https://app-router.vercel.app/
TypeScript
2,482
star
25

geist-font

2,250
star
26

virtual-event-starter-kit

Open source demo that Next.js developers can clone, deploy, and fully customize for events.
TypeScript
2,131
star
27

async-retry

Retrying made simple, easy and async
JavaScript
1,808
star
28

react-tweet

Embed tweets in your React application.
TypeScript
1,569
star
29

little-date

A friendly formatter to make date ranges small & sweet
TypeScript
1,553
star
30

nft

Node.js dependency tracing utility
JavaScript
1,287
star
31

arg

Simple argument parsing
JavaScript
1,222
star
32

style-guide

Vercel's engineering style guide
JavaScript
1,209
star
33

avatar

๐Ÿ’Ž Beautiful avatars as a microservice
TypeScript
1,119
star
34

nextjs-postgres-nextauth-tailwindcss-template

Admin dashboard template.
TypeScript
998
star
35

next-react-server-components

Demo repository for Next.js + React Server Components
JavaScript
980
star
36

nextjs-postgres-auth-starter

Next.js + Tailwind + Typescript + Drizzle + NextAuth + PostgreSQL starter template.
TypeScript
887
star
37

nextgram

A sample Next.js app showing dynamic routing with modals as a route.
TypeScript
867
star
38

edge-runtime

Developing, testing, and defining the runtime Web APIs for Edge infrastructure.
TypeScript
790
star
39

on-demand-isr

TypeScript
745
star
40

server-components-notes-demo

Experimental demo of React Server Components with Next.js. Deployed serverlessly on Vercel.
TypeScript
732
star
41

micro-dev

The development environment for `micro`
JavaScript
702
star
42

nextjs-portfolio-starter

Easily create a portfolio with Next.js and Markdown.
JavaScript
657
star
43

static-fun

A fun demo for wildcard domains
TypeScript
629
star
44

async-sema

Semaphore using `async` and `await`
TypeScript
624
star
45

hyperpower

Hyper particle effects extension
JavaScript
619
star
46

react-keyframes

Create frame-based animations in React
TypeScript
617
star
47

title

A service for capitalizing your title properly
JavaScript
579
star
48

fetch

Opinionated `fetch` (with retrying and DNS caching) optimized for use with Node.js
JavaScript
568
star
49

serve-handler

The foundation of `serve`
JavaScript
564
star
50

vrs

A serverless virtual reality e-commerce experience powered by Vercel
TypeScript
517
star
51

storage

Vercel Postgres, KV, Blob, and Edge Config
TypeScript
491
star
52

fun

ฦ’un - Local serverless function ฮป development runtime
TypeScript
480
star
53

swr-site

The official website for SWR.
MDX
479
star
54

mongodb-starter

A developer directory built on Next.js and MongoDB Atlas, deployed on Vercel with the Vercel + MongoDB integration.
TypeScript
457
star
55

spr-landing

Serverless Pre-Rendering Landing Page
CSS
432
star
56

pkg-fetch

A utility to fetch or build patched Node binaries used by `pkg` to generate executables. This repo hosts prebuilt binaries in Releases.
TypeScript
428
star
57

hyper-site

The official website for the Hyper terminal
JavaScript
426
star
58

analytics

Privacy-friendly, real-time traffic insights
TypeScript
421
star
59

sveltekit-commerce

SvelteKit Commerce
Svelte
376
star
60

reactions

Next.js Incremental Static Regeneration Demo
JavaScript
308
star
61

email-prompt

CLI email prompt with autocompletion and built-in validation
JavaScript
274
star
62

uid-promise

Creates a cryptographically strong UID
TypeScript
252
star
63

fetch-retry

Wrapper around `fetch` with sensible retrying defaults
JavaScript
221
star
64

git-hooks

No nonsense Git hook management
JavaScript
202
star
65

zsh-theme

Yet another zsh theme
179
star
66

remote-cache

The Vercel Remote Cache SDK
TypeScript
174
star
67

cosmosdb-server

A Cosmos DB server implementation for testing your applications locally.
TypeScript
170
star
68

update-check

Minimalistic update notifications for command line interfaces
JavaScript
159
star
69

test-listen

Quick ephemeral URLs for your tests
JavaScript
153
star
70

react-transition-progress

Show a progress bar while React Transitions run
TypeScript
150
star
71

install-node

Simple one-liner shell script that installs official Node.js binaries
Shell
141
star
72

terraform-provider-vercel

Terraform Vercel Provider
Go
139
star
73

title-site

A website for capitalizing your titles
JavaScript
124
star
74

preview-mode-demo

This demo showcases Next.js' next-gen Static Site Generation (SSG) support.
TypeScript
110
star
75

community

Welcome to the Vercel Community. Discuss feature requests, ask questions, and connect with others in the community.
107
star
76

nextjs-discord-bot

Discord bot for the official Next.js Discord
TypeScript
105
star
77

commerce-framework

TypeScript
98
star
78

beginner-sveltekit

The complete course to start your journey building Svelte applications.
JavaScript
98
star
79

err-sh

Microservice that forwards you to error messages
JavaScript
97
star
80

webpack-asset-relocator-loader

Used in ncc while emitting and relocating any asset references
JavaScript
96
star
81

hyperyellow

Example theme for hyperterm
JavaScript
87
star
82

schemas

All schemas used for validation that are shared between our projects
JavaScript
80
star
83

opentelemetry-collector-dev-setup

Shell
72
star
84

nuxt3-kitchen-sink

An example template showing all Nuxt 3 features on Vercel.
Vue
65
star
85

next-codemod

codemod transformations to help upgrade Next.js codebases
JavaScript
63
star
86

speed-insights

Vercel Speed Insights package
TypeScript
56
star
87

wait-for

Small utility that waits for a file to exist and optionally have some permissions set.
C
46
star
88

async-listen

Promisify server.listen for your HTTP/HTTPS/TCP server.
TypeScript
45
star
89

tracing-js

An implementation of Opentracing API for honeycomb.io
TypeScript
43
star
90

example-integration

TypeScript
42
star
91

dns-cached-resolve

Caching DNS resolver
TypeScript
41
star
92

fetch-cached-dns

A decorator on top of `fetch` that caches the DNS query
JavaScript
36
star
93

ai-sdk-rag-starter

TypeScript
31
star
94

resolve-node

API endpoint to resolve an arbitrary Node.js version with semver support
JavaScript
30
star
95

remark-capitalize

Transform all markdown titles with title.sh
JavaScript
29
star
96

cra-to-next

An example of migrating Create React App to Next.js.
JavaScript
27
star
97

otel

OTEL tracing for Vercel
TypeScript
24
star
98

rcurl

`curl --resolve` helper script
Shell
23
star
99

stripe-integration

A Vercel deploy integration to automatically set up your Stripe API keys and webhook secrets.
TypeScript
22
star
100

release-auth

Handles the authentication for `release`
JavaScript
21
star