tip20

Stop tipping AIs who won't spend it wisely. Extract any kind of data using prefix completion.

Usage

npm install tip20

Bun is still the preferred runtime (bun install tip20).

tip20 includes both a streaming and a regular async function for output.

shortCircuit is a boolean that will stop the streaming once enough is found in the response. This should reduce response times by about 90%.

import fs from "fs";
import { tip20streaming } from "tip20";

const tip20stream = await tip20streaming(
  "typescript",
  "Some mixed response",
  "claude-3-haiku-20240307",
  true // short ciruit.
);

for await (const token of tip20stream) {
  console.log("Got packet ", token);
}

The tests folder has more examples.

Tip20 is named after the (half serious) joke that tipping LLMs results in better performance. Using prefix completion (on supported LLMs - currently for the package that's Haiku and Sonnet) you can get streaming results of extracted data from almost any input.

from @voooooogel

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. TODOs.md is a good place to start.