Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WASM/JS bindings #50

Open
wants to merge 230 commits into
base: main
Choose a base branch
from
Open

Add WASM/JS bindings #50

wants to merge 230 commits into from

Commits on Feb 8, 2023

  1. refactor

    eisber committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    6c49a41 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2023

  1. ported loading to Rust

    eisber committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    09dada4 View commit details
    Browse the repository at this point in the history
  2. fixed?

    eisber committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    4c6afcb View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    502a6ec View commit details
    Browse the repository at this point in the history
  4. moving registry into rust

    eisber committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    c685d99 View commit details
    Browse the repository at this point in the history
  5. return ref

    jackgerrits committed Feb 9, 2023
    Configuration menu
    Copy the full SHA
    6dcfa91 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2023

  1. split into multiple libs

    eisber committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    25306a5 View commit details
    Browse the repository at this point in the history
  2. fixed core

    eisber committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    ba69bd9 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c98f824 View commit details
    Browse the repository at this point in the history
  4. java bindings...

    eisber committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    f248694 View commit details
    Browse the repository at this point in the history
  5. more TODO

    eisber committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    2f371f6 View commit details
    Browse the repository at this point in the history

Commits on Feb 11, 2023

  1. Configuration menu
    Copy the full SHA
    1d3f707 View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2023

  1. Add wasm-bindgen, inline ranks

    dqbd committed Feb 19, 2023
    Configuration menu
    Copy the full SHA
    95aef95 View commit details
    Browse the repository at this point in the history
  2. v0.2.1

    dqbd committed Feb 19, 2023
    Configuration menu
    Copy the full SHA
    799df7f View commit details
    Browse the repository at this point in the history
  3. Update README.md, polish API

    dqbd committed Feb 19, 2023
    Configuration menu
    Copy the full SHA
    5a32b88 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2023

  1. improve error handling

    eisber committed Feb 22, 2023
    Configuration menu
    Copy the full SHA
    97d8dea View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2023

  1. chore: add README

    dqbd committed Feb 23, 2023
    Configuration menu
    Copy the full SHA
    3139d04 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    02d132e View commit details
    Browse the repository at this point in the history
  3. Merge pull request #2 from dqbd/extendability

    feat: add option to extend special tokens and to provide custom ranks
    dqbd authored Feb 23, 2023
    Configuration menu
    Copy the full SHA
    46668df View commit details
    Browse the repository at this point in the history
  4. Bump version, update README

    dqbd committed Feb 23, 2023
    Configuration menu
    Copy the full SHA
    3a39f24 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    d0dc32b View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    90ee9f6 View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2023

  1. Configuration menu
    Copy the full SHA
    f7fe717 View commit details
    Browse the repository at this point in the history
  2. Validate the values properly

    dqbd committed Feb 24, 2023
    Configuration menu
    Copy the full SHA
    d38c936 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    53628a4 View commit details
    Browse the repository at this point in the history
  4. Bump version to 0.4.0

    dqbd committed Feb 24, 2023
    Configuration menu
    Copy the full SHA
    87603e9 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2023

  1. Merge pull request #1 from eisber/error_handling

    Improve error handling in JNI functions
    eisber authored Feb 27, 2023
    Configuration menu
    Copy the full SHA
    42548d8 View commit details
    Browse the repository at this point in the history
  2. moved config into json

    eisber committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    e3ab3f6 View commit details
    Browse the repository at this point in the history
  3. add github action

    eisber committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    22584d4 View commit details
    Browse the repository at this point in the history
  4. add jar build

    eisber committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    42efde4 View commit details
    Browse the repository at this point in the history
  5. fix rust build

    eisber committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    484f15a View commit details
    Browse the repository at this point in the history
  6. build jar

    eisber committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    6c45bf6 View commit details
    Browse the repository at this point in the history
  7. fix path

    add matrix java test
    eisber committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    b883d0c View commit details
    Browse the repository at this point in the history
  8. fix java

    eisber committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    98070f1 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    3b39f6b View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2023

  1. cleanup

    eisber committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    4e70da0 View commit details
    Browse the repository at this point in the history
  2. update groupid

    eisber committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    381d32b View commit details
    Browse the repository at this point in the history
  3. remove comments

    eisber committed Feb 28, 2023
    Configuration menu
    Copy the full SHA
    f1560bd View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2023

  1. Move to separate js folder

    dqbd committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    01d4f9e View commit details
    Browse the repository at this point in the history
  2. Merge branch 'eisber/main'

    dqbd committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    4d8f9af View commit details
    Browse the repository at this point in the history
  3. Make sure JS builds

    dqbd committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    ef77b1a View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2023

  1. Attempt to fix sdist

    dqbd committed Mar 2, 2023
    Configuration menu
    Copy the full SHA
    b04f0cf View commit details
    Browse the repository at this point in the history
  2. Match sdist

    dqbd committed Mar 2, 2023
    Configuration menu
    Copy the full SHA
    bbcb591 View commit details
    Browse the repository at this point in the history
  3. Remove the _js suffix

    dqbd committed Mar 2, 2023
    Configuration menu
    Copy the full SHA
    d1c4af2 View commit details
    Browse the repository at this point in the history
  4. Update to newer build wheels

    dqbd committed Mar 2, 2023
    Configuration menu
    Copy the full SHA
    33207e6 View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2023

  1. Fix wrong result for None

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    2370284 View commit details
    Browse the repository at this point in the history
  2. Add CI step to build and test

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    98ac953 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d989c22 View commit details
    Browse the repository at this point in the history
  4. Fix Python CI

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    01bf979 View commit details
    Browse the repository at this point in the history
  5. Replace yum with apt

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    82cd413 View commit details
    Browse the repository at this point in the history
  6. Try yum once again

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    7db26cb View commit details
    Browse the repository at this point in the history
  7. debug ci

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    08403a1 View commit details
    Browse the repository at this point in the history
  8. Try CI again

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    12805e3 View commit details
    Browse the repository at this point in the history
  9. Fix CI again

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    ae30c13 View commit details
    Browse the repository at this point in the history
  10. invalid -y command

    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    dabb296 View commit details
    Browse the repository at this point in the history
  11. Revert "debug ci"

    This reverts commit 08403a1.
    dqbd committed Mar 5, 2023
    Configuration menu
    Copy the full SHA
    8e40682 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    4866cf9 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2023

  1. Configuration menu
    Copy the full SHA
    33bb13d View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2023

  1. Flatten the structure

    dqbd committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    03cdbb3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7c8cd78 View commit details
    Browse the repository at this point in the history
  3. Rename TiktokenEmbedding to TiktokenEncoding

    Fixes Rename TiktokenEmbedding → TiktokenEncoding? #4
    dqbd committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    fcb52cf View commit details
    Browse the repository at this point in the history
  4. Reverse order of default

    dqbd committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    945d4f2 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #12 from dqbd/async-init-bundler

    Add custom initialisation, bundler changes
    dqbd authored Mar 8, 2023
    Configuration menu
    Copy the full SHA
    9ea421b View commit details
    Browse the repository at this point in the history
  6. Update README.md

    dqbd committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    680fbc5 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    7c75b04 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    c1d11fb View commit details
    Browse the repository at this point in the history
  9. Add README.md

    dqbd committed Mar 8, 2023
    Configuration menu
    Copy the full SHA
    268bc5c View commit details
    Browse the repository at this point in the history
  10. Merge pull request #13 from dqbd/gpt-3.5-turbo

    Add gpt-3.5-turbo support in types and matchers
    dqbd authored Mar 8, 2023
    Configuration menu
    Copy the full SHA
    9f9ad0d View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2023

  1. Add caveats for CFW

    dqbd committed Mar 10, 2023
    Configuration menu
    Copy the full SHA
    15dd0f2 View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2023

  1. Configuration menu
    Copy the full SHA
    2c7e0e4 View commit details
    Browse the repository at this point in the history
  2. Fix README.md

    dqbd committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    586d205 View commit details
    Browse the repository at this point in the history
  3. Cleanup

    dqbd committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    6d8c1dc View commit details
    Browse the repository at this point in the history
  4. Expose loading script

    dqbd committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    64178e5 View commit details
    Browse the repository at this point in the history
  5. Add polyfill for Buffer.from

    dqbd committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    17dd0ba View commit details
    Browse the repository at this point in the history
  6. Merge pull request #15 from dqbd/lite-build

    Create a lite build which defers loading of weights to consumers
    dqbd authored Mar 11, 2023
    Configuration menu
    Copy the full SHA
    4f6745f View commit details
    Browse the repository at this point in the history
  7. Fix exports for CJS (Node ESM)

    dqbd committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    e6f0726 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    1ee26c3 View commit details
    Browse the repository at this point in the history
  9. Add disclaimer for CFW

    dqbd committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    8984ea7 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2023

  1. Add support for GPT-4

    dqbd committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    9c8caec View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2023

  1. Configuration menu
    Copy the full SHA
    103d010 View commit details
    Browse the repository at this point in the history
  2. Bump to 1.0.0-alpha.5

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    79ac036 View commit details
    Browse the repository at this point in the history
  3. Fix lite crash, add README.md

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    f748754 View commit details
    Browse the repository at this point in the history
  4. Update README.md

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    bfe3817 View commit details
    Browse the repository at this point in the history
  5. Bump to 1.0.0-alpha.6

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    9847f4d View commit details
    Browse the repository at this point in the history
  6. Add JSON importable modules

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    c01af19 View commit details
    Browse the repository at this point in the history
  7. Bump to 1.0.0-alpha.7

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    23bb57d View commit details
    Browse the repository at this point in the history
  8. Bump to 1.0.0-alpha.8

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    a3baa6c View commit details
    Browse the repository at this point in the history
  9. Compress ranks

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    6ac1a1a View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    7efba72 View commit details
    Browse the repository at this point in the history
  11. Bump to 1.0.0-alpha.10

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    4d4b921 View commit details
    Browse the repository at this point in the history
  12. Update README.md

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    efe3728 View commit details
    Browse the repository at this point in the history
  13. Bump to 1.0.0

    dqbd committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    8476eca View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2023

  1. Configuration menu
    Copy the full SHA
    e1c4313 View commit details
    Browse the repository at this point in the history
  2. Bump to 1.0.1

    dqbd committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    c4dfaad View commit details
    Browse the repository at this point in the history
  3. Clarifies usage

    Fixes Crash in Cloudflare Workers / Vercel Edge Runtime #20
    dqbd committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    68efa86 View commit details
    Browse the repository at this point in the history
  4. Bump to 1.0.2

    dqbd committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    481fb45 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2023

  1. Configuration menu
    Copy the full SHA
    6823b8e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b9a03a7 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2023

  1. Configuration menu
    Copy the full SHA
    6bba615 View commit details
    Browse the repository at this point in the history
  2. Update lib.rs

    christophwitzko authored Apr 4, 2023
    Configuration menu
    Copy the full SHA
    10e17c3 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #25 from christophwitzko/patch-1

    Add gpt-4 to model_to_encoding.json
    dqbd authored Apr 4, 2023
    Configuration menu
    Copy the full SHA
    a3c88a5 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2023

  1. Configuration menu
    Copy the full SHA
    ee98f78 View commit details
    Browse the repository at this point in the history
  2. Bump to 1.0.3

    dqbd committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    7261f28 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2023

  1. Configuration menu
    Copy the full SHA
    ff230c8 View commit details
    Browse the repository at this point in the history
  2. Bump to 1.0.4

    dqbd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    83c2a85 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2023

  1. Configuration menu
    Copy the full SHA
    d40517f View commit details
    Browse the repository at this point in the history
  2. Bump to 1.0.5

    dqbd committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    496168c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0c0cf00 View commit details
    Browse the repository at this point in the history
  4. Bump to 1.0.6

    dqbd committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    26bf591 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2023

  1. Configuration menu
    Copy the full SHA
    33405e9 View commit details
    Browse the repository at this point in the history
  2. Publish 1.0.7-alpha.0 build

    dqbd committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    a54dd57 View commit details
    Browse the repository at this point in the history
  3. Bump to 1.0.7

    dqbd committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    ea11d75 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2023

  1. Remove Java bindings

    dqbd committed May 8, 2023
    Configuration menu
    Copy the full SHA
    754a378 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fe721cd View commit details
    Browse the repository at this point in the history
  3. Replace README.md for clarity

    dqbd committed May 8, 2023
    Configuration menu
    Copy the full SHA
    9390c03 View commit details
    Browse the repository at this point in the history
  4. Remove duplicate README.md

    dqbd committed May 8, 2023
    Configuration menu
    Copy the full SHA
    b8e7817 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2023

  1. Add weak refs

    dqbd committed May 10, 2023
    Configuration menu
    Copy the full SHA
    83de511 View commit details
    Browse the repository at this point in the history
  2. Add changesets folder

    dqbd committed May 10, 2023
    Configuration menu
    Copy the full SHA
    243c061 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2023

  1. Configuration menu
    Copy the full SHA
    67d21c3 View commit details
    Browse the repository at this point in the history
  2. Add JS port

    dqbd committed May 12, 2023
    Configuration menu
    Copy the full SHA
    150cf05 View commit details
    Browse the repository at this point in the history
  3. Rename WASM workflows

    dqbd committed May 12, 2023
    Configuration menu
    Copy the full SHA
    fdf6121 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2023

  1. Generate ranks

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    6b13f93 View commit details
    Browse the repository at this point in the history
  2. Monorepo changes

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    7e5ab35 View commit details
    Browse the repository at this point in the history
  3. Fix actions

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    1bf31d3 View commit details
    Browse the repository at this point in the history
  4. More descriptive name

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    1845548 View commit details
    Browse the repository at this point in the history
  5. Add CDN hosting of ranks

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    3debe21 View commit details
    Browse the repository at this point in the history
  6. Fix sdist

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    990d892 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    81bd689 View commit details
    Browse the repository at this point in the history
  8. Rename description

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    f245e69 View commit details
    Browse the repository at this point in the history
  9. Add publish CI step

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    6b00942 View commit details
    Browse the repository at this point in the history
  10. Normalise workflows

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    7c6a304 View commit details
    Browse the repository at this point in the history
  11. Rename package once again

    dqbd committed May 13, 2023
    Configuration menu
    Copy the full SHA
    975ed0f View commit details
    Browse the repository at this point in the history

Commits on May 14, 2023

  1. fix: typo

    rpidanny authored May 14, 2023
    Configuration menu
    Copy the full SHA
    14f108c View commit details
    Browse the repository at this point in the history
  2. Split into smaller files

    dqbd committed May 14, 2023
    Configuration menu
    Copy the full SHA
    3240a8e View commit details
    Browse the repository at this point in the history
  3. Merge pull request #40 from rpidanny/fix-typo

    fix: typo
    dqbd authored May 14, 2023
    Configuration menu
    Copy the full SHA
    dd6a6ef View commit details
    Browse the repository at this point in the history

Commits on May 15, 2023

  1. Configuration menu
    Copy the full SHA
    2d18fcc View commit details
    Browse the repository at this point in the history
  2. Bump js-tiktoken to 1.0.1

    dqbd committed May 15, 2023
    Configuration menu
    Copy the full SHA
    1da0b9d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    621ae35 View commit details
    Browse the repository at this point in the history
  4. Add disallowed tokens

    dqbd committed May 15, 2023
    Configuration menu
    Copy the full SHA
    de929d4 View commit details
    Browse the repository at this point in the history
  5. Version Packages

    github-actions[bot] committed May 15, 2023
    Configuration menu
    Copy the full SHA
    dba22a9 View commit details
    Browse the repository at this point in the history
  6. Merge pull request #42 from dqbd/changeset-release/main

    Version Packages
    dqbd authored May 15, 2023
    Configuration menu
    Copy the full SHA
    97fe465 View commit details
    Browse the repository at this point in the history
  7. Expose ranks

    dqbd committed May 15, 2023
    Configuration menu
    Copy the full SHA
    265e01d View commit details
    Browse the repository at this point in the history
  8. Bump to 1.0.4

    dqbd committed May 15, 2023
    Configuration menu
    Copy the full SHA
    7fbd899 View commit details
    Browse the repository at this point in the history
  9. Bump js-tiktoken to 1.0.5

    Use require instead for exports, add lite entrypoint
    dqbd committed May 15, 2023
    Configuration menu
    Copy the full SHA
    4d3f97f View commit details
    Browse the repository at this point in the history

Commits on May 17, 2023

  1. Bump to 1.0.6

    dqbd committed May 17, 2023
    Configuration menu
    Copy the full SHA
    1fcb052 View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2023

  1. Configuration menu
    Copy the full SHA
    3f2c59c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c4a6180 View commit details
    Browse the repository at this point in the history
  3. Update README.md

    dqbd committed Jun 2, 2023
    Configuration menu
    Copy the full SHA
    c7faff0 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2023

  1. Add new models

    - gpt-3.5-turbo-0613
    - gpt-3.5-turbo-16k
    - gpt-3.5-turbo-16k-0613
    - gpt-4-0613
    - gpt-4-32k-0613
    rodumani committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    ebfc814 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2023

  1. Configuration menu
    Copy the full SHA
    a5fb1f4 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #51 from rodumani/main

    Add new models
    dqbd authored Jun 15, 2023
    Configuration menu
    Copy the full SHA
    d1f8f9f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    844343f View commit details
    Browse the repository at this point in the history
  4. Add changeset

    dqbd committed Jun 15, 2023
    Configuration menu
    Copy the full SHA
    baca5b1 View commit details
    Browse the repository at this point in the history
  5. Version Packages

    github-actions[bot] committed Jun 15, 2023
    Configuration menu
    Copy the full SHA
    a25079d View commit details
    Browse the repository at this point in the history
  6. Merge pull request #52 from dqbd/changeset-release/main

    Version Packages
    dqbd authored Jun 15, 2023
    Configuration menu
    Copy the full SHA
    0d7e051 View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2023

  1. feat: Add missing models

    kdwkr authored Jun 16, 2023
    Configuration menu
    Copy the full SHA
    5a7be3c View commit details
    Browse the repository at this point in the history
  2. Update lib.rs

    kdwkr authored Jun 16, 2023
    Configuration menu
    Copy the full SHA
    1bc9ada View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2023

  1. Merge pull request #53 from kdwkr/patch-1

    feat: Add missing models
    dqbd authored Jun 22, 2023
    Configuration menu
    Copy the full SHA
    b75f172 View commit details
    Browse the repository at this point in the history
  2. Bump tiktoken to 1.0.9

    dqbd committed Jun 22, 2023
    Configuration menu
    Copy the full SHA
    3310bfe View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    15b8a8b View commit details
    Browse the repository at this point in the history
  4. Use NODE_AUTH_TOKEN

    dqbd committed Jun 22, 2023
    Configuration menu
    Copy the full SHA
    71db434 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2023

  1. Remove prefix from folder

    Closes When running 1.0.9 there is a Missing tiktoken_bg.wasm error #56
    dqbd committed Jun 24, 2023
    Configuration menu
    Copy the full SHA
    8e776d3 View commit details
    Browse the repository at this point in the history
  2. Add changeset

    dqbd committed Jun 24, 2023
    Configuration menu
    Copy the full SHA
    a691152 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #57 from dqbd/prefix-fix

    Remove prefix from folder
    dqbd authored Jun 24, 2023
    Configuration menu
    Copy the full SHA
    b1260d9 View commit details
    Browse the repository at this point in the history
  4. Version Packages

    github-actions[bot] committed Jun 24, 2023
    Configuration menu
    Copy the full SHA
    41bb68a View commit details
    Browse the repository at this point in the history
  5. Merge pull request #58 from dqbd/changeset-release/main

    Version Packages
    dqbd authored Jun 24, 2023
    Configuration menu
    Copy the full SHA
    c86963f View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2023

  1. Configuration menu
    Copy the full SHA
    82d44e1 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. Merge pull request #66 from nikwen/electron-readme

    Add Electron setup instructions to README
    dqbd authored Aug 7, 2023
    Configuration menu
    Copy the full SHA
    072dd12 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. feat: add gpt3.5-turbo-instruct model

    Signed-off-by: Prince Mendiratta <[email protected]>
    Prince-Mendiratta committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    db9a804 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2023

  1. Configuration menu
    Copy the full SHA
    4a91a81 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. Add new encodings

    dqbd committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    4cda379 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    32882f7 View commit details
    Browse the repository at this point in the history
  3. Update examples

    dqbd committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    10302a2 View commit details
    Browse the repository at this point in the history
  4. Add new changeset

    dqbd committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    bd3a360 View commit details
    Browse the repository at this point in the history
  5. Version Packages

    github-actions[bot] committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    20d610f View commit details
    Browse the repository at this point in the history
  6. Merge pull request #81 from dqbd/changeset-release/main

    Version Packages
    dqbd authored Nov 15, 2023
    Configuration menu
    Copy the full SHA
    74c147e View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2024

  1. Add new models

    risu729 committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    ff9c637 View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2024

  1. Merge pull request #86 from risu729/main

    Add new models
    dqbd authored Jan 28, 2024
    Configuration menu
    Copy the full SHA
    e1b2472 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #71 from Prince-Mendiratta/main

    Add gpt3.5-turbo-instruct model
    dqbd authored Jan 28, 2024
    Configuration menu
    Copy the full SHA
    5414d83 View commit details
    Browse the repository at this point in the history
  3. Fix build

    dqbd committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    221a30e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    43e99b4 View commit details
    Browse the repository at this point in the history
  5. Version Packages

    github-actions[bot] committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    708f77c View commit details
    Browse the repository at this point in the history
  6. Merge pull request #87 from dqbd/changeset-release/main

    Version Packages
    dqbd authored Jan 28, 2024
    Configuration menu
    Copy the full SHA
    a020c6a View commit details
    Browse the repository at this point in the history
  7. Fix invalid model

    dqbd committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    de6d1c7 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    4491997 View commit details
    Browse the repository at this point in the history
  9. Fix typings for wasm

    dqbd committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    e3fcb12 View commit details
    Browse the repository at this point in the history
  10. Fix typo

    dqbd committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    f39912b View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    d54f04c View commit details
    Browse the repository at this point in the history
  12. Version Packages

    github-actions[bot] committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    f6b2819 View commit details
    Browse the repository at this point in the history
  13. Merge pull request #89 from dqbd/changeset-release/main

    Version Packages
    dqbd authored Jan 28, 2024
    Configuration menu
    Copy the full SHA
    ffa8283 View commit details
    Browse the repository at this point in the history
  14. Update development README.md

    Closes #54
    dqbd committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    110eef4 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. Add GPT-4 Turbo

    akynau committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    3768231 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. Merge pull request #97 from akynau/add-gpt-4-turbo

    Add GPT-4 Turbo
    dqbd authored Apr 12, 2024
    Configuration menu
    Copy the full SHA
    1d8a762 View commit details
    Browse the repository at this point in the history
  2. Add changelog

    dqbd committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    f386b28 View commit details
    Browse the repository at this point in the history
  3. Version Packages

    github-actions[bot] committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    f6165e7 View commit details
    Browse the repository at this point in the history
  4. Merge pull request #98 from dqbd/changeset-release/main

    Version Packages
    dqbd authored Apr 12, 2024
    Configuration menu
    Copy the full SHA
    045f0e4 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. Configuration menu
    Copy the full SHA
    f2e1ac2 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #104 from neonredwood/support-gpt4-omni

    Add support for GPT-4-O, "Omni" model
    dqbd authored May 13, 2024
    Configuration menu
    Copy the full SHA
    3c118a6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ed9cc4f View commit details
    Browse the repository at this point in the history
  4. Add entry to tsup

    dqbd committed May 13, 2024
    Configuration menu
    Copy the full SHA
    e228b47 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #106 from dqbd/dqbd/o200k_base

    fix(js): specify exports in js-tiktoken
    dqbd authored May 13, 2024
    Configuration menu
    Copy the full SHA
    5de6be5 View commit details
    Browse the repository at this point in the history
  6. Add changeset

    dqbd committed May 13, 2024
    Configuration menu
    Copy the full SHA
    791cf62 View commit details
    Browse the repository at this point in the history
  7. Version Packages

    github-actions[bot] committed May 13, 2024
    Configuration menu
    Copy the full SHA
    50f70dd View commit details
    Browse the repository at this point in the history
  8. Merge pull request #107 from dqbd/changeset-release/main

    Version Packages
    dqbd authored May 13, 2024
    Configuration menu
    Copy the full SHA
    a7cce99 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. Configuration menu
    Copy the full SHA
    28ae6b3 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Update js/src/core.ts

    Co-authored-by: Luigi Pederzani <[email protected]>
    mbarretol and Pederzh authored Jul 22, 2024
    Configuration menu
    Copy the full SHA
    25bb43a View commit details
    Browse the repository at this point in the history
  2. Update tiktoken/model_to_encoding.json

    Co-authored-by: Luigi Pederzani <[email protected]>
    mbarretol and Pederzh authored Jul 22, 2024
    Configuration menu
    Copy the full SHA
    396a8d0 View commit details
    Browse the repository at this point in the history
  3. Update wasm/src/lib.rs

    Co-authored-by: Luigi Pederzani <[email protected]>
    mbarretol and Pederzh authored Jul 22, 2024
    Configuration menu
    Copy the full SHA
    be5db34 View commit details
    Browse the repository at this point in the history
  4. Update wasm/src/lib.rs

    Co-authored-by: Luigi Pederzani <[email protected]>
    mbarretol and Pederzh authored Jul 22, 2024
    Configuration menu
    Copy the full SHA
    f7e3df1 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2024

  1. Update js/src/core.ts

    Co-authored-by: huytool157 <[email protected]>
    mbarretol and huytool157 authored Aug 10, 2024
    Configuration menu
    Copy the full SHA
    de6a052 View commit details
    Browse the repository at this point in the history
  2. Update tiktoken/model_to_encoding.json

    Co-authored-by: huytool157 <[email protected]>
    mbarretol and huytool157 authored Aug 10, 2024
    Configuration menu
    Copy the full SHA
    fb1ca7b View commit details
    Browse the repository at this point in the history
  3. Update wasm/src/lib.rs

    Co-authored-by: huytool157 <[email protected]>
    mbarretol and huytool157 authored Aug 10, 2024
    Configuration menu
    Copy the full SHA
    26ec6cc View commit details
    Browse the repository at this point in the history
  4. Update wasm/src/lib.rs

    Co-authored-by: huytool157 <[email protected]>
    mbarretol and huytool157 authored Aug 10, 2024
    Configuration menu
    Copy the full SHA
    a32f0a4 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. Fix unreachable code

    dqbd committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    bba4ed6 View commit details
    Browse the repository at this point in the history
  2. Add changeset

    dqbd committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    4796f27 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9d385cb View commit details
    Browse the repository at this point in the history
  4. Add changeset

    dqbd committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    831a7e8 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #112 from mbarretol/add-gpt4o-mini

    Add support for gpt-4o, gpt-4o-mini, text-embedding-3-small and text-embedding-3-large
    dqbd authored Aug 15, 2024
    Configuration menu
    Copy the full SHA
    e77e339 View commit details
    Browse the repository at this point in the history
  6. Version Packages

    github-actions[bot] committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    ed2e785 View commit details
    Browse the repository at this point in the history
  7. Merge pull request #116 from dqbd/changeset-release/main

    Version Packages
    dqbd authored Aug 15, 2024
    Configuration menu
    Copy the full SHA
    60ff9bb View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    a825d5f View commit details
    Browse the repository at this point in the history
  9. Changeset

    dqbd committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    74e7870 View commit details
    Browse the repository at this point in the history
  10. Merge pull request #117 from dqbd/dqbd/js-tiktoken-missing-files

    fix(js): add missing rank files back to js-tiktoken
    dqbd authored Aug 15, 2024
    Configuration menu
    Copy the full SHA
    25f0656 View commit details
    Browse the repository at this point in the history
  11. Version Packages

    github-actions[bot] committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    386acb8 View commit details
    Browse the repository at this point in the history
  12. Merge pull request #118 from dqbd/changeset-release/main

    Version Packages
    dqbd authored Aug 15, 2024
    Configuration menu
    Copy the full SHA
    f8aa455 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. Add missing o1 models

    dqbd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    0ba7dbb View commit details
    Browse the repository at this point in the history
  2. Changeset

    dqbd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    53bd14a View commit details
    Browse the repository at this point in the history
  3. Merge pull request #121 from dqbd/dqbd/o1

    feat: add missing o1 models
    dqbd authored Oct 3, 2024
    Configuration menu
    Copy the full SHA
    503da21 View commit details
    Browse the repository at this point in the history
  4. Version Packages

    github-actions[bot] committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    7e6cddc View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. Configuration menu
    Copy the full SHA
    b742a86 View commit details
    Browse the repository at this point in the history