hckrnws

Understanding GPT tokenizers

by simonw

mike_hearn
2y
s-macke
2y
josecyc
2y
sillysaurusx
2y
thyselius
2y
hammeiam
2y
og_kalu
2y
mike_hearn
2y
og_kalu
2y
kordlessagain
2y
base698
2y
kordlessagain
2y
belladoreai
2y
mike_hearn
2y
belladoreai
2y
mike_hearn
2y
belladoreai
2y
hammeiam
2y
belladoreai
2y
belladoreai
2y

Comment was deleted :(

weinzierl
2y
resters
2y
mike_hearn
2y
anotherpaulg
2y
Der_Einzige
2y
simonw
2y
weinzierl
2y
jcims
2y
PeterisP
2y
emmelaich
2y
f137
2y
ftxbro
2y
Der_Einzige
2y
nsinreal
2y
spywaregorilla
2y
ftxbro
2y
spywaregorilla
2y
whimsicalism
2y
yorwba
2y
sp332
2y
jcims
2y
zecg
2y
Der_Einzige
2y
chaxor
2y
bpiche
2y
justanotheratom
2y
LudwigNagasena
2y
ajuc
2y
gwern
2y
pmoriarty
2y
gwern
2y
zarzavat
2y
telotortium
2y
gwern
2y
burnished
2y
Der_Einzige
2y
TechBro8615
2y
jamesk_au
2y
wolfgang42
2y
belladoreai
2y
Majromax
2y
simonw
2y
kgeist
2y
Ambix
2y
szopa
2y
Ambix
2y
jameshart
2y
api
2y
PeterisP
2y
sebzim4500
2y
sanxiyn
2y
belladoreai
2y
sandinmyjoints
2y
seanhunter
2y
simonw
2y
seanhunter
2y
seanhunter
2y
ftxbro
2y
simonw
2y
Ambix
2y
throwaway2016a
2y
nighthawk454
2y
CuriousSkeptic
2y
quickthrower2
2y
nighthawk454
2y
nighthawk454
2y
binarymax
2y
RC_ITR
2y
z3c0
2y
Ambix
2y

Comment was deleted :(

samstave
2y
plewd
2y
washadjeffmad
2y
samstave
2y
hsjqllzlfkf
2y
ftxbro
2y
kenjackson
2y
GeneralMayhem
2y
SkyPuncher
2y
thaumasiotes
2y
wolfgang42
2y
doormatt
2y
hsjqllzlfkf
2y
mike_hearn
2y
hsjqllzlfkf
2y
SkyPuncher
2y
wolfgang42
2y
washadjeffmad
2y
whimsicalism
2y
SkyPuncher
2y
simonw
2y
llm_nerd
2y

Comment was deleted :(

Der_Einzige
2y
isoprophlex
2y
h_mirin
2y
YetAnotherNick
2y
bluepoint
2y
simonw
2y
jiggawatts
2y
sebzim4500
2y
jiggawatts
2y
PeterisP
2y
PeterisP
2y
minimaxir
2y
ywmario
2y
Ambix
2y
pietroppeter
2y
tabtab
2y
jjtheblunt
2y
lee101
2y
buggythebug
2y

Crafted by Rajat

Source Code