hckrnws

Someone needs to do a TypeScript compiler plugin to add multidimensional array slicing and operator overloading to the language, so these libraries can actually work the way PyTorch does.

I know operator overloading is controversial, but the way it allows automatic differentiation to work transparently through regular arithmetic expressions is very helpful. Without it these libraries will never feel like PyTorch.

JavaScript is so much faster than Python, and the package management experience is so much better, that I really think a JS library would have a chance to take market share from PyTorch in the long term if it had those features.

What's wrong with creating a function that does those things? It would be less surprising to people new to the library, would be self-documenting by having a name and an easily inspected declaration with named arguments, and it would be idiomatic JS. You could also have variants that are purely functional and return a new value or ones that mutate in place that you could use depending on your needs.

What's wrong is it obscures simple math expressions behind tons of dots and parentheses. The thing is that the core of deep learning algorithms is usually very simple math. It's useful to be able to translate that math directly from research papers into straightforward expressions that mirror the structure in the paper like a = b / c + d * e rather than something less similar like a = b.divide(c).add(d.multiply(e)).

Depending on what you learned first, dots and parentheses are a lot simpler to understand than math expressions.

You could probably build tagged template literal like:

a = e`${b} / ${c}`

Not ideal, but much better and without magic pre processors

In python when I try a new gpu accelerated array library, to write norm_arr = arr / lib.sqrt( lib.sum( arr*2, axis=-1, keepdims=True) , I have to read documentation for sum to see whther they use axis or dim in sum.

In javascript, to write the same thing I need to read documentation for a sum function, a broadcasted division function, a multiply function. I can probably assume that the sqrt function behaves and is named as I would expect

If each of those operators were implemented as functions then you'd have different names for different implementations in order to avoid confusion over what type of division or multiplication they were performing. It's more verbose but that's a good thing since it prevents you from making incorrect assumptions about what's going to happen when you do a * b.

Why can you make assumptions about operation overloads but not functions?

Because there is nothing to make assumptions about. In the example code, both multiplication and division have a scalar on one side, there's no possible ambiguity of behavior. But there is the eternal question of terminology: do you specify dimensions by "axis" or "dim" and does your API actually use both terms in different places?

(that's what I think the GP meant, anyway).

Agreed. My suggestion is for Bun to do that since they are already transforming typescript to JS.

But you can do it —- there a babel plugin in the TC39 proposal:

https://github.com/tc39/proposal-operator-overloading

I'm glad that proposal was withdrawn; it would've been by far, in my opinion, the worst implementation of operator overloading, in any (mainstream) language

I had glanced through it a long time ago. Maybe it’s time for someone to create a new (and better!) proposal.

Could not agree more hahaha! I tried to work around it building methods like “torch.add(a, b)” for operator overloading and “torch.at(index)” for slicing. But it’s definitely not as seamless as these features you proposed.

You should do it! If you actually had a solution for operator overloading you'd really stand out from the other various JS deep learning libraries. Save me from pip and conda please :)

I can try implementing it in the future lol It would surely be a quality of life improvement. But with the current tools, I tried my best to make the syntax as similar as possible to PyTorch’s!

If you do want to add evaluation of mathematical expressions you should check out Math.js since they provide a parser among other utilities. Please make it optional though, it would be a nightmare to debug if everything was written in strings.

https://mathjs.org/docs/expressions/parsing.html

Thanks for the tip, will look into it! Yes, I think it would always be better to leave a "vanilla" option available.

Curious, why do you need to construct these as class instances, like operation = new Exp() ? Seems like a lot of extra overhead constructing those objects. Why not just have Exp contain static methods for forwards and backwards?

[edit] nevermind, I missed the cache step. Still not sure it wouldn't be more performant to centralize caches as plain objects somewhere rather than to call new() on every op...?

I centralized the entire backpropagation around the Operation objects. They store the data about the forward prop in the cache, and serve as the connections in the graphs between tensors. Each tensor has a “parent”, “child” and “operation”. These store who generated the tensor, what tensors it generated, and how it was generated (what operation). I could store the backward function inside of each tensor instead of an Operation object, but I chose the slightly more verbose option because I think it is a little more interpretable and simpler to add new operations.

Would JS be faster than Python when it comes to Pytorch? For example, I seriously doubt that would be the case for Numpy, since it's a wrapper for C code, with the ability to use Fortran libraries for optimization.

There is often a lot of work that has to be done before and after PyTorch gets involved. For example the code I'm working on right now involves reading and parsing a bunch of files, filtering and extracting a bunch of data based on various criteria, formatting that data, passing it to a PyTorch model and then taking the results from PyTorch, validating it, reformatting it and then writing it to disk. The PyTorch part is probably as fast as it can get, but most of the overall runtime is spent doing all that other stuff and if you can speed that up then that is a clear win in many cases.

The benefit of having it in JS is not speed but portability and access to the JS ecosystem of tools. Having the code run in the browser without needing a complex setup is a huge benefit for sharing demos. Node.js provides a way to use native code as well and it's quite commonly used https://github.com/nodejs/node-gyp so there's no reason you couldn't use those same or similar libraries in a JS implementation.

C running on the CPU isn't fast enough for ML. You need to run on GPUs or TPUs if you're serious.

Yes, most of the tensor operations in PyTorch do their math in native code. However, Python still does orchestration and other tasks like data loading and because it is so slow it still ends up causing a ton of overhead in many cases despite offloading most of the work. It's very common for the GPU to sit idle between kernels while Python spins. So JavaScript being faster could still be a big advantage.

Very much the opposite, since this is pure JS. PyTorch uses tuned native-code and GPU components for the heavy lifting and can in some cases compile Python code using PyTorch JIT / torch.compiler / torch.fx.

It'd be shocking if it was. PyTorch isn't particularly slow.

You could use eval to expand whatever syntax you like into function calls. It could be done once at the start of the program and act like a js preprocessor.

I know eval is not kosher but this problem is also not real, so why not.

That's an interesting idea as well, could definitely see that working in some use cases.

Wow! Thank you for doing this. It looks like a great starting point for anyone approaching deep learning from the JS ecosystem. It is very plainly written and looks like it will be a joy to learn from. Thank you for adding JSDoc comments with type hints!

Are you open to pull requests? If I have the time I'd love to contribute. I'm sure others would as well.

You should write up a short article on this, even something really simple like one of the examples in the README but with some commentary and examples of output and then post it to a few places like https://dev.to/ or maybe https://hashnode.com/ or even Medium (even though I'm not a big fan). There aren't many newer implementations of PyTorch in JS and I've been looking for one to learn from for some time so I'm sure there are a lot of other JS/TS developers out there that would be interested. Getting to the front page of HN certainly helps but having an article somewhere will help everyone after this week find it through a Google search.

Again, thanks so much for doing this work! It's really helpful to have everything spelled out in JS for those of us who haven't used Python much (I'm sure Python devs can relate when they think about JS projects).

Thank you so much for the feedback! I’m open to pull requests, absolutely. And about the article, that’s a great tip, I’ll definitely do that as well.

Learning Machine Learning, I’ve always been interested in PyTorch and its Automatic Backpropagation.

In this project, I tried to reimplement most of PyTorch in Javascript. It is implemented from scratch in a well-documented, unit tested, and interpretable way, so it can help other JavaScript learners get into Machine Learning!

Hope you enjoy!

You might look into node api or ffi and add some native code to speed up some operations on the server.

I helped with this project to do that and get prebuilt binaries: https://github.com/ashvardanian/SimSIMD

I haven't looked at ffi over node yet.

Thanks for the tip! I will look into it. Speed in the tensor calculations is definitely one of the largest challenges in this project.

This is awesome! Congrats. Clearly a lot of work and a very cool library.

Thank you very much for the feedback! If you have any question about it, let me know.

BTW: you might want to add support for typed arrays.

See: https://github.com/xenova/transformers.js/blob/8804c36591d11...

This is really old, but added as part of the shape of the vector as well: https://github.com/nicolaspanel/numjs/blob/master/src/dtypes...

I had read about this as a possibility, but will definitely look more into it now, thank you for the tip!

I agree. This advice should not be ignored; and if the OP is complaining about performance, the answer is lying here in plain sight.

Very very cool to see!!!

So yours is SO MUCH MORE fleshed out than mine, but I have a similar (again very fledgling) one going here that uses Typescript:

https://github.com/Marviel/lab-grad

I think there are some really neat opportunities to increase the approachability of ML algorithms when you take advantage of the Typescript templating system to explicitly ensure you're not accidentally composing tensors of incorrect shapes.

Unfortunately, my above lib above doesn't yet handle tensors, it's still at the level of individual neurons. Hopefully someday I can bring it further along, use WASM or WebGPU, etc. :)

Thanks for the tip! I will look into your project, for sure! I am still a beginner on Typescript, but when I get the hang of it, I think I might refactor my package to Typescript. Again, thanks for the tip!

Typescript is something you learn progressively over years, and from what others have posted it might not be possible to refactor your package in Typescript (some approaches to problems that are possible in JS aren't supported by TS). But type declarations at the very least should be possible

Fair enough, I’m currently studying what would be the best option, but I think you are right!

Edit: Great work! I'd love to have a nice alternative to PyTorch in JavaScript :)

Edit: formatting

Making JavaScript look like Python in this case could potentially bite you in the ass.

From the example:

    const torch = require("js-pytorch");

    // Instantiate Tensors:
    x = torch.randn([8,4,5]);
    w = torch.randn([8,5,4], requires_grad = true);
    b = torch.tensor([0.2, 0.5, 0.1, 0.0], requires_grad = true);

    class Transformer extends nn.Module {
      constructor(vocab_size, hidden_size, n_timesteps, n_heads, p) {
        //.....
        this.b1 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p=p);

    /**
     * All of the arguments for this function are positional
     * and cannot be provided in a different order than defined
     */
    function performOperation(values, arg1 = false, arg2 = -1) {
        //....
    }

    /**
     * This works, but only because of the order
     */
    const result = performOperation([1, 2, 3], arg1 = true, arg2 = 10);

    /**
     * This does not work
     */
    const result = performOperation([1, 2, 3], arg2 = 10, arg1 = true);

    /**
     \* What is actually happening
     \*/
    arg1 = true; // global variable
    arg2 = 10;   // global variable

    const result = performOperation([1, 2, 3], 10, true);

Thats true, it’s a limitation of working between these languages. I tried to mitigate it by using clear JSDoc, so that each variable pops up alongside an explanation when calling a function.

I feel you - Python has much better (more flexible) argument support than JavaScript in this case. Converting the entire set of arguments into a keyed object is usually what happens, but then it wouldn't look like PyTorch anymore.

Many people seem to be unaware of tensorflow.js, an official JS implementation of TF

https://github.com/tensorflow/tfjs

I'd love to see PyTorch in JS, but I think unless you get it running on the GPU it won't be able to do much.

tfjs is dead, looking at the commit history. The standard now is to convert PyTorch to onnx, then use onnxruntime (https://github.com/microsoft/onnxruntime/tree/main/js/web) to run the model on the browser using webassembly/webGL (and nodejs if you wanted to, but why?).

> nodejs if you wanted to, but why?

Node.js is better backend than something like Flask.

Performance is a lot worse on NodeJS with a WebAssembly/WebGL backend versus Flask with a PyTorch/CUDA backend.

If you're using Node you can write whatever you want in C++ and then add a binding to call it from within your Node app. Don't need WebGL.

But you're having to write whatever in C++ versus just using Flask/Pytorch.

A lot of what you need is already written, you just need to find the right libraries and write the bindings. From my encounters with Python ML it seems like "just use Pytorch" is a bit like "simply walk into Mordor".

Yeah, but why...?

Adding GPU support soon is absolutely my goal in the future! I think a PyTorch-based JavaScript library could be useful, as PyTorch has been way more dominant than TensorFlow recently.

With PyTorch dominating the landscape, my guess was that tf will resurrect itself through tfjs. Seems may not.

I had the same thought, but it does seem that tensorflow usage is in steady decline.

Cool project! I worked on something similar a while ago to learn about automatic differentiation: https://github.com/hexhowells/onegrad.js

Needs more layers to be really useful though.

I always thought examples using browser extensions would be neat since their built in JS and you only need to download the model once.

Really cool implementation. Thanks for the feedback as well!

Is the goal here to do everything in Javascript, or it will also be Javascript for the glue and c++/c/fortran/other-compiled-language for heavy lifting? This is more of a node-based effort, or it will work both frontend and backend.

Initially I'll stick to Javascript (plus some libraries to improve performance), but I'll likely look into that in the future.

The nn.Block should probably be renamed to nn.DecoderBlock as it's not very clear if it's supposed to be an encoder or a decoder block, also an option to disable the mask from attention. That said, very cool project.

You are absolutely right! Will solve that as soon as possible. Thanks for the feedback!

If you have the option to disable the mask, isn't it then a generic nn.Block?

The ability to disable the mask in nn.MultiHeadSelfAttention, then having nn.DecoderBlock and a nn.EncoderBlock.

I think that’s it. I’ll probably add that soon

It would be with some simple tweaks. For instance, the current block does not support Cross-Attention, just Self-Attention.

I like to see progress with multiple languages in the space. Julia, R, etc. It does seem though, that python really has a big and commanding lead. So much so that mojo is betting their business on it.

That’s true. I myself learned ML in python first, and I’m just now trying to implement some stuff in JavaScript. I guess it could be useful to make web integration with neural networks easier in this case, or at least as an educational package!

What language is Python calling to do this? Hint: a language closer to the silicon.

C++, CUDA apis?

This is great. Code seems to have good comments / jsdoc from the bit studied, tests and all. Will take a closer look in the morning but congratulations on the release.

Thank you for the feedback! If you have any questions or suggestions looking it over tomorrow, I’d love to hear them!

This is great!

I was looking for something like this.

Now I just need a guide that tackles the principles of this, but from a typescript/javascript perspective.

PS: TyTorch (from Typescript) sounds nice!

I plan on adding Typescript support soon! Thanks for the feedback

Why not using a faster language that compiles to WebAssembly?

Now that I've tried Javascript, I could branch out to other languages in the future. I chose Javascript initially for its popularity and easy use on web browsers.

Wouldn’t WebGPU be the key to unlocking speed?

That too, but WebGPU isn’t very well supported yet

One of the learning tool is notebook. I even has a lisp backend to enabled it to be used. It will be nice if there is a notebook version and easy to run on web (like colab).

I’m planning on creating a small article explaining the syntax and the functionalities of the Deep Learning library. I think that could be a useful learning tool, do you think it would help?

Do you plan to implement WebGPU acceleration to make it production-ready?

I am currently studying Typescript and other JavaScript libraries to improve the library’s performance. So adding GPU support is in sight in the future.

You might already be familiar, but a GPU.js backend can provide some speedups via good old WebGL -- no need for WebGPU just yet!

[0]: https://github.com/gpujs/gpu.js/

FWIW also taichi is quite popular in python and seems has some javascript related implementation (I haven't used it though), taichi.js [0]

[0] https://github.com/AmesingFlank/taichi.js

Cool! Will take a look

Thanks for the tip! I’ll definitely take a look. Adding GPU support is my next step!

Not a problem, I've benchmarked some GPU backed js libraries here:

[0]: https://observablehq.com/@dleeftink/ultimate-compute-show-do...

Thank you! That’s going to help out a lot!

nice work but why code is in closures using ugly `var` statement?

[flagged]

[flagged]