hckrnws

This seems really neat!

DeepMind and AlphaFold are clearly moving in a closed-source direction, since they created Isomorphic Labs as a division of Alphabet essentially focused on doing this stuff closed source. In theory it seems nice for academic tools to have an open source version, although I'm not familiar enough with this field to point to a specific benefit of it.

So what's your plan for the company itself, do you intend to continue working on this open source project as part of your business model, or was it more of a one-off? Your website seems very nonspecific about what exactly you intend to be selling.

Our long term goal is to design enzymes for chemical manufacturing. We decided to build AlphaFold3 because we had seen how useful AlphaFold2 had been for the protein design field. No one else was building it fast enough for us, so we decided we should do it ourselves. We are committed to training and open-sourcing the full version with ligand and nucleic acid prediction capabilities as well since it is so useful for the biotech industry.

Have you considered publishing your own paper about your implementation? It would make it easier to cite in the literature later on. Would major journals accept such a paper? I would assume they would if they really had questions about reproducibility.

OpenFold, which was AlphaFold2's open-source implementation was published in Nature Methods. We will prepare a similar publication once the model is more mature and when we have a nice set of experiments showing the model's interesting properties.

Hi, how are predictions verified? Does one still do experimental techniques (X-ray crystallography, cryogenic-em etc.) one you have the prediction? Or are predictions so close to reality you can progress without experiment?

The predictions can be verified by comparing the predicted structure to the experimentally solved structure, either crystal or cryoEM. The model is still training and improving, we will release the benchmarking results after it's complete.

Thanks for releasing this, I've been looking forward to a truly open version I can use in a commercial setting. What a way to launch the company!

Thanks!

You probably want to change the name of this implementation as it's not truly AlphaFold3. I wouldn't be surprised if you got a C&D from DM for using the name.

Yes this is a good point. We are actively speaking with our counsel to check this. Thanks for flagging, though.

Who would've thought only releasing pseudo-code isn't good enough...glad to see the scientific immune system fighting back against closed-source science. Your move Google.

How dare they make money with something that is not advertising!

There's nothing wrong with trade secrets, but that's business not science.

I mean it shouldn't be enough to publish in nature. The whole point of science is that it can be validated. It's totally fine that they're hosting their models for free on closed servers with limits, even though it's not exactly the most ergonomic.

It was already validated by winning CASP and the paper by Paul Adams (https://www.nature.com/articles/s41592-023-02087-4) which, although it reads like criticism is actually high praise. Everything the model can do, will be (or already has) replicated by the open community.

Also, for work of the highest art (of which AF3 is an example), publication in nature really is the fundamental unit of scientific currency because it ensures all their competitors will get hyped up and work extra-hard to disprove it.

The paper by Paul Adams used an earlier version of AlphaFold that was publicly available, not AlphaFold 3 which is not.

My statement is correct; both AF papers were published in nature, and both won casp. AF3 is superior to AF2 which means if adams wrote another paper, it would be on increasingly less interesting fine details.

To be clear, I don't think anyone distrusts the benchmarking work nor even the reported architecture, but also no one should need to operate on faith when it comes to work that presents itself as groundbreaking. Probably the first thing everyone did when they tested the model was run a sequence w/ a known cryo structure, but that's insufficient for how deepmind knows researchers will use the model.

> Also, for work of the highest art (of which AF3 is an example), publication in nature really is the fundamental unit of scientific currency because it ensures all their competitors will get hyped up and work extra-hard to disprove it.

IDK about disproving it, again nobody is distrusting the work, but let's also not pretend that a prestige journal is necessary to promote AF3. They could publish in the Columbia Undergraduate Science Journal and get the same amount of press. And to be clear the controversy has largely center on Nature for allowing AF3 to get away with more than they would most other projects, and the wasted time and effort it's taking to reimplement the work so people can add to it. FWIW an author did state that they're attempting to release the code but that's not like a binding vow.

Finally, AF3 strictly speaking didn't win CASP (it almost certainly would) but again this isn't necessarily the point when people talk about validation. The diffusion process does seem to result in notable edge cases (most obviously in IDPs and IDRs but also non-existent self-interactions), it's not a straight improvement in that respect.

I did a very brief stint on computational proteomics. That stuff is absolutely next level.

Amazing! What kind of things did you work on?

My job was mostly mundane machine learning: classification over very large categorical sets.

I never had anything more than a dim intuition of the serious chemistry going on before the bytes got to me.

Where were you working? That sounds super interesting

I’m a big fan of what you folks are doing by the way.

Haskell (and Nix) people are fond of talking about “constraints as power”.

https://github.com/Ligo-Biosciences/AlphaFold3/blob/ebdf3b12...

I was a contractor for like a month so I’m not at liberty to talk about the details.

There are a number of companies doing innovative things around quantifying proteins and their concentrations in various samples.

I had the privilege to rub elbows with folks working on such cool stuff.

Does this win the Folding@home competition, or is/was that a different goal than what AlphaFold3 and ligo-/AlphaFold3 already solve for?

Folding@Home https://en.wikipedia.org/wiki/Folding@home :

> making it the world's first exaflop computing system

Folding@home and protein structure prediction methods such as AlphaFold address related but different questions. The former intends to describe the process of a protein undergoing folding over time, while the latter tries to determine the most stable conformation of a protein (the end result of folding).

Folding@home uses Rosetta, a physics-based approach that is outperformed by deep learning methods such as AlphaFold2/3.

Folding@home uses Rosetta only to generate initial conformations[1], but the actual simulation is based on Markov State Models. Note that there is another distributed computing project for Rosetta, Rosetta@home.

[1]: https://foldingathome.org/dig-deeper/#:~:text=employing%20Ro...

If I'm understanding correctly, the model code itself is only a tiny proportion of the challenge. The training compute and training data are far bigger parts.

Google has access to training compute on a scale perhaps nobody else has.

Is that really the case though? Available compute sounds unlikely to be the limiting factor here, compared to data which is way scarcer than what's being used to train LLMs, and I suspect Google used mostly publicly available data for training unless they signed deals beforehand with biotechnology companies which have access to more data. That's possible of course, but that doesn't feel very google-y.

Yes, all data Google used was public. We have enough compute from YC (thanks YC!) to do this. The main thing is the technical infrastructure - processing the data, efficient loading at training time, proper benchmarking, etc. We are building these now.

Thanks for the answer! It's much better to have the definitive answer rather than rely on gut feeling (even though it was right in this case).

Keep up the good work!

How much compute does YC give you access to btw? Is that just things like azure credit or do YC have actual hardware?

What's your next step? Why did you decide to focus on enzyme design?

We think enzymes are super cool! You can build molecular assembly lines at the atomic scale with them. Many pharmaceuticals are already manufactured with enzymes such as the diabetes drug Januvia. Engineering them is a big bottleneck though - takes years and millions of dollars. We want to speed this up with AI-powered design. Next step is ligand-protein prediction capability of AlphaFold3, which is also super useful for modelling enzyme-substrate interactions.

Possibly because it dovetails with pharma mfg and [potentially] food mfg. Could see a case made for enzymatically brewed 'meat inks' [very sorry for this term ;p] for 3d printing the next gen of lab meats.

Are you familiar with ColabFold?

https://github.com/sokrypton/ColabFold

What an unfortunate naming, I thought I'd see some gravitational waves (as I have no idea what alphafold is).