hckrnws

Your biggest obstacle is proving fine-tuning is more effective than prompting, workflow design, RAG, etc during the initial pass. Most of my customers are still getting big improvements by picking the low-hanging fruit with those approaches. A much smaller fraction is at a place where they're ready to start fine-tuning. Obviously, this will change as AI programs mature.

Exactly! Finetuning needs at least 10 examples to even work. That’s why Promptrepo begins with prompting and schema-based generation when teams have little or no data. As they gather more examples, it gradually shifts to fine-tuning. It’s the classic cold start problem and we’ve simplified it for product teams who want to launch quickly but improve accuracy over time.

Can you share an example of such real world win where fine tuning was less effective ? I’m curious about sample business cases.

Incredibly crowded space, but this is a great insight and UI... engineers probably have to integrate the model but we should empower non-technical customer-facing people to give feedback to the model in a way that improves it.

The blocker for me (and likely other cost-conscious early stage groups)? I have free credit and existing integrations with more mainstream platforms (OpenAI, anthropic, together). Trying this out will cost both eng time and money, so I won't be an early adopter. I wonder if there's a way to pass the cost through / use my API keys with credits. Maybe it's more for enterprise teams or cases where you're already confident about the fine-tuning approach.

Anyway, congrats on the launch!

Thanks! Yes, that's a popular request from other developers as well. That's why our basic plan is self managed plan (Bring your own OpenAI account) - https://promptrepo.com/finetune/pricing.html

We have one month free trial as well. Free free to ping me if you need more time.

I love integrating into spreadsheets. Super easy to use. Reminds me a bit of mailmerge.

I don't have much experience with modern finetuning, but isn't it highly technical. How many layers do you want to change? What is the learning rate? Does that need to be visible to the user? How many examples are needed in practice

Finetuning is technical, but OpenAI hides things like layers, learning rates, and uses LoRA under the hood. You just upload examples - usually around 50.

But even that’s too much for most business users. Choosing input/output fields, generating and validating JSONL still feels like coding. That’s why we built PromptRepo: it turns finetuning into a guided, no-code workflow using spreadsheets.

I don't like the pricing. $38 a month headline much smaller billed annually with the price more than doubling. This is the trick your customers strategy and hope for no chargebacks.

I understand your concern. We’ve had cases where short-term users abused our product for phishing, which led us to remove the monthly plan initially. To address that without locking out genuine users, we’ve added an option to extend the trial for up to 3 months before choosing a monthly or yearly plan. Feel free to message me if you’d like more time.

When I see a product — especially an early-stage product — trying to funnel me into a yearly subscription, the signal I get is that you think I’ll quit in less than a year which, in turn, implies that your product isn’t very good. It just makes me trust you less.

There's some truth to that. Early stage products often start out rough and part of the journey is finding early users who believe in the potential enough to stick around while we improve.

That said, in our case, the switch away from monthly plans wasn't just about churn. We actually got shut down by GoDaddy due to phishing abuse [1], which forced us to rethink our approach. We've since added a flexible trial extension to avoid punishing genuine users, but I’m open to feedback and willing to change if enough people feel strongly.

[1] https://news.ycombinator.com/item?id=36934109

This is anecdotal but I often buy yearly plans on products that I've got a pretty good idea that I'll use, even if it's the first time I buy them. Though I'll freely admit it's always been the option of getting 1-2 months for "free" by signing up for the yearly plan. I've done it with various products, Proton, Disney+ (which saved them from my Danish boycot wrath to my daughter's delight) and so on. I've never even thought about it as a way of tricking me into buying the yearly plan, and now that I've thought a bit about it, I don't think your pricing would either. It's hard to say how I would've thought if I'd looked at the pricing page before reading these comments, but I genuinely don't believe I would've thought of it like that.

That being said, I would not base my pricing too much on a few random comments on HN. These responses can be a good indication, but I'd frankly reach out to some of my trial sign-ups who didn't transition into a paying client as well as some of my actual customers to get their view on it. I think pricing is going to be especially tricky in the AI space since it's so new and there is so much competition.

Good luck though.

What’s the thinking of spreadsheet first? Just making it super accessible for people who already have data?

I’m building a UI for fine tuning (and evals, and synthetic data gen) - https://github.com/Kiln-AI/Kiln - and went the custom UI route. From chatting with folks - most people don’t have datasets, and need help building them.

It depends on the use case. For many business workflows, where structured data is key - spreadsheets are already the source of truth. But for chat-based or unstructured tasks, a custom UI might make more sense.

How do you make sure the fine-tuning process stays effective without the risk of overfitting on specific data?

Ideally, you want to start small and iterate. With Promptrepo, you can use versioning to compare model outputs across different datasets. In the test UI, we calculate confidence scores using @promptrepo/score [1], which parses OpenAI’s logprobs and shows field-level reliability. Fields with low confidence are highlighted in red, making it easy to catch signs of overfitting or data drift.

[1] https://github.com/ManiDoraisamy/promptrepo-score

Neat! Wonder though if you should be even offering the BYO option as a separate lite package. As a dev I would not buy this and as a non-tech person I would be confused by your pricing page.

But I do see the value! Think sales or marketing folks looking to get a bit more hands on. These will likely be your first visitor and be okay with your 50 dollar price. Then, their IT department will say “we want to hook up our own API key for that,” to which you can confidently say “sure, we can do that too.”

N=1, just my two cents. Good luck!

Who is your target customer?

From reading the site, it's not clear that someone who doesn't already know about fine tuning and want to do it would know what your service does or why they would need it.

Recommendation: describe the process, and give some examples of applications.

Our target customers are product teams like ours. I’m the technical founder and was building a feature to extract product prices from unstructured text. But my co-founder, who understands all the pricing formats — is non-technical. I built Promptrepo, so he could finetune the model himself and improve accuracy without relying on me. My bet is that this same dynamic exists in many other product teams.

Thanks again for the feedback and recommendation. We’ll update our site with clearer examples and target use cases!

this is great, and right up my alley. i built a customer support chatbot using a google sheet for our CS folks to input & structure their question/answer pairs. that is auto xformed into markdown and fed into the bot for context. it works fairly well considering how simple it was to do. i'm really intrigued as to what promptrepo can add to that. will definitely give it some R&D time!

Yeah, Google Sheets is a surprisingly powerful interface for business teams, especially when they’re the ones curating the data. Curious to see how Promptrepo fits into your workflow. Happy to help if you explore finetuning on top of your existing setup.

Heads up I get "ERR_CONNECTION_RESET" when navigating to https://promptrepo.com/finetune

Thanks for the heads up. Looks like a traffic spike from HN. Can you please check now?

Still down for me but I bet it's the traffic spike still. I'll check back in later!

FineTuning Ops platform requirement - compare evals, latency, cost across models.

We don’t have automated evals, latency, or cost comparisons yet. But, Promptrepo does offer versioning and lets you deploy the same model across providers for comparison. Automating these comparisons is definitely on our roadmap.

honestly i like seeing tools get simpler like this, makes me wonder though how much of the real value comes from the tech itself vs just making everything less scary for folks who aren't engineers. you think easy interfaces actually help people get better results or just get more folks trying stuff?

Depends on who’s using the tool. Developers might be fine coding a form into their app, but an HR person needs a form builder. Similarly, the data for training models usually lives with domain experts, but they don’t have a tool to actually do the training. That’s why, in this use case, a simple interface makes sense, IMO.

How is this different than OpenAI projects?

Can you please clarify what you mean by "OpenAI projects"? Are you referring to the playground or the API for prompting or fine-tuning?

In OpenAI Pro ($20/mo) one can start a project with a set of files. Various chats can be had about this project topic with the files providing additional information. I've discovered the projects are isolated. They'll use memory configurable in settings but they don't use chat history outside the project. This can give ChatGPT chats in projects a different tone.

My question is this: Is this fine tuning with those project documents or RAG and what's the difference?

Thanks for the clarification. OpenAI projects in ChatGPT are meant for end users to get personalized help using their own documents, inside the ChatGPT UI.

Promptrepo is for developers and product teams to build new AI-powered features in their product. It’s about creating custom models that run behind the scenes in apps, not just improving a personal chat experience.

So while OpenAI projects use RAG for better chats, Promptrepo helps teams build and deploy fine-tuned APIs that serve structured outputs like JSON, labels, or extracted fields to build your own AI powered product.

Why Google Sheet though? Why would you want your customers to give their training data to Google?!

Don’t want to sound like a shill, but Google doesn’t use data from your Google Sheets to train its models or for advertising. By default, your data stays private and is protected under Google Workspace’s privacy policies.

You are still giving your data away to Google, and it's one line of ToS change from allowing them to do it. And we know this Lind of changes happen all the time.

This is awesome, I don't have a fine-tuning use case yet, but I can't imagine something being easier than a spreadsheet.

Agree, not super familiar with this space but that you are using GS as the UI makes this super accessible to so many more people given familiarity and low intimidation factor. Best of luck.

Essentially this is a frontend to automate the process of converting a csv file into jsonl and pass through a fine-tuning service.

Yeah, just like Dropbox was a passthrough for aws s3.

Edit: Sorry about the snide comment. But if this ends up as a simple utility for finetuning, I would be happy with that too. Just want to share a tool that's been very useful in building ai features in our products.

That's the biggest problem with showcasing to developers. They are just not representative of the user base at large. For example, any Desktop application showcased will getting annihilated for being Electron. You very much take a risk putting it out to developers first because they will leave commentary about your product in this way (forever, this is the internet, it's not going anywhere). We eat out our own basically.

I am not saying there isn't value for this, just saying what it does. I might actually use this if they don't tie the fine tuned models to only be used within their platform.