hckrnws

I was one of the “lucky” few to witness the school of slur-fish.

Being in security I laughed because of how egregious it was but also because I knew someone on HN with some actual time on their hands to help properly would be along soon.

I also appreciate this post mortem. Vibe-coded anything in prod is a lot of my work load in IR these days but it was nice to see such a low stakes project properly documented.

People will be quick to jump on the "it was vibe coding's fault" but at least two of the issues are pretty common even in designed systems without AI - leaving in a "test admin" access and verifying tokens but not cross-checking them.

This is pretty reductive of the actual problem people typically complain about with vibe coding - It produces very workable prototypes fairly quickly and without a lot of hassle. Great! The problem is, and this is a great example (of many) where someone mistook the working prototype with a system that was ready for production. The JWT thing in particular is not really a mistake many people who work on that kind of thing would make.

People need more understanding of the risks of vibe coding and YOLOing to prod with these tools. They are powerful, but like all powerful tools, can be wielded irresponsibly.

it's just incompleteness -- a human issue.

most in-use LLMs prompted with a simple "You're in charge of infrastructure security, let's review possible problem points" would have uncovered this.

I wouldn't fault a compiler for erring when someone left out a period; i'd tell the person to start including it -- but for some reason the expectation for LLMs is hands-off work ; I guess we're just in that phase of the hype at the moment.

I think it's pretty reasonable to expect AI to produce systems with issues "pretty common even in designed systems without AI" because that's what AI was trained on.

But that isn’t the expectation or what is being marketed

I expect these AI and LLM to be, basically, a middle of the bell-curve type producer of code. Just like their other output. Not terrible, not exceptional, just what a Mid could do - only faster.

Not sure what's being marketed, but I expect mediocre.

Being marketed by who? Be specific.

“Be specific.”

Am I the only one that feels like it’s really condescending when people say this on the internet?

It sounds like something you would see on a community college writing assignment

Great post-mortem, especially since it's a vibe-coded app.

Curious if you were inspired by Lego's build-a-fish* exhibit at the Lego House? I visited recently and it is ridiculously addictive to see a fish you create swim with others :)

https://www.youtube.com/watch?v=KYs3ne0HCwM

Oh! I hadn't seen this. I was more inspired by the St Louis aquarium (where you color a fish and it swims) and Google's Quickdraw (a memory from like 2016)

TeamLabs also has this in both of their main Tokyo art installations (Borderless & Planets).

I've noticed with nearly all of these "Vibe Code" security fatalities, they're nearly ALWAYS using Firebase as a backend. I get it, I've used Firebase for a number of enterprise and personal projects, its convenient and easy to setup.

But even before LLM coding, I had team members walk into its numerous footguns - especially around public buckets and bad firestore rules. How many of these stories are really to be blamed on the AI tooling, and how many could be blamed on the very poor default settings of Firebase?

The "S" in "vibe-coded" stands for security.

The S is for snake... errr.. dragon...

Wild that some random used a security hole to try and counter the malicious actions actively lol

This has happened a number of times that I remember - one was a worm/exploit that would patch the hole.

I remember reading about 5 or so years ago that the FBI was doing exactly this to counter the EternalBlue exploit (I might be getting the exploit name wrong)

I've seen multiple articles where Microsoft does this by taking over C&C domains with court orders

Relevant link maybe: https://www.justice.gov/archives/opa/pr/justice-department-a...

this was awesome. people are surprising

There's a long history of this. A defense against the Morris worm made use of this as well.

There's a swastika fish on there right now. Folks get through the filter by putting it inside a fish shape:

https://drawafish.com/rank.html?userId=1754341779700_log2xle...

Edit: it's been deleted.

That fish bears the touch of the Buddha himself.

Got a killer original idea, got it built, learned a ton, is embarrassed about a few failures because he’s actually a solid professional. Well done, mate. If it didn’t cost you $100k, take down the network or lose you your job, you’re ahead and will laugh in a year.

You can also upvote any fish without auth, limit is 20 votes per minute per IP

POST https://fishes-be-571679687712.northamerica-northeast1.run.a... {"fishId":"xxxx","vote":"up"}

That's actually intentional desgin - I think you can like a fish a little or like a fish a lot, and therefore should be able to upvote/downvote to your hearts content :)

The only potential risk I see with anonymous voting is some 4chaner is apt to create a bunch of fish with horrific names and use distributed IPs to upvote them to the top. I guess regular moderation could filter that out.

IP based breaks users with CGNAT and people connecting from corporate networks

In those cases you'd be denied votes since someone else used them up

Any button a human can touch is a vector for abuse...

Great project and a good post too, you write well and are funny. Would like to see more for my rss reader :-)

…if you had RSS…

I was 15 years old when they killed google reader, so can you blame me for forgetting about RSS?

Added :) https://aldenhallak.com/blog/rss.xml

Ha! Subscribed, and now the pressure is on to write something ;-)

If you are the dev think you need to add an option to mirror the fish drawing.

I can draw a fish facing left, but for some reason it's very difficult to draw one facing right.

Can't you just draw facing left and then flip it horizontally?

Sorry, that's what i meant, i think it needs an option to flip it horizontally on the page, not mirror it.

> You may have heard differently, especially if you saw my doxxing[4b] on the unsavory website.

Is it pretty common to get doxxed for getting to the top of HN?

It is not.

But if you show HN something... and it makes HN-reading KF users think "OH EXPLOITABLE!!!111!" so they post it to KF... they're basically going to test if your moderation works or not. If you only have some automated rules or even "AI" as the moderation, not humans, then they're very quickly going to work out what gets past your automated filter and what doesn't, faster than 4chan can make Microsoft's Tay say "Hitler did nothing wrong".

Then the KF users will gloat about the slur-ridden fish they've made.

Then KF-reading Sharty users will learn about it, the rest of the Sharty will join in the game.

Then they'll get angry that your "normie" userbase has given a Palestine flag fish named "River to the Sea" the most upvotes, and an Israel flag fish the most downvotes, they'll want to cheat and hack their own creations to the top. So they'll start looking for an admin interface.

When they find one, but find it needs credentials, the first place they'll go looking for password clues is all your socials, all your previous forums, basically your whole identity. And if they hit paydirt, of course they'll post whatever they find, because for them, doxxing people is fun too.

That was quite something. Thanks for all the fish. Also for posting this.

> I used the JWT to authorize login, but never confirmed that the JWT token belonged to the userId / email associated with it in the admin actions. So you could log in with my username and password, grab the JWT, and then send that along with your request.

IANAWD: What is more appropriate than an admin token being able to authenticate admin actions?

I wish there some screenshots of the vandalism.

Here in Zurich there's a mural of maybe twenty dinosaurs (not accurate but something that looks like it would be in a children's book). One day someone drew a dick on every single dinosaur. Even the flying pterodactyl had a big dick hanging off of him. It was so puerile and primitive it cracked everyone up that saw it. No tags. No football club graffiti. Just dicks everywhere. Thankfully the mural was repaired pretty quickly.

There's a screen shot of a certain site in the footnotes of the blog and if you visit it, you can probably find some screenshots in the 30-something page thread about OP's game.

Edit: yes, the site too terrible to name as evidenced by the reply to this comment becoming dead within minutes l-o-l

I've never understood why everybody gets upset over New Zealand farmers.

[dead]

> I vibe coded...

Well, there ya go.

To be fair, it says the attention was unexpected, and this was just a coding exercise... And the port-mortem shows what I'd hope to see: digging in and figuring out root causes. So I'm not judging OP poorly over this.

But still. Launching a vibe-coded app that accepts input from anonymous users is just asking for trouble. I'm frankly surprised it ran as long as it did without such problems. (Although I did see a few weenies swimming around even before the problem hours.)

The lesson I'd pull from this is that if you are not the type of dev who could put together a post-mortem along these lines... don't launch a vibe-coded app.

So long and thanks for all the fish.

On the allure of vide coding the author says,

> It is really fun to just have high velocity, and it is really fun to not do code reviews and to just push stuff.

Was slurfish fun?

Looks like if you don't like doing deep and thorough code reviews, LLM-generated code is not for you.

As the author concludes, "...LLMs are a tool. They let you generate a lot of code really fast...it is up to you to review it"

>Was slurfish fun?

It was for someone!

I am sure someone somewhere works on making LLMs commit code. Aside from that it was great witnessing the site in action and reading the postmortem. I wonder how the "hacker" made the connection to the user acount on neopets.com but maybe they just tried something like "ahallak"?

This was so fun. Hope to see more like this

When i tried it, it almost blew my cpu.

this is why we cant have anything nice.

I read the HN thread first, and the first comment I saw from the author was about AI nazi symbol detector they put in... I wonder how many orders of magnitude that comment increased interest in making offensive fish?

Later I saw images of the attacked site posted elsewhere and thought they were both predictable and hilarious.

I was proud of my ability to have an extremely penis looking fish get past filters. I feel like when presented with censorship my instinct is to always test its limits

[flagged]

Why would you do this?

A bit of harmless fun?