hckrnws

Oh Sandstorm, what could've been...

Taking from https://sandstorm.org/about

> Kenton Varda [NB: kentonv around here] launched Sandstorm in 2014 via an Indiegogo campaign, before co-founding Sandstorm Development Group with Jade Wang to develop Sandstorm as both a Software-as-a-Service [...]

> In early 2017, Sandstorm Development Group ran out of funding and the team primarily joined Cloudflare. [NB: Where kentonv works to this day, leading the Cloud Workers team. Arguably related?] [...]

> In 2020, a group of Sandstorm enthusiasts began a community effort to revive development of Sandstorm. [...] As of 2022, Sandstorm Development Group has been completely dissolved, and development of the Sandstorm project has transitioned to a community-run model.

kentonv actually posted a recap of the history, including the tragic passing of Ian "zenhack" Denhart who was leading the community effort https://sandstorm.io/news/2024-01-14-move-to-sandstorm-org

What could have been? I remember being so excited about this at that point, I was sure it was going to take a significant chunk of [famous web services still around]. They even send me stickers for Cap'n'proto..

If I understood it well enough, the intention was to formalize/standardize self-hosting such that it became easier (and with an emphasis on security).

With that done well, perhaps the next step would have been ready-made images. I could see it reducing friction for people if they'd rent a VPS, upload an ISO/IMG, and the resulting system was reliably secure. Then they'd use the Sandstorm interface to install apps with a web GUI.

There is still a roadmap documented here:

https://github.com/sandstorm-io/sandstorm/tree/master/roadma...

Hello everyone. I have been using Sandstorm and put it to good use in the last few years.

I used it with Wekan for project management and I also run Dokuwiki for self-hosted docs. It has been zero maintenance for me so it has been great.

However, the packages ecosystem seems unmaintained. It is a pitty because I think the tool has a ton of potential and I really liked it.

I am considering moving to Yunohost or something similar but right now my little server hosts, together with other services, Sandstorm and I think Yunohost needs to monopolize the server.

So I would ask for recommendations on similar tools. Not bare Docker containers but fully lanaged platforms wirh one click installs where it is easy to add/remove users.

I have been thing a lot about why none of those self-hosting solutions never took off and most of them died off over the last decade.

Why are we going dehydrated in the middle of the ocean, with docker and so many open source alternative to the common software and services ?

My conclusion is this: just pick the distro you like, whether it is Debian, Fedora, Arch or FreeBSD. Preferably one with the selection of package you need. All those will be maintained in a few years too, you just need to upgrade.

Because in the end the solution was the problem. A Debian for example was meant to host Internet services, it is well put together and has a large selection of software and can be trusted. It is more than enough security features for hosting your own apps, especially if you access them through something like tailscale.

A lot of people (me included) thought that since containers were the hype we should build something new and setup everything like a corporation would do. Looking back it was a bad idea and it did not work (fact).

> It is more than enough security features for hosting your own apps, especially if you access them through something like tailscale.

But part of the whole point of Sandstorm and its ilk was that we could have so much better security. Sandboxing all the services really is better than what we're getting now.

I for one use Cloudron.io . Paying customer for 6 years.

> So I would ask for recommendations on similar tools. Not bare Docker containers but fully lanaged platforms wirh one click installs where it is easy to add/remove users.

I've done a similar journey for my self-hosted stuff, started with Sandstorm, moved to Yunohost, but got frustrated with the configuration, and how different every package was and eventually have landed on using NixOS for my home servers. It's not a "fully managed platform" in the traditional sense, but if you're a developer, that's almost what you get. Adding new services is usually just adding the configuration for them.

Bit of a learning curve learning the language, tooling and ecosystem, but once you're over that hurdle, having all declerative configuration in SCM together with easy linking of configuration options together (define service ports once, reference them in other services, for example), and everything being easy to rollback, have been a god-send so far. Been running it for maybe 2 years or something by now, with more or less zero issues besides the ones I introduce myself.

Adding/removing users can be as easy as adding/remove one line of configuration, and redeploying. Simple enough for me and my family so far.

For Sandstorm alternatives with one-click installs and user management, consider Cloudron, Caprover, Umbrel, Unraid, TrueNAS SCALE apps, or Portainer with its app templates - most allow running alongside other services without monopolizing your server.

I’ve been using Caprover for the last 4-5 years, and it’s great for my purposes.

It sounds like the OP may be looking for something that can be provisioned for multiple users, which Caprover doesn’t really have facility for. That said, the one-click-app templates are very easy to read/edit/update/create, if you can create a Docker Compose spec, you can create a template for Caprover.

I like that it’s really just a frontend for Docker, all of its functions are built around standard API calls to Docker. It means that I can install Portainer as a one-click-app, and use it to manage all of my other apps, without sacrificing any features. I can also spin up Docker containers outside of Caprover, and as long as I don’t run into port conflicts, it functions exactly as it should.

Caprover is just a one-man-show, so development may not be as responsive or rapid as some may wish, but the maintainer is very active, and he responds to pull requests promptly. One Click App templates are largely community maintained, but since they’re so easy to deal with, it’s trivial to work around most common issues, like updating versions or exposing new environment variables, and once again, the maintainer is very good about merging pull requests for app templates.

I loved the idea of Sandstorm as well, but ended up moving my stuff to Proxmox and TurnKey Linux. TKL is a similar idea to Sandstorm, but on the container level, so what you download are LXC images, ready to go after a minimum of user-friendly configuration. They are going since 2008, and they have around 120 offerings.

TKL is integrated into Proxmox, so for example if you'd like a new Wordpress site or a Nextcloud or Mattermost instance, you just click a bunch on the web interface, it downloads the image and installs and runs it, you do the initial setup my giving it a bunch of info on a user interface, and the thing is ready to go.

There are lots — “selfhosted OS” is the term to look for. Umbrel and CasaOS are some other popular ones. I don’t personal experience with any of them though.

There also used to be Freedom Bone [https://gitlab.com/bashrc2/freedombone] ...but, honestly, I've never used it nor am even sure if its still alive as a project.

I was initially enthousiastic about sandstorm when I encountered it, but in the end my preferred solution for self hosting has been Docker Swarm. Dead simple setup, low maintenance, everything easily deployable within Swarm (crons, backups, first deployment setup, reverse proxy config incl. certificates, etc).

Additionally a lot of projects provide a Docker compose file which is mostly compatible with swarm. I started using Swarm [1] when k8s was already ruling, but never regretted my choice.

1: https://www.yvesdennels.com/posts/docker-swarm-in-2022/

Yes! Docker Swarm is still being supported in 2025, and we have community and solution resources in an Awesome List https://github.com/BretFisher/awesome-swarm

sandstorm.org is an offshoot of sandstorm.io, my old (failed) startup. Some folks who really wanted to keep the project running took over ownership of the open source code. I'm not involved these days. Full story at: https://sandstorm.io/news/2024-01-14-move-to-sandstorm-org

The people still trying to run it are good people, so I hate to say this, but... it doesn't seem like they are going to make progress. There hasn't been a new release in years, and the codebase is rotting. I wouldn't recommend running it in its current state.

Meanwhile, though, I feel like the advent of vibe coding has made Sandstorm's approach seem newly relevant. Sandstorm's architecture made "novice" code safe to run even for users with sensitive security needs. Well well well, suddenly we have a whole lot more novice code than we used to.

Is it... time for me to take another pass at this?

There are a lot of people talking about Docker, Cloudron, etc. as alternatives, but nothing else approaches Sandstorm's grain concept, which is a huge differentiator.

As an internet random, please believe I have the least creepy intentions when I say I really enjoy following your career. Through Cap'n Proto, Sandstorm, CF Workers and Durable Objects, there's a throughline I feel you have ben chasing. And it's a super interesting thing, a unique emphasis you seem to have brought to the world in several different ways. So, keep up the good work :).

I could have definitely used it in my company nowadays, as a paying org. A refresh of available apps would maybe be necessary now. Sandstorm was so awesome.

> Is it... time for me to take another pass at this?

Yes please. I was very excited for Sandstorm when it first started. Sad to see it's current stage.

Also I think the world around has evolved quite a bit wrt containerization from when Sandstorm first started. I wonder how you would build it today, if you were to build from scratch. Could you utilize docker for most of the containerization?

I think if I were doing it again I would not use containers at all. Instead I'd use isolates -- like Cloudflare Workers. In fact I'd probably build in on workerd (open source Cloudflare Workers runtime).

I'm obviously a bit biased here, as the architect of Cloudflare Workers. But, I think it's a much better fit for Sandstorm than containers were. Sandstorm suffered from some really bad cold start times, especially with every "grain" (e.g. document) running in its own container. It also led to a lot of other inefficiencies.

The down side of this is of course that it'd be much harder to port existing apps to the platform (unless maybe if they were already written to target Workers). But I think:

* Sandstorm didn't support existing apps very well anyway. It took a lot of work to convert apps for Sandstorm's security model. Many of the best Sandstorm apps were written from scratch for Sandstorm -- which incidentally was a lot easier than writing a traditional app, because Sandstorm took care of a lot of the work for you.

* AI-assisted coding takes a lot of the pressure off to support existing apps, because it'll be that much easier to build new ones. A Sandstorm-like environment would be a great fit for vibe coding since it takes care of so much of the boilerplate and enforces security in a way that the app can't screw it up.

What do you think? Would this ruin it for you?

The ability to convert existing apps is possibly one of the strongest benefits, even if it is a lot of work.

One person actually trying to take a pass at this without trying to support app porting is Olivier Forget's Dropserver. I'd argue he's got the closest model to Sandstorm's security focus without worrying about supporting legacy packaging.

I do think if vibe coding is up to the task it should be possible to vibe code Sandstorm right out of its dependency lock. ;)

> I do think if vibe coding is up to the task it should be possible to vibe code Sandstorm right out of its dependency lock. ;)

Not really -- you'd still need a systems engineer who is capable of doing it themselves to guide the AI and review the output. Otherwise it'll probably end up a buggy, insecure mess.

But buggy insecure messes are not a big deal when implementing application code inside the sandbox. So for that you can have a non-engineer prompt the AI and get a perfectly usable result.

Sandstorm, to me, feels like the ideal place to build apps which are "home cooked meals":

https://www.robinsloan.com/notes/home-cooked-app/

AI assisted vibe coding is part of that, like Maggie Appleton talks about with her barefoot developers:

https://www.localfirst.fm/13

Sandstorm takes away the concerns of implementing a SaaS, it takes away the identity handling, the hosting, all the nonsense and just gives you your own little world. It seems like it would be a really nice way to maintain family or comminity, or even small business tools, if it were just a bit more polished and focused on use by non-experts.

Yes that's exactly how I'm thinking of it!

What I do want to understand is, why would anyone use this instead of NextCloud, for example. I'm all for having local setups though, as a general policy. It's just that from what I know, the difference in feature sets is just too huge right now.

Sandstorm was like an App Store for any server app. You could install nextcloud with it.

Was a pretty good idea but there wasn’t any sustainable business model/funding and it was also pre docker era which would have made things more secure/containerised and easier to maintain.

You could not install Nextcloud on Sandstorm. Sandstorm has an app that used to work with Nextcloud's mobile app, but Nextcloud makes a very poor Sandstorm app because it is a monolith, instead of being granular.

(Personally I also find Nextcloud incredibly sluggish, but it is quite popular so it must work better for others.)

This still exists? Is anyone using it? What's the use case?

Imagine iCloud but you can manage more than 1,000 files at once, the files transfer much more quickly and don't have to use a touch interface that flakes out, plus no rent.

Is it just me, or is Sandstorm just not maintained any more?

The most recent closed issues were self closed rather than as the result of development, while meanwhile the open issues continue to pile up with virtually no code changes made to the tree…

It’s a shame because it seems like it could have been a thing. Sadly though it’s hard to justify time investment into a platform like this if you know there’s little to no chance of getting any issues fixed.

There is a small community working on it (hi!) but there is several hard problems in the way of moving forward, all of which can be solved with time and money, neither of which we have a lot of. :/

I think ten years on there remains nothing even remotely comparable to Sandstorm for a dozen reasons, but I also can't assure you an issue you find would get fixed expediently, for sure.

There is a small community maintaining it - there should be a link to the Zulip on the website - but it is a small group and a complex beast. Some effort is going into a rewrite that keeps the same app/security model, but moves from C++ to Go and simplifies the database layer. I believe that's taking up a fair bit of contributor energy.

They didn’t quite reach the MVP level and so in a way it wasn’t really maintained in the early days either.

I don't know about that, I happily use it to host a couple of Wekan and Dokuwiki instances, and some random Etherpad docs. It's certainly MVP-complete.

And even in its current state, it has some fantastic affordances that I don't feel I could get anywhere else.

For example, I had an instance running on my laptop to fiddle around with developing apps. I wanted to take a Dokuwiki on my laptop and move it to my cloud-hosted instance. I clicked "download backup" of the grain, which gave me a zip file which was whatever, 40MB or something, and then I went into my cloud setup, clicked "import from backup", and voila - my entire Dokuwiki grain was now in the cloud.

I remember Sandstorm but don't remember it ever billing itself as a web productivity suite... It was a repository of self-hostable apps with their own auth layer integrated so managing users and permissions was easier.

This always sounded like a monumental task. I've since gone with Authelia.

I think this particular marketing choice was at the time intended to pursue enterprise users. It's been there since before the original startup shut down.

I looked into Sandstorm when I moved away from NethServer; I'm a strong believer in self-hosting. Sandstorm was too haphazard with apps and security of apps didn't seem to be their highest priority. I went with Cloudron, it's a nice mix of good app selection and security.

The entire security model of sandstorm is incredibly strong. This criticism is hard to understand. Can you elaborate at all? Do you recall any specific issues?

The problem with the strong security is that it makes it expensive to port into the system. They also assumed more sharing than people actually seem interested in. I'm mostly just running apps that have no reason to share with each other because they each have their own domain anyhow. So the end result is the ecosystem was more expensive to join if you had an existing system than it might have been otherwise, and it inhibited getting projects onboarded.

I am not the craziest self-hoster, but I've got several things now. I run a core syncthing node, Immich, Jellyfin, and Pihole. (Honorable mention I suppose to a Vaultwarden image, which is run on the public internet but my scripts treat it like it's another self-hosted option, rsync'ing it down locally and including it in the daily backup.) None of those are on Sandstorm, and a major reason why is the security system. They don't match it and porting it is a rather large amount of effort.

I haven't used any of the self-hosting options, so I can't review if any of them are as nice as Sandstorm. All of the above is running in Docker on an Ubuntu N150 and a USB hard drive, home-grown, with a backup script (restic over S3, true backup) that covers them all. It ought to in principle do most of what Sandstorm does now by driving docker, albeit missing the sharing, which I can't say looked all that compelling anyhow, automatic backup integration, etc., because it really isn't all that hard to set up.

Security was their highest priority.

IIRC idea was that all security was done by sharing links (with capabilities) to documents.

Smaller cats, bigger screenshots, please.

Cool, what would be the differences with a tool like https://dokploy.com/ ?

That seems to be totally different. Sandstorm was a way of building your own private ~equivalent of Google Workspace out of existing open source web apps running in containers behind a common auth system.