hckrnws
The approach at the end of declaring invariants to the compiler so the compiler can eliminate panics seems accidentally genius. You can now add the same invariants as panicking asserts at the end of each function, and the compiler will prove to you that your functions are upholding the invariants. And of course you can add more panicking asserts to show other claims to be true, all tested at compile time. You've basically built a little proof system.
Sure, Rust is hardly the first language to include something like that and adoption of such systems tends to be ... spotty. But if it was reliable enough and had a better interface (that preferably allowed the rest of your program to sill have panics) this might be very useful for writing correct software.
>and the compiler will prove to you that your functions are upholding the invariants
From the article and only vague background Rust knowledge, I'm under the impression that the opposite is true: the compiler does not prove that. Hence why it's "assert_unchecked" - you are informing the compiler that you know more than it does.
You do get panics during debug, which is great for checking your assumptions, but that relies on you having adequate tests.
That's my understanding as well. The thing I was wondering as I read it: how difficult would it be for someone to make an extension or fork of Rust that allows annotating sufficient type information to prove these kinds of invariants, like F*?
This isn't quite the same, but it reminds me of something a bit less clever (and a lot less powerful) I came up with a little while back when writing some code to handle a binary format that used a lot of 32-bit integers that I needed to use for math on indexes in vectors. I was fairly confident that the code would never need to run on 16-bit platforms, but converting from a 32-bit integer to a usize in Rust technically is considered fallible due to the fact that you can't necessarily assume that a usize is more 16 bits, and frustrating `usize` only implements `TryFrom<u32>` rather than conditionally implementing `From<u32>` on 32-bit and 64-bit platforms. I wanted to avoid having to do any casting that could silently get messed up if I happened to switch any of the integer types I used later, but I also was irrationally upset at the idea of having to check at runtime for something that should be obvious at compile time. The solution I came up with was putting a static assertion that the target pointer width was either 32 or 64 bits inside the error-handling path, followed by marking the code path as `unreachable!()` that would never get executed (because either the error handling path wouldn't be taken, or the static assertion would stop the code from having been compiled in the first place. Even though this wasn't meaningfully different from just conditionally compiling to make sure the platform was suitable and then putting `unreachable!()` unconditionally in the error handling path, having the compile-time insertion locally in the spot where the error was being handled felt like I magically turned the runtime error into a compile-time one; it was quite literally possible to write it as a function that could be dropped into any codebase without having to make any other changes to ensure it was used safely.
What about just doing something like
#[cfg(any(target_pointer_width = "32", target_pointer_width = "64"))]
#[inline(always)]
const fn usize_to_u32(x: usize) -> u32 {
x as u32
}
Even though I can visually verify that it's safe in this context, I really don't like casting integers as a rule when there's a reasonable alternative. The solution I came up with is pretty much equally readable in my opinion but has the distinction of not having code that might in other contexts look like it could silently have issues (compared to an `unreachable!()` macro which might also look sketchy but certainly wouldn't be quiet if it accidentally was used in the wrong spot). I also prefer having a compiler error explaining the invariant that's expected rather than a missing function (which could just as easily be due to a typo or something). You could put a `compile_error!()` invocation to conditionally compile when the pointer width isn't at least 32, but I'd argue that tilts the balance even more in favor of the solution I came up with; having a single item instead of two defined is more readable in my opinion.
This wasn't a concern for me but I could also imagine some sort of linting being used to ensure that potentially lossy casts aren't done, and while it presumably could be manually suppressed, that also would just add to the noisiness.
Oops, I flipped the conversion here, that should have been `u32_to_usize()`.
For systems where correctness is actually important, not just a nice-to-have (in most systems, it's nice-to-have,) we have had an increasing number of options over the years.
From tools like "spin" and "tla+" to proof assistants like Coq to full languages like Idris and Agda.
Some of the stronger-typed languages already give us some of those benefits (Haskell, OCaml) and with restricted effects (like Haskell) we can even make the compiler do much of this work without it leaking into other parts of the program if we don't want it to.
While I would love this to be true, I'm not sure that this design can statically prove anything. For an assert to fail, you would have to actually execute a code sequence that causes the invariant to be violated. I don't see how the compiler could prove at compile time that the invariants are upheld.
An assert is just a fancy `if condition { panic(message) }`. If the optimizer can show that condition is always false, the panic is declared as dead code and eliminated. The post uses that to get the compiler to remove all panics to reduce the binary size. But you can also just check if panic code was generated (or associated code linked in), and if it was then the optimizer wasn't able to show that your assert can't happen.
Of course this doesn't prove that the assert will happen. You would have to execute the code for that. But you can treat the fact that the optimizer couldn't eliminate your assert as failure, showing that either your code violates the assert, your preconditions in combination with the code aren't enough to show that the assert isn't violated, or the whole thing was too complicated for the optimizer to figure out and you have to restructure some code
Ah, I see what you are saying. Yes, if the optimizer is able to eliminate the postcondition check, I agree that it would constitute a proof that the code upholds the invariant.
The big question is how much real-world code the optimizer would be capable of "solving" in this way.
I wonder if most algorithms would eventually be solvable if you keep breaking them down into smaller pieces. Or if some would have some step of irreducible complexity that the optimizer cannot figure out, now matter how much you break it down.
I wonder if there is a simple way to stop compilation quickly, with a readable error message, if the optimizer isn't able to eliminate the check.
edit: There is https://github.com/dtolnay/no-panic
> Functions that require some amount of optimization to prove that they do not panic may no longer compile in debug mode after being marked #[no_panic].
So you’re probably going to have to protect this with a cfg_attr to only apply the no_panic in release only.
Probably, yes.
That or forcing compilation to always target release.
To be clear, the optimizer doesn’t uphold the post condition. It just says “the post condition is usable to elide other checks”. But if the condition itself is incorrect, the compiler will STILL elide those checks. If the compiler could prove the condition on its own it would have done so. That’s why that assert is named unchecked and is itself unsafe!
Generally you run into the halting problem and whatnot real quick.
To be clear as there’s a lot of nuance. Assert unchecked is telling the compiler the condition must always hold. The optimizer and compiler don’t make any assumption about the assert. That information is then used by the compiler to optimize away checks it otherwise would have to do (eg making sure an Option is Some if you call unwrap).
If you have an assumption that gives unhelpful information, the optimizer will emit panic code. Worse, if the assumption is incorrect, then the compiler can easily miscompile the code (both in terms of UB because of an incorrectly omitted panic path AND because it can miscompile surprising deductions you didn’t think of that your assumption enables).
I would use the assume crate for this before this got standardized but very carefully in carefully profiled hotspots. Wrapping it in a safe call as in this article would have been unthinkable - the unsafe needs to live exactly where you are making the assumption, there’s no safety provided by the wrapper. Indeed I see this a lot where the safety is spuriously added at the function call boundary instead of making the safety the responsibility of the caller when your function wrapper doesn’t actually guarantee any of the safety invariants hold.
> Wrapping it in a safe call as in this article would have been unthinkable - the unsafe needs to live exactly where you are making the assumption, there’s no safety provided by the wrapper.
I want to be sure I understand your meaning. In your analysis, if the check_invariant function was marked unsafe, would the code be acceptable in your eyes?
Unsafe is the P0. I'd avoid this approach wholesale UNLESS it was a critical hot path with no other way to get safe Rust to elide the check (hint - you'd be surprised how much gets elided in idiomatic Rust). Basically, this is a nice sharp knife in the toolbox to know about and so sharp I'd rarely use it.
I've had an unpleasant amount of crashes with Rust software because people are way too quick to grab `panic!` as an out.
This was most shocking to me in some of the Rust code Mozilla had integrated into Firefox (the CSS styling code). There was some font cache shenanigans that was causing their font loading to work only semi-consistently, and that would outright crash this subsystem, and tofu-ify CJK text entirely as a result.
And the underlying panic was totally recoverable in theory if you looked at the call stack! Just people had decided to not Result-ify a bunch of falliable code.
Sometimes the program is in an invalid state. You don't want to keep running the program. Better to fail spectacularly and clearly then to fail silently and try to hobble along.
The thing with functional programming (specifically, immutable data,) is that as long as the invalid state is immutable, you can just back up to some previous caller, and they can figure out whether to deal with it or whether to reject up the its previous caller.
This is why Result (or Maybe, or runExceptT, and so on in other languages) is a perfectly safe way of handling unexpected or invalid data. As long as you enforce your invariants in pure code (code without side effects) then failure is safe.
This is also why effects should ideally be restricted and traceable by the compiler, which, unfortunately, Rust, ML, and that chain of the evolution tree didn't quite stretch to encompass.
Say a function has some return type Result<T, E>. If our only error handling mechanism is Err(e) then were restricted to E representing the set of errors due to invalid arguments and state, and the set of errors due to the program itself being implemented incorrectly.
In a good software architecture (imo) panics and other hard failure mechanisms are there for splitting E into E1 and E2, where E1 is the set of errors that can happen due to the caller screwing up and E2 being the set of errors that the caller screwed up. The caller shouldn't have to reason about the callee possibly being incorrect!
Functional programming doesn't really come into the discussion here - oftentimes this crops up in imperative or object oriented code where function signatures are lossy because code relies on side effects or state that the type system can't/won't capture (for example, a database or file persisted somewhere). Thats where you'll drop an assert or panic - not as a routine part of error handling.
You shouldn't pass invalid values to a function. If a function can return some sensible value for some input, then the input is not invalid - even if the return type is an error by name.
Ideally, you can constrain the set of inputs to only valid ones by leveraging types. But if that's not possible and a truly invalid input is passed, then you should panic. At least that's the mental model that Rust is going with.
You do lose out on the ability to "catch" programming errors in subcomponents of your program. For example, it's extremely useful to catch exceptions related to programming errors for called code in response to a web request, and return a 500 in those cases. One could imagine a "try" "catch" for panics.
The thing is, it takes a lot of discipline by authors to not riddle their code with panics/exceptions when the language provides a try/catch mechanism (see C# and Java), even when a sensible error as value could be returned. So Rust opts to not introduce the footgun and extra complexity, at the expense of ungraceful handling of programming errors.
> Ideally, you can constrain the set of inputs to only valid ones by leveraging types. But if that's not possible and a truly invalid input is passed, then you should panic.
But how can the caller know what is "a truly invalid input"? The article has an example: "we unfortunately cannot rely on panic annotations in API documentation to determine a priori whether some Rust code is no-panic or not."
It means that calling a function is like a lottery: some input values may panic and some may not panic. The only way to ensure that it doesn't panic is to test it with all possible input values, but that is impossible for complex functions.
It would be better to always return an error and let the caller decide how to handle it. Many Rust libraries have a policy that if the library panics, then it is a bug in the library. It's sad that the Rust standard library doesn't take the same approach. For println!(), it would mean returning an error instead of panicking.
The program can detect invalid state, but your intention was to never get to that state in the first place. The fact that the program arrived there is a Logic error in your program. No amount of runtime shenanigans can repair it because the error exists without your knowledge of where it came from. You just know it's invalid state and you made a mistake in your code.
The best way to handle this is to crash the program. If you need constant uptime, then restart the program. If you absolutely need to keep things running then, yeah try to recover then. The last option isn't as bad for something like an http server where one request caused it to error and you just handle that error and keep the other threads running.
But for something like a 3D video game. If you arrive at erroneous state, man. Don't try to keep that thing going. Kill it now.
> The program can detect invalid state, but your intention was to never get to that state in the first place. The fact that the program arrived there is a Logic error in your program. No amount of runtime shenanigans can repair it because the error exists without your knowledge of where it came from.
True: a program can't fix an internal assertion error, and the failed component might not be recoverable without a reset. But that doesn't means that the whole program is doomed. If the component was optional the program might still work although with reduced functionality. Consider this: way you do not stop the whole computer if a program aborts.
As I mentioned elsethread, in unsafe languages an assertion error might, with high probabilty, be due to the whole runtime being compromised, so a process abort is the safest option.
> The fact that the program arrived there is a Logic error in your program.
No, your program correctly determined that user input was invalid.
Or your parser backtracked from parsing a Bool and decided to try to parse an Int instead.
That’s not invalid state. Your program correctly determined input is invalid.
Say user input is a number and can never exceed 5. If the user input exceeds 5 your program should handle that gracefully. This is not invalid state. It is handling invalid input while remaining in valid state.
Let say it does exceed 5 and You forget to check that it should never exceeds 5 and this leads to a division by zero further down your program. You never intended for this to happen and you don’t k ow why this happened. This is invalid state.
Now either this division by zero leads to an error value or it can throw an exception. It doesn’t matter. Do you know how to recover? You don’t even know where the bug is. A lot of times your don’t even know what to do. This error can bubble up all the way to main and now what do you do with it?
You crash the program. Or you can put in a fail safe before the error bubbles up to main and do something else but now (if your program retains and mutates that state) has invalid values in it and a known bug as well.
Imagine that input number represented enumeration values for movement of some robot. You now have one movement that doesn’t exist or was supposed to be something else. Thus if you keep your program running the robot ends up in an unexpected and erroneous place. That’s why I’m saying a program should crash. It should not be allowed to continue running with invalid state.
Like imagine if this was a mission critical auto pilot and you detect negative 45 meters for altitude. Bro crash and restart. Don’t try to keep that program running by doing something crazy in attempt to make the altitude positive and correct. Reset it and hope it never goes to the invalid state again.
> You crash the program.
No thank you.
> Or you can put in a fail safe before the error bubbles up to main and do something else
In other words,
>> your parser backtracked from parsing a Bool and decided to try to parse an Int instead
> but now (if your program retains and mutates that state) has invalid values in it and a known bug as well.
Unless,
>>> The thing with functional programming (specifically, immutable data,) is that as long as the invalid state is immutable, you can just back up to some previous caller, and they can figure out whether to deal with it or whether to reject up the its previous caller.
Your parser backtracking from a bool when it tried to parse an int is VALID behavior. Your parser tries to parse text into several possible types at runtime. Backtracking is STILL valid state. You’re not thinking clearly.
Look at my auto pilot example. You have a bug in your program you don’t know about. Altitude reads -9999 meters your heading and direction is reading at the speed of light. Your program sees these values and recognizes invalid state. There is a BUG in your program. Your program WAS never designed to go here.
You want to try to recover your autopilot program? How the fuck are you gonna do that? You don’t even know where the bug is. Does your recovery routine involve patching and debugging? Or are you gonna let your autopilot keep operating the plane with those nonsense values? That autopilot might hit the wind breaks to try to slow down the plane and end up crashing the thing.
You don’t recover this. You restart the program and pray it doesn’t hit that state again then when you are on the ground you debug it.
>>> The thing with functional programming (specifically, immutable data,) is that as long as the invalid state is immutable, you can just back up to some previous caller, and they can figure out whether to deal with it or whether to reject up the its previous caller.
This makes no fucking sense. Where the hell will you back up to? Your error came from somewhere but you don’t know where. Return an error value in the child function the parent function receives the error and returns an error itself and this keeps going until you bubble up to the main function because you don’t know where that error came from.
Now you have an error in the main function. Wtf are you gonna do? How do you handle an error in the main function that you have no idea where it came from? Here’s an idea. You have the main function restart the program loop. See a similarity here? It’s called crashing and restarting the program. Same effect!
This isn’t a functional programming versus non functional thing. It’s a concept that you’re not seeing.
I know this is not your point for the last paragraph, but have you read about the C-130 complete navigation system failure while trying land below sea level at the Dead Sea? :)
good one.
> This is why Result (or Maybe, or runExceptT, and so on in other languages) is a perfectly safe way of handling unexpected or invalid data.
They are great for handling expected errors that make sense to handle explicitly.
If you try to wrap up any possible error that could ever happen in them you will generate horrendous code, always having to unwrap things, everything is a Maybe. No thanks.
I know it is tempting to think "I will write the perfect program and handle all possible errors and it will never crash" but that just results in overly complex code that ends up having more bugs and makes debugging harder. Let it crash. At the point where the error happened, don't just kick the bucket down the road. Just log the the problem and call it a day. Exceptions are an amazing tool to have for things that are.. exceptions.
The monad and lifting fixes this problem of having to unroll the maybe type. But this is an advanced abstraction.
I actually disagree with this. Use the maybe type religiously, even without the monad because it prevents errors via exhaustive matching.
The exception should only be used if your program detects a bug.
I understand this belief abstractly. In the cases I was hitting, there would have been easy recovery mechanisms possible (that would have been wanted because there are many ways for the system to hit the error!), but due to the lowest level "key lookup" step just blowing up rather than Result (or Option)-ing their lookup, not only would the patch have been messy, but it would have required me to make many decisions in "unrelated" code in the meanwhile.
I understand your point in general, I just find that if you're writing a program that is running on unconstrained environments, not panic'ing (or at least not doing it so bluntly at a low level) can at the very least help with debugging.
At least have the courtesy to put the panic at a higher level to provide context beyond "key not found!"!
Without knowing the exact situation, if you follow the guidelines in this article, this is a library bug (documentation or actual code).
Either the library should have enforced the invariant of the key existing (and returned an equivalent error, or handled it internally), or documented the preconditions at a higher level function that you could see.
This is correct in a memory unsafe language like C, where you can't make any assumption on the state of the program, so the only safe escape is to the address separation boundary (i.e. the program). In a safe language you can usually make reasonable assumptions [1] regarding the blast radius of an assertion violation. And that's why rust allows catching panics.
[1] even safe languages have unsafe escape hatches and safe abstractions sometimes have bugs, but on most cases you can assume that the runtime is not compromised.
At least sources of panic! are easily greppable. Cutting corners on error handling is usually pretty obvious
I don't think grepping for panics is practical, unless you are trying to depend on exclusively no-panic libraries.
Even if you are no_std, core has tons of APIs like unwrap(), index slicing, etc. that can panic if you violate the preconditions. It's not practical to grep for all of them.
There is panic-analyzer [1] that searches for code that needlessly panics. You can also use the no-panic macro [2] to turn possible panics in a specific function (including main) into a compile error
Panic-analyzer looks like it is based on heuristics, searching for known-panicing APIs. I tried it on a workspace that uses io::stdout() and it did not flag this as potentially panicing.
No-panic looks nifty: it appears to be reliable, which is great. I wish there was an easy way to automatically apply this annotation to every single function in a given file or crate.
I think the article is wrong in that std::io::stdout would be panicking, or that "the panic is reachable somehow". It's just the optimizer doesn't see it doesn't panic.
https://doc.rust-lang.org/beta/src/std/io/stdio.rs.html#674-...
The implementation calls indeed a panicking API, OnceLock::get_or_init:
https://doc.rust-lang.org/beta/std/sync/struct.OnceLock.html...
But it only panicks if it is being used in a wrong way, which it isn't. The usage is contained within the implementation of std::io::stdout, so it's an implementation detail.
It's a shame that there are no better ways to eliminate panics in case they are impossible to trigger. The article shows some tricks, but I think the language is missing still some expressability around this, and the stdlib should also thrive harder to actually get rid of hard-to-optimize links to panic runtime in case of APIs that don't actually panic.
If the only problem was OnceLock, the simplest solution would be to replace it with a const initializer.
The main reason getting panic-free stdout is hard, is that the STDOUT initializers both call allocating constructors, and LineWriter doesn't have non-allocating constructors.
Solving this elegantly seems hard.
Thanks for the clarification on OnceLock::get_or_init. It's interesting to know that this is dynamically unreachable, but in a way that the compiler could not statically prove.
I 100% agree with your last paragraph. I would love if the language and stdlib made it easier to make panic-free binaries.
There are panics that aren't greppable that way. For instance `some_array[past_bounds]` causes a panic.
It is interesting to consider how `panic!` serves as some documentation of explicitly giving up. Easy to see in a pull request. And having the string packed alongside it is nice.
Still miffed, but we'll get there.
I mean... rust modules aren't typically in your CWD, no? they're not in some node_modules that you can grep, but in a cargo folder with /all of the libraries you ever used/, not just the ones you have for this one project.
Putting them all in the project root takes just a single `cargo vendor` command.
But I would assume that for mozilla their entire CSS subsystem is pulled in as a git (hg?) submodule or something anyways.
not sure how the CSS subsystem is included (I remember it is shared with Servo?), but in general all of the Rust dependencies in mozilla-central is vendored, so nothing is in your ~/.cargo directory.
For what it's worth, eg vscode can jump to definition even when your code is in a different crate that's not in your repository.
If you run cargo vendor, they end up in a neat directory.
While sure, more things could be baked as results, most of the time when you see a panic that's not the case. It's a violation of the callee's invariants that the caller fucked up.
Essentially an error means that the caller failed in a way that's expected. A panic means the caller broke some contract that wasn't expressed in the arguments.
A good example of this is array indexing. If you're using it you're saying that the caller (whoever is indexing into the array) has already agreed not to access out of bounds. But we still have to double check if that's the case.
And if you were to say that hey, that implies that the checks and branches should just be elided - you can! But not in safe rust, because safe code can't invoke undefined behavior.
I understand the value of panic when your invariants really are no longer holding. What I have seen is many cases of "oh a micro-invariant I kind of half believe to be true isn't being held, and so I will panic".
Obviously context-free this is very hand wave-y, but would you want Firefox to crash every time a website prematurely closes its connection to your browser for whatever reason? No, right? You would want Firefox to fail gracefully. That is what I wanted.
Comment was deleted :(
> fn check_invariant(&self) { > unsafe { assert_unchecked(self.ofs < > self.data.len()) } > }
Is fundamentally unsound `check_invariant` needs to be unsafe as it doesn't actually check the invariant but tells the compiler to blindly assume they hold. Should probably also be named `assume_invariant_holds()` instead of `check_invariant()`.
I personally would have used the assume crate but I guess this got standardized more recently. They call out the safeness requirement and that it’s a sharp edge but like you I think they understate the sharpness and danger.
This seems to obviate a lot of Rust's advantages (like a good std library). I wonder what it would take to write a nopanic-std library?
Panics really seem bad for composability. And relying on the optimzer here seems like a fragile approach.
(And how is there no -nopanic compiler flag?)
Rust doesn't want to add any proof-system that isn't 100% repeatable, reliable, and forwards compatible to the language. The borrow checker is ok, because it meets those requirements. The optimizer based "no panic" proof system is not. It will break between releases as LLVM optimizations change, and there's no way to avoid it.
Trying to enforce no-panics without a proof system helping out is just not a very practical approach to programming. Consider code like
some_queue.push_back("new_value");
process(some_queue.pop_front().unwrap());
Or consider code like
let mut count: usize = 0;
for item in some_vec {
// Do some stuff with item
if some_cond() {
count += 1;
}
}
Or so on.
Basically every practical language has some form of "this should never happen" root. Rust's is panics. C's is undefined behavior. Java's is exceptions.
Finally consider that this same mechanism is used for things like stack overflows, which can't be statically guaranteed to not occur short of rejecting recursion and knowledge of the runtime environment that rustc does not have.
---
Proof systems on top of rust like creusot or kani do tend to try to prove the absence of panics, because they don't have the same compunctions about not approving code today that they aren't absolutely sure they will approve tomorrow as well.
To add to this, I believe that there will always be some amount of "should never happen but I can't prove it" due to Rice's Theorem[1].
> This code is obviously correct. It never panics
It doesn't panic within the code you typed, but it absolutely still can panic on OOM. Which is sort of the problem with "no panic"-style code in any language - you start hitting fundamental constructs that can can't be treated as infallible.
> Basically every practical language has some form of "this should never happen" root.
99% of "unrecoverable failures" like this, in pretty much every language, are because we treat memory allocation as infallible when it actually isn't. It feels like there is room in the language design space for one that treats allocation as a first-class construct, with suitable error-handling behaviour...
Assuming memory is infinite for the purposes of a program is a very reasonable assumption for the vast majority of programs. In the rare contexts where you need to deal with the allocation failure it comes at a great engineering cost.
It's not really what this is about IMV. The vast majority of unrecoverable errors are simply bugs.
A context free example many will be familiar with is a deadlock condition. The programmer's mental model of the program was incomplete or they were otherwise ignorant. You can't statically eliminate deadlocks in an arbitrary program without introducing more expensive problems. In practice programmers employ a variety of heuristics to avoid them and just fix the bugs when they are detected.
Deadlocks also don’t result in panics in most environments. The problem isn’t so much bugs - those can be found and fixed. The problem is more that no_panic in most languages implies no_alloc, and that eliminates most useful code
The standard library is slowly adding non-panicking options. The article shows some of them (like vec.push_within_capacity()) and ignores some others (vec.get_unchecked()). There is still a lot of work to do, but it is an area where a lot of work gets done. The issue is just that a) Rust is still a fairly young language, barely a decade old counting from 1.0 release, and b) Rust is really slow and methodical in adding anything to stdlib because of how hard/impossible it is to reverse bad decisions in the stdlib.
The same article written a couple years in the future would look very different
To be slightly pedantic, Vec::get() is the non-panicking version. Vec::get_unchecked() is just the version thereof that elides the bounds check.
What I would like to see is a reliable distinction of different types of panics. In the environments where software I write is typically run, panics due to heap allocation failure are generally acceptable and rarely an indication of fragility. (By the time a heap allocation failure occurs, the computer is probably already thrashing and needs to be rebooted.) On the other hand, other kinds of panics are a bad sign. For example, I would frown on any library that panics just because it can't reach the Internet.
In other environments, like embedded or safety-critical devices, I would need a guarantee that even heap allocation failure can not cause a panic.
> Unrecoverable
panics are very much designed to be recoverable at some well defined boundaries (e.g. the request handler of a web server, a thread in a thread pool etc.)
this is where most of it's overhead comes from
you can use panic=abort setting to abort on panics and there is a funny (but unpractical) hack with which somewhat can make sure that no not-dead-code-eliminated code path can hit a panic (you link the panic->abort handler to a invalid symbol)
I would say that you are using them incorrectly if you assume them as recoverable. You should make everything you can so that they never happen.
However, since it is still possible to have them in a place where the exiting the process is not okay, it was beneficial to add a way to recover from them. It does not mean that they are designed to be recoverable.
> this is where most of it's overhead comes from
Overhead comes from the cleaning process. If you don't clean properly, you might leak information or allocate more resources than you should.
> I would say that you are using them incorrectly if you assume them as recoverable.
no it's them being recoverable at well defined boundaries is a _fundamental_ design aspect of rust
> Overhead comes from the cleaning process. If you don't clean properly, you might leak information or allocate more resources than you should.
and that same cleanup process makes it recoverable
Even the book uses the word "unrecoverable". The wording is communication. The intention is not to recover them, while it could be possible.
https://doc.rust-lang.org/book/ch09-01-unrecoverable-errors-...
[dead]
There is a social norm to treat panics as unrecoverable (in most cases — some do use panics to perform cancellation in non-async code).
they are still anyway designed to be recoverable as a fundamental aspect of rust
so that e.g. you web server doesn't fall over just because one request handler panics
and it still is fundamental required that any code is safe in context of panic recovery (UnwindSafe is misleading named and actually not about safety, anything has to be "safe" in context of unwind, UnwindSafe just indicates that it's guaranteed to have sensible instead of just sound behavior in rust)
people over obsessing with panic should be unrecoverable is currently IMHO on of the biggest problems in rust not in line with any of it's original design goals or how panics are in the end design to work or how rust is used in many places in production
yes they are not "exceptions" in the sense that you aren't supposed to have fine grained recoverablility, but recoverability anyway
without recoverable panics you wouldn't be able to write (web and similar) servers in a reasonable robust way in rust without impl some king of CGI like pattern, which would be really dump IMHO and is one of the more widely used rust use cases
At Oxide we ship our binaries, including our HTTP servers, with panic = abort, fwiw.
I know some do.
But as long as you don't do anything unusual you are basically introducing a (potentially huge) availability risk to your service for no reason but except not liking panics.
Like it's now enough for there to be a single subtle bug causing a panic to have a potentially pretty trivially exploitable and cheap DoS attack vector. Worse this might even happen accidentally. Turning a situation where some endpoints are unavailable due to a bug into one where your servers are constantly crashing.
Sure you might gain some performance, but for many use cases this performance is to small to reason in favor of this decisions.
Now if you only have very short lived calls, and not too many parallel calls in any point in time, and anyway spread scaling across many very very small nodes it might not matter that you might kill other requests, but once that isn't the case it seems to most times be a bad decision.
It also doesn't really add security benefits, maybe outside of you having very complicated in memory state or similar which isn't shared across multiple nodes through some form of db (if it is, you anyway have state crossing panics, or in your case service restarts).
Well, no it's not just not liking panics — it's that panics can leave memory in an inconsistent state. std::sync::Mutex does poisoning, but many other mutexes don't. And beyond that, you could also panic in the middle of operating on an &mut T, while state is invalid.
Tearing down the entire process tends to be a pretty safe alternative.
> but many other mutexes don't.
yes and that's most times the better design decision
> operating on an &mut T, while state is invalid.
but you normally don't and for many use cases is in my experiment a non-issue
panic recovery isn't fine grained and passing `&mut T` data across recovery boundaries is bothersome
at the same time most of the cross request handler in-memory state is stuff like caches and similar which as long as you don't hand role them should work just fine with panic recovery
At the same time deciding to not recover some kinds of shared state and recreate them isn't hard at all, at least assuming no spaghetti code with ton's of `static` globals are used.
And sure there are some use cases where you might prefer to tear down the process, but most web-server use cases aren't anywhere close to it in my experience. I can't remember having had a single bug because of panic recovery in the last like ~8?? years of using it in production (yes a company I previous worked for started using it in production around it's 1.0 release).
EDIT: Actually I correct myself 2 bugs. One due to Muxtex poison which wouldn't have been a problem if the Muxtex didn't had poison. And another due to a certain SQL library insisting of not fixing a long standing bug by insisting on reexport a old version of a crate known to be broken which had been fixed upstream because they didn't like the way it was fixed and both not documenting that you can't use it with panic recovery and closing all related issues because "if you use panics you are dump".
But both of them where like 6-8 years ago.
This website makes by browser freeze... No idea why. Not able to read the article.
Author here -- that is surprising. What browser/OS are you on? I haven't had anyone else report this problem before.
I’m seeing the same problem, the page crashes on Safari on iOS, saying a problem repeatedly occurred. Haven’t seen a webpage do that in quite a while.
Yep, same experience, same platform. I guess straight to reader mode, it is.
EDIT - shockingly, reader mode also fails completely after the page reloads itself
I'm seeing the same problem--the page crashes on mobile (Brave). On desktop it loads, but all of the code cells have a crashed page symbol in them.
Same here: Chrome on a Google Pixel 4a. Page freezes while scrolling down and eventually oh-snaps.
I'm also seeing this on Android Chrome. When I opened the page on my Linux desktop, I also saw the crashes (though they only affected the godbolt iframes).
Note that on Android process separation is not usually as good, so a crashing iframe can bring down the whole page.
Same issue on Android using Brave.
same on chrome android
same for me. Chrome on Pixel 8
This happened to me too (Safari on iOS).
Here’s a archived link:
https://web.archive.org/web/20250204050500/https://blog.reve...
Same, the web view in my Android client crashed after a couple seconds.
I wonder if it's all the Godbolt iframes. Do you have the same problem on other pages, like https://blog.reverberate.org/2025/01/27/an-ode-to-header-fil... ?
Yeah, I think it's all those iframes. I'm seeing something weird on my Linux desktop - all the godbolt iframes crash on reload unless I have another tab with godbolt open. I didn't see anything obvious in Chrome's log.
I can't replicate the crash at all on my Linux cloud VM though. Usually the only difference there is that advertisers tend to not buy ads for clients on cloud IPs.
Other pages on the site work fine for me yeah. But the OP blog post is crashing my Android browser, like the other commenters have mentioned.
Yeah no problem on other pages
OP sounds like they'd be very interested in Zig to tackle this particular problem. they'd get to a very similar place and not have to fight the language or the standard library to get there.
This blog post is very interesting, using Rust’s compiler optimizer as a theorem prover. This makes me wonder: are there any formal specifications on the complexity of this "optimizer as theorem prover"?
Specifically, how does it handle recursion? Consider, for example, the following function, which decrements a number until it reaches zero. At each step, it asserts that the number is nonzero before recursing:
fn recursive_countdown(n: u32) -> u32 { assert!(n > 0, "n should always be positive"); if n == 1 { return 1; } recursive_countdown(n - 1) }
Can the compiler prove that the assertion always holds and possibly optimize it away? Or does the presence of recursion introduce limitations in its ability to reason about the program?
> This makes me wonder: are there any formal specifications on the complexity of this "optimizer as theorem prover"?
Basically, the promise here "We formally promise not to promise anything other than the fact that optimized code should have 'broadly' the same effects and outputs as non-optimized code, and if you want to dig into exactly what 'broadly' means prepare to spend a lot of time on it". Not only are there no promises about complexity, there's no promises that it will work the same on the same code in later versions, nor that any given optimization will continue firing the same way as you add code.
You can program this way. Another semi-common example is taking something like Javascript code and carefully twiddling with it such that a particular JIT will optimize it in a particular way, or if you're some combination of insane and lucky, multiple JITs (including multiple versions) will do some critical optimization. But it's the sort of programming I try very, very hard to avoid. It is a path of pain to depend on programming like this and there better be some darned good reason to start down that path which will, yes, forever dominate that code's destiny.
Compilers can typically reason fairly well about tail recursion like this. In this case the compiler cannot remove the assertion because you could pass in 0. But if you change the assert to be a >= 0 (which is vacuously true, as the warning indicates) it will optimize the code to "return 1" and eliminate the recursive call: https://godbolt.org/z/jad3Eh9Pf
Does anyone know if there's an obvious reason that adding a `no_panic` crate attribute wouldn't be feasible? It certainly seems like an "obvious" thing to add so I'm hesitant to take the obvious nerd snipe bait.
The standard library has a significant amount of code that panics, so a `no_panic` crate attribute would currently only work for crates that don't depend on the standard library. I imagine most interesting crates depend on the standard library.
What caught my eye in the article was the desire to have something that doesn't panic with a release profile, while allowing for panics in dev profiles. Based on other comments I think the general "allow use of std, but don't panic" seems like something that could be useful purely on the "Wait, why doesn't that exist?" reactions.
You could do it, but I would prefer guarantees on a per-call chain basis using a sanitizer. It should be quite easy to write.
I'm no rustc expert, but from what little I know it seems like disabling panics for a crate would be an obvious first step. You make a great point though. Turning that into a compiler assertion of "this function will never panic" would also be useful.
It’s a good first step, but half of the crates in crates.io have at least 40 transitive dependencies. Some have hundreds or thousands. A big effort.
It should be possible to write a sanitizer which verifies no panic behavior on a call graph, just as you can to verify no blocking, or no races.
Why worry about the code size if the code size is up to the library consumer (through their choice of panic handler)? If the consumer worries about code size, then their application has a minimal panic handler. If the consuming application does not have a minimal panic handler, then it must not worry about code size?
Is there some context I'm missing here? Is this to be used from non-Rust applications for example?
As the author mentions, `panic!` is also not an acceptable failure mode in some applications. If you're developing safety-critical software and a process stopping is part of your risk analysis, many frameworks will ask you about the frequency of that happening. In that analysis, you may be required to set all systematic contributions to that frequency to zero. This happens, for example, if you try to control the associated risk using redundancy. If there is a code path that may panic, you may not be able to do this at all as you maybe just cannot conclude that your code does not panic systematically.
Yes that condition I understand. But that seems orthogonal to the code size issue. Having no panics in code where the stdlib is riddled with panics for exceptional situations (allocation failure, for example) seems like a situation where you would just always go with no_std?
It is orthogonal, yes. To your question, I have an example from the same domain, where it is reasonable to mix unrolling panic with code that never panics.
Typically, safety-related processes are set up in two phases. First they set up, then they indicate readiness and perform their safe operation. A robot, for example, may have some process checking the position of the robot against a virtual fence. If the probability for passing through that fence passes some limit, this requires the process to engage breaks. The fence will need to be loaded from a configuration, communication with the position sensors will need to be established, the fence will generally need to be transformed into coordinates that can be guaranteed to be checked safely, taking momentum and today's breaking performance in account, for example. The breaks itself may need to be checked. All that is fine to do in an unsafe state with panics that don't just abort but unroll and full std. Then that process indicates readiness to the higher-level robot control process.
Once that readiness has been established, the software must be restricted to a much simpler set of functions. If libraries can guarantee that they won't call panic!, that's one item off our checklist that we can still use them in that state.
Most people are using a prebuild standard library. That comes with the problem that it comes with the features it was build for. Most of the bloat around panic for example can be eliminated by just compiling the std library yourself. This is done via the `-Zbuild-std` flag.
Using this flag one than can use `panic_abort`. This will eliminate the unwinding part but would still give a "nice" printout on a panic itself. This reduces, in most cases, the mention bloat by a lot. Though nice printouts also cost binary space. For eliminating that `panic_immidiate_abort` exists.
But yeah the above is only about bloat and not the core goal to eliminate potential path's in your program, that would lead to a panic condition itself.
Also currently building the std library yourself needs a nightly compiler. There is afaik work on bringing this to a stable compiler but how exactly is still work in progress.
Does Rust have something like a deep-codemodding macro that could be used to un-panic-fy an entire function etc. automatically?
Something like: Given a function, rewrite its signature to return a Result if it doesn't already, rewrite each non-Resulty return site to a Some(), add a ? to every function call, then recurse into each called function and do the same.
It has `catch_unwind` [1], but that still retains the panicking runtime, so not sufficient in the context of the post.
[1] https://doc.rust-lang.org/std/panic/fn.catch_unwind.html
It's also not guaranteed to catch every panic - sometimes (notably if a destructor panics during unwinding) a panic can turn into a process-abort.
To add to that, Rust code is generally not written to be 'exception-safe' when panics occur: if a third-party function causes a panic, or if your own code panics from within a callback, then memory may be leaked, and objects in use may end up in an incorrect or unusable state.
You really want to avoid sharing mutable objects across a catch_unwind() boundary, and also avoid using it on a regular basis. Aside from memory leaks, panicking runs the thread's panic hook, which by default prints a stacktrace. You can override the panic hook to be a no-op, but then you won't see anything for actual panics.
While not as strict, you can filter lints by their description and apply a policy to your crate https://rust-lang.github.io/rust-clippy/rust-1.84.0/index.ht...
> Protocol Buffers
Instead of serializing data (to disk, not the network), it would be much faster if Rust allowed us to allocate datastructures directly in an mmapped file, and allowed us to read back the data (basically patching the pointers so they become valid if the base address changed).
This is basically what Cap'n Proto does, and it uses offsets instead of pointers so that way the mmapped data can be used as-is.
Good to know, but does Rust have the machinery to make it work with standard Rust data structures?
What do you mean by "standard Rust data structures"? Like protobuf and other such formats, capnp uses code generation. The tooling for Rust is at https://github.com/capnproto/capnproto-rust and the generated code gives you structs that wrap borrowed data and give you accessors rather than direct field access.
Well, it would be super convenient if you could use the standard collections like Vec, LinkedList, HashMap, etc. just like you would use them normally except now they are allocated inside an mmapped file.
This page reliably crashes Chrome on my Android phone as a well as on my Ubuntu laptop.
Comment was deleted :(
Perhaps verus would be helpful to the op: https://github.com/verus-lang/verus
> not panic'ing (or at least not doing it so bluntly at a low level) can at the very least help with debugging.
Wouldn’t panicking asap make debugging easier?
> If we are trying to port a C library to Rust, we really do not want to introduce panics in the code, even for unusual error condition.
"We'd much rather like to make library to corrupt the memory of the rest of the application and generally make the demons fly out of the users' noses, as it does when written in C"?
I believe implementations of C stdio also can abort on program startup if somehow the pthreads' locking mechanism is broken (or if e.g. fcntl(2)/open(2) keeps returning -1), Rust is not that unique in this regard.
This website panics on my iOS version 18.1.1 in Safari. Is anyone else having issues on mobile?
Funnily enough this website hard-crashes Android Chrome and DuckDuckGo browsers after loading.
This seems..absurd for a programming language with goals like Rust. Why isn't this a compiler option? Just set -nopanics and the compiler errors and flags anything which is pulling in a panic at the very least?
You can set panic: abort [1] if you don't want the unwinding mechanism. You still get a nice error message on panic, which causes the compiler to link in some formatting code from the standard library. I'm not sure if you can get rid of that.
On the same page are also the options for controlling debug assertions and overflow checks, which would get rid of the "panics in debug, but not release", if that behavior bugs you
1: https://doc.rust-lang.org/cargo/reference/profiles.html#pani...
https://doc.rust-lang.org/book/ch09-01-unrecoverable-errors-...
``` [profile.release] panic = 'abort' ```
that just makes the panics unrecoverable. It doesn't statically guarantee no panics.
It presumably avoids the linked in 300Kb that was supposedly part of the motivation for doing this though?
There is no-panic [1] to turn panics in a function into compile errors, or the older no-panics-whatsoever [2] to do the same for the entire binary
no-panic uses link-time shenanigans to prevent panics in the compiled binary, but this isn't 100% reliable (as explained in the README, just pointing this out)
After using rust for several years, I'm shocked this isn't the default. Making panics recoverable leads to them being used incorrectly.
Panics also unwind the stack and run all your Drop destructors.
Setting `panic = abort` disables unwinding and this means leaking memory, not closing file descriptors, not unlocking mutexes and not rolling back database transactions on panic. It's fine for some applications to leave this to the operating system on process exit but I would argue that the default unwinding behavior is better for typical userspace applications.
All the things you describe are done automatically on program exit, even if the program is SIGKILL’ed. The kernel cleans up file descriptors, database transactions are rolled back automatically when the client disconnects (which it should be observed to be when the program exits and the kernel closes the connection as part of cleanup), and I’m not sure what you mean about mutexes, but if you mean in-memory ones, those don’t matter because the program is gone (if you mean like, file-based ones, those also should be implicitly unlocked by the kernel when the program exits, at least that’s how a good implementation is supposed to work, e.g. by writing the pid to the file or something.)
The whole of modern operating systems are already very familiar with the idea of programs not being able to exit gracefully, and there’s already a well understood category of things that happen automatically even if your program crashes ungracefully. Whole systems are designed around this (databases issuing rollbacks when the client disconnects, being a perfect example.) The best thing to do is embrace this and never, ever rely on a Drop trait being executed for correctness. Always assume you could be SIGKILLed at any time (which you always can. Someone can issue a kill -9, or you could get OOM killed, etc.)
I'm well aware of this and good that the option exists to bail out with abort instead.
But there are still cases where you would like to fsync your mmaps, print out a warning message or just make sure your #[should_panic] negative tests don't trigger false positives in your tooling (like leak detectors or GPU validators) or abort the whole test run.
It's not perfect by any means but it's better than potentially corrupting your data when a trivial assert fires or making negative tests spew warnings in ci runs.
It's very easy to opt out from, and I don't consider the price of panic handlers and unwinding very expensive for most use cases.
Right, sorry for the patronizing tone, I’m sure you know all this.
But I tend to lament the overall tendency for people to write cleanup code in general for this kind of thing. It’s one of those “lies programmers believe about X” kinds of scenarios. Your program will crash, and you will hit situations where your cleanup code will not run. You could get OOM killed. The user can force quit you. Hell, the power could go out! (Or the battery could go dead, etc.)
Nobody should ever write code that is only correct if they are given the opportunity to perfectly clean up after any failure that happens.
I see this all the time: CLI apps that trap Ctrl-C and tell you you can’t quit (yes I bloody well can, kill -9 is a thing), apps which don’t bother double checking that the files they left behind on a previous invocation are actually still used (stale pid files!!!), coworkers writing gobs of cleanup code that could have been avoided by simply doing nothing, etc etc.
Well, if they did that, then people could expect/demand stability with regard to what scenarios get the checks/panics optimized out. This would be a bit of a burden for the Rust maintainers. It would effectively make the optimizer part of the language specification, and that's undesireable.
Yeah, I'm fairly sure that there is such a flag/toplevel attribute... and if there isn't, there should be one.
It also feels like most of the pains on avoiding panics centers around allocations which, though a bit unfortunate, makes sense; it was an intentional design choice to make allocations panic instead of return Results, because most users of the language would probably crash on allocation fails anyways and it would introduce a lot of clutter. There was some effort some while ago on having better fallible allocations, but I'm not sure what happened over there.
It's sad. If anything panics in the stdlib should be opt-in, not opt-out.
[dead]
Crafted by Rajat
Source Code