hckrnws

I see a lot of discussion in this thread stemming from some confusion+not reading the actual report[0].

Some key points:

1. The Camera+Card was encased in a separate enclosure made of titanium+sapphire, and did not seem to be exposed to extreme pressures.

2. The encryption was done via a variant of LUKS/dm-crypt, with the key stored on the NVRAM of a chip (Edited; not in TrustZone).

3. The recovery was done by transplanting the original chip onto a new working board. No manufacturer backdoors or other hidden mechanisms were used.

4. Interestingly, the camera vendor didn't seem to realize there was any encryption at all.

[0] https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&Fi...

Unless I misread the article, the key was stored in the NVRAM and not the TrustZone.

IIRC, the article stated that if the key(s) had been stored in the TrustZone then the data would have been irrecoverable.

Good catch; it was somewhat ambiguous in the report.

If the encryption was that easy to bypass, was it worth it at all?

The manufacturer didn’t even know encryption was enabled, because as long as the camera was working, it would just provide all files over USB without any encryption.

It was basically enabled by accident, and the only thing it prevented was recovery of files directly from the SD card when the camera was damaged.

There are some reasons you'd want to encrypt even without a secret key. One is it makes it easier to erase data (just erase the key).

It also makes bit flip errors a lot more obvious, which is another way of saying harder to ignore, so that can go either way.

Can't bit flip errors also destroy encrypted volumes much more easily?

I think it depends. Encrypted filesystems typically encrypt contents of each file separately - that way you don't need to read / write the whole disk to read it write any individual file contents. Of course that means metadata may be in plain text or may be separately encrypted - again possibly folder by folder instead of all metadata at once. Exact details would vary with different file system encryption schemes.

Whereas if you image the disk and encrypt the image properly, that gives you all the great confidentially guarantees but no random access.

> Encrypted filesystems typically encrypt contents of each file separately - that way you don't need to read / write the whole disk to read it write any individual file contents.

Ah, that's not true of "full disk encryption". It usually encrypts the disk blocks.

File-based encryption is stronger; you can use different protection classes on different files, you can use authenticated encryption, etc. iOS does it this way and I assume other systems have caught up, but don't know any in particular.

Sure. If the card was recovered without the camera motherboard then the decryption key would not have been recovered.

Stealing a camera is much harder than stealing an SD card out of a camera.

Citation needed. It might be slightly easier, but most cases where you can get part of the camera, you can get the whole camera. This isn't a little point-and-click with a handy spring-loaded slot either.

Yeah but the Camera's owner is much more likely to notice "my camera is missing" than "the SD card is blank for some reason... the SD card must have failed"

EDIT: The linked PDF has a photo, the camera literally opens up to access the SD card.

The camera's (former) owner may very well notice, but that will have little effect. It's much more common that cameras (security, photography, phones) get stolen with cards inside, rather than someone extracting the card and leaving the camera.

Worth mentioning that I would immediately know if a different SD card was in my camera the moment I turned it on or ejected the card. If somebody knew to buy the same exact model and storage size that would be truly impressive.

0. They were too cheap to use an industrial grade SD. Mind boggling.

If you read the article, the SD card was placed there by the camera manufacturer and then the device was sealed so it would withstand pressure, and then sold to divers. Blame the camera manufacturer's engineers.

Seems like the SD card of all things performed just fine, so it hardly seems like the weak point.

The NTSB's original report has more detail on how the SD Card was encrypted and how the NTSB managed to decrypt it:

https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&Fi...

The System on Module board is an Inforce 6601 SOM. [0]

It uses a Qualcomm Snapdragon 820 and they provide prebuilt Ubuntu Linaro distros for it, preconfigured for the board.

The camera manufacturer likely just tossed it straight in as configured and thus didn't know how the full disk encryption was setup.

This whole camera design looks like one of those 'we gave this project to some undergrad engineering students who've never designed a commercial product before and had no price target and thus it has a whole damn embedded linux system inside it for merely taking some HD video and stills triggered by some external wiring and saving them to an SD card'.

See also: almost any specialty medical electronic device ever manufactured.

[0] https://linuxgizmos.com/tiny-rugged-com-runs-linux-or-androi...

I'll admit I only watched a video on it not the report, but it had pictures reportedly redacted at manufacturer request. It showed a teensy 3 and some adafruit qwiic board in there. Obviously the real engineering is in the enclosure. Otherwise it could just be a webcam. But still, it's clearly not a very in depth electrical design. I'm all for SoMs if you can but they don't guarantee you the adventure of custom hardware bringing moving through all the software stacks and whatnot.

No serious commercial product should be using a Teensy under basically any circumstance.

Can I ask why? I'm not really into microprocessors.

The 3D-printed (and hot glued?) part in Figure 3 further support this theory (not that 3D prints can't be used in production).

Indeed this is massively overcomplicated, as one only needs to see what dashcams use to know that you don't need, or perhaps want, an entire OS on it.

Does not leave SubC in a particularly flattering light...

They had no idea how their own product worked. They didn’t even know it used encrypted storage.

This was either outsourced or done by some junior engineer who was putting pieces together like it was another Raspberry Pi project that just needed to kind of work.

The longer I last in this world the more products I realize are the result of telling a few people who don't know what they are doing to "make it kind of work."

That’s my entire experience in embedded. Everything I get from other companies basically looks like an internship project right down to the pointer arguments with unspecified bounds on the function calls. One of the companies we bought hardware from keeps representing things are working when they only work on devices in the lab. Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.

This is in fact the case.

It survived the pressure, does the rest matter?

> No deep-sea shenanigans around the Titanic wreck were revealed. Manley explains in his Twitter thread that “the camera had been configured to dump data onto an external storage device, so nothing was found from the accident dive.” Nothing particularly pertinent to the tragic accident, that is.

This is about camera hardware and how it survived. It provides no information or footage about the incident (in case you were looking for it like I was).

Scott Manley’s 45 minute video covering the NTSB report this information comes from went up today, it’s quite interesting.

https://youtu.be/qMUjCZ7MMWQ

If a hardened camera can survive, I'm surprised subs don't have a floating black box that can survive an implosion and then float to the surface and begin emitting a radio signal.

I guess the trick would be finding a way to securely attach the black box in a way that would ensure its release in a catastrophic disaster.

The ones that aren’t accompanied by a surface ship are military, and they really don’t want anything that might automatically deploy at the wrong moment.

This was part of what complicated the response to the Kursk disaster - they had a rescue buoy but it was welded in place so it couldn't deploy.

Probably commerical subs aren't a common thing with lots of regulation.

Just guessing here? :)

I guess he wasn’t planning for a catastrophic failure ?

He ridiculed anyone who told him he should, taking it as evidence that he was disrupting an industry.

There is something slightly romantic about dying in such a way that his body turned to mist and floated away in the current. A bit like having your ashes spread at sea. With fewer steps.

I keep thinking about the teenager.

Who didn't want to go and didn't feel safe, but who was pushed to come along by his father because it was his father's birthday.

There are a zillion applications for black-boxes, so why not start somewhere more accessible and with more impact? Your own house, car, and person, for instance. Think of how many elderly people die at home and nobody knows the details leading up to it. I'm being a bit facetious here - perhaps we don't need to know in those cases, nor in the Titan case. It's not as if there could be any data there which advances submarine safety - unless somebody is planning to build a Titan v2 with the same technology, marginally improved safety, and similar lack of testing?

Figure 3 from the report- that's an Adafruit sensor module on a 3d printed bit of plastic with a teensy-brand microcontroller just sitting in there! Actually, the entire electronics enclosure appears printed.

Very funny to see in what I assume is a million-dollar product.

What was the water condensation situation like in this submarine? Semi bare electronics sounds very very bad.

Any money spent coating those semi-bare electronics would have been wasted. There's an engineering lesson to be taught here, I'm sure.

On-brand, though. And speak some respect to Adafruit's name! Lady Ada's product isn't what failed.

It continues to amaze me how indestructible SDCards are.

It's a solid piece of silicon encased in epoxy, so there's nothing really to get crushed. Contrast this to something like a cellphone that's made of hundreds of separate parts and has void space that will get crushed.

So were the flash chips on the SSDs they found. It didn’t save them.

Are consumer grade cards really reliable though? Not so much against physical damage, but of data integrity over extended periods? "Industrial" SD cards can be 10 times or 100 times more expensive than consumer grade cards.

Say for argument’s sake there was a small air bubble in the resin. Couldn’t that result in cavitation?

Why isn't a cellphone filled with epoxy?

How would you do screen replacement? That is a common repair since people drop their phones and currently you can get your phone repaired by some teenager in a booth at the mall. If you fill the phone with epoxy, how are you detaching the screen, and getting a new ribbon cable through the epoxy?

So what if you can't replace a screen on an epoxy-filled cell phone? That's a small price to pay for knowing that your camera will survive if you take a one-way trip to crush-depth.

use pogo pins or a board to board connector

Which means air space that can get crushed. Either the phone is solid or it isn't.

> Which means air space that can get crushed

Would note that air isn't the only substance in a phone that compresses under 38 MPa. (Batteries come to mind.)

Just like they do it today - a lot of grinding, swearing and overall understanding what the civilization is going in not quite the right direction.

I'm sure there are some companies who want to do that, as long as they can convince people it's better for security or something.

The GoPro Session actually took this tack to achieve waterproofness without a secondary case.

When was the last time your phone stopped working due mechanical PCB damage?

Typically the limiting factor on your phone is the screen breaking, your battery life getting too short, wear and tear on components like buttons or the charging port, and factory defects. Epoxy isn't going to help with any of those. The only thing it would help with is exposure to water, but if other parts of your phone like your screen aren't water proof, what's the point?

Epoxy adds weight and manufacturing cost. It introduces design challenges as you need to balance the thermal expansion of the various parts. It's an extra step that can go wrong, and makes repair of other defects far more difficult. What benefit is there for the typical consumer that outweighs these costs?

To add to that. My son got his phone caught in a reclining chair without realizing it. The fact that the phone bent in half instead of destroying the chair is a nice bonus. Replacing the phone was cheap, replacing a chair would not have been — yes, both are insured, but replacing/repairing a chair takes a hell of a lot longer.

I think most would disagree XD.

Phones these days are often more expensive than the chair and can be pretty inconvenient to replace, especially if you have nonrecent backups.

Yeah not sure about you guys but me and everyone I know buys their stuff in ikea where a chair definitely doesn't cost more than a good cell phone

The heavy components on a cell phone PCB are reinforced with spot applications of adhesive to the PCB.

Filling the entire cell phone with epoxy wouldn’t help. The parts that break on drops are external like the screen.

This SD card was enclosed in a sealed metal container so it wasn’t exposed to water.

  > Why isn't a cellphone filled with epoxy?

OTOH, adding epoxy on top of everything else would probably only reduce their iFixit repairability score from 1 to 0, so...

A conformal coating wouldn't give much more weight.

A conformal coating isn't "filled with epoxy", which is the concern I was answering.

There is very little empty space in a phone, so conformal coating is practically the same as filling it.

Anyway, I wasn't disagreeing, just reasoning a bit further.

No, conformal coating and potting are extremely different things done for different reasons.

I'm not talking about which methods are being used, I'm talking about which methods could be used. Further, potting, where you let the epoxy drip off, gives you a conformal coating.

Conformal coating is much less viscous and would leave a layer orders of magnitude thinner then letting potting epoxy drip off. It's not at all comparable.

The point of filling it is to remove the compressible empty space so that large pressure gradients won’t crush it.

Neigh?

I didn't notice that, I was dictating to Gboard. If that's what was heard, then I should probably go eat some hay and get my tail brushed.

Some claim we are centaurs, we say Neigh!

I think they meant “nigh on irreparable“.

Well, most cellphones aren't subjected to the conditions found under three miles of frigid sea water. Epoxy is also really, really expensive.

Because then it gets a 0/10 repairability score on ifixit :)

Thermal concerns perhaps - how does epoxy dissipate heat?

Some types of epoxy actually conduct heat quite well.

I don't need any extra grams in my phone!

It's just not necessary, while having reliability problems of its own.

Imagine how much drama they could have avoided if they filled the entire submersible.

That would be a problem for the mic and speaker, and has relatively few use cases.

This comment made me wonder how much easier proximity fuzes would have been to develop in WW2 had they had transistors (or integrated circuits). I assume making modern solid state electronics 20,000g shock resistant is much easier than doing the same to vacuum tubes.

No need to wonder, proximity fuzes are still used today. And yes, they are much smaller, cheaper, more reliable, and precise.

So that's the next phase of making devices thinner? /s

It wasn't in the crushed part, it was in the camera's shell, and the camera was mounted outside, if I understood properly.

And:

> This still and video camera is rated to withstand depths up to 6,000m (19,685 feet, 3,281 fathoms)

Unlike the Titan sub...

The picture looks like the camera + storage SD card were in a sealed metal tube that was untouched.

Although the entire enclosure was shaken around enough to tear bits off the PCB via sheer inertia and crack the CPU (hence the need for the recovery process described).

It clearly received a nasty shock when the sub imploded; that's why the internal components were so broken.

Heat and wear are the greatest dangers to flash memory, and this was found in a cold dark place, with presumably plenty of life remaining.

The SDCard that was in another sub, properly constructed from titanium not carbon. The sub housed a camera, no humans.

It continues to amaze me how indestructible SDCards are.

Until they're sold as supplemental hard drives (cough Transcend Jetdrive cough). Then they'll fail if you even look at them strangely.

Put one in a Raspberry pi and it will be dead in a month.

remember the noatime mount option for the root fs!

It also amazes me how incredibly unbrowseable tomshardware is now with all the ads and pop-ups.

It also amazes me that people are using the internet w/o an adblocker in the year 2025

I haven’t bothered working out how to install one on mobile. I just don’t visit websites with shitty ads.

Just use the Brave browser. No plugins necessary.

It has poor compatibility on the iOS version that I've got installed, sadly.

Firefox on mobile supports uBlock Origin

On iOS, every browser is required by Apple to use WebKit. I just tried it again myself and FireFox on iPhone has no ublock Origin add on possibility.

Firefox Focus does work as an alternative.

Apple created a special system-level API for Safari Content Blockers. Apps like Firefox Focus, AdGuard, 1Blocker, Wipr can register filtering rules with Safari using this API. That’s why Focus can block ads/trackers inside Safari if you enable it under Safari

I think you mean HN readers.

i was also in shock, then someone reminded me there are iphone users.

the horror.

paying thousands of dollars just to be forbidden to block ads.

?

There are countless free and paid options on iOS too

Firefox Focus, Brave

AdGuard Pro, $9.99 once and you can use any blocklist you want (you can just copypaste from uBlock Origin if you wish) and it works system-wide with Safari

etc

what? there are many fantastic ad blockers on ios. Weird thing to crow about.

With UblockOrigin blocking the ads, there were no ads and pop-ups.

Since not everyone reads articles:

> Somewhat disappointingly, the images and videos shared in the report were taken in the vicinity of the ROV shop at the Marine Institute, also in Newfoundland. The location was the logistical base for Titanic dive missions. No deep-sea shenanigans around the Titanic wreck were revealed.

Wouldn’t it have been streaming it to disk without creating the file? Kinda like how if your camera dies while it’s recording, there is no recording. You have to manually recreate the file.

I think that’s what they think. It was being recorded by the onboard PC to its SSDs, which were completely destroyed in the implosion.

It might be advertising, but I'll allow it because it's so metal

also basically if enough companies agrees on helping the cause your crypto secrets are quite more likely to be exposed...

Isnt the weakness here that there was nothing encrypting the actual key? On a laptop luks key stored in a tpm would usually be encrypted using your passphrase

The NTSB report noted that if the TrustZone secure enclave system was being used, then yeah this data would be toast.

But it speaks more to Oceangstrs negligence that this situation even existed: why wasn't any potential encryption keys escrowed ashore to ensure they could be recovered later? This shouldn't have even been an issue.

It seems the manufacturer of the camera didn't even know (at least in the part of the org communicating with the NTSB) that their storage was encrypted. In any case the media recovered were from testing/non-dive environments, and during an actual dive footage would presumably be recorded directly to the onboard computers (which were irrecoverably destroyed).

Oceangate should take the blame for a lot of things but probably not this.

is this a common setup to have the camera store to external storage device without storing to the SD card as well?

Yes because external storage is much larger, and theres nothing more annoying than being in the middle of doing some science with 30 other bits of complex equipment, and then the camera stops working with storage full errors and youre 7000m underwater in a cramped sub trying to navigate a camera UI to find the setting.

Configure your systems so they are in the configuration that is less likely to cause random disruption in the field.

Which makes me wonder why they bother with the SD card at all. What was it meant to be storing? If it is not intended to be the real storage area, why not just have it in a loop, constantly over-writing the oldest material?

They probably used it for testing only, hence why it had irrelevant footage.

They might have forgot to remove or just didn't care.

But how did anyone figure out it was a SanDisk SD card? Card details were redacted.

Presumably because it looks identical to a Sandisk extreme pro 512gb, with grey boxes drawn over the logo.

The report has a heavily redacted interview with a submarine expert. Who directed Titanic and The Abyss.

They’re not good at redacting.

There's only 3 manufacturers of SD cards in any volume, you can compare the branding and font choices and see who's it is.

SanDisk is one of the big three on SD-3C/SD Association.. so kinda regardless of the MFG it's 'one of theirs' in a roundabout way.

What I also learned from this article is that Scott Manley is still on Twitter.