hckrnws

It's both. Anything that you effectively have optimized to the details of its environment will end up dependent on that environment. If that environment is the real world, then it's fine. If that environment is a simulator, now you have sim-to-real problems.

There's always going to be a gap between reality and simulation, because simulation fidelity has an incredibly long tail. "Digital twins" are relatives, not identical replicas, because everything about capturing the real world for simulation involves simplification and discretization and abstraction.

That’s a pretty excellent article actually and I think does a good job describing how having over-specifications in your simulated model makes it brittle in the transfer.

I think something that was kind of overlooked was fine tuning post transfer so that you’re basically transferring a gross motor policy and then fine-tuning it with the PID feedback into a fine-grained motor policy

This reads like head-cannon

Ka-boom!