Correct, it’s all about what’s closest to the scale of movement you’re needing. Close to a body, you’ll likely line up with its rotation if nothing else. Away from a body, you use the next frame of reference that makes sense. That could be the next closest body’s plane of rotation (which could be the same as the solar system, but not a given, i.e Uranus), the solar system itself, or if approaching another system, line up with its plane. You don’t have to do this, a polar orbit is just as valid as one within the plane.