Jamie Dornan 'doesn't worry' about being remembered as Christian Grey