You could machine a spacer for the outer race to add 2mm to the OD. Just make sure that there is enough of a shoulder in the steering head to support the bearing.
Strength will not be an issue. The forks and clamps on a YZ are designed for much higher loads than the XS would ever see.
Wolfe is...