Just an update.
I pulled the front back apart and then put it back together with the same bearing. What I found is that, when tightening the nut all the way down for the bearing to seat on the stem, it makes it loose. I checked it as I slowly tightened the nut, it was tight at the top of the...