I also pronounce those vowels differently. The Americans from whom I learned the new coworker's name don't, which I guess is how I got it wrong.

Do you remember when I was in preschool and started working out that I should insert an r into anything Mum pronounced with a long ahh sound? For a while I said "mother and farther." Overfitting!