Day, 28, rushed from the Rodney Parade pitch at the end of Tuesday's fourth-round replay, having learnt his wife Lizzie had gone into labour.
ROBBIE WILLMOTT and Padraig Amond were the toast of Newport after their goals helped League Two County stun Middlesbrough.
"The professional that he is, he turned his phone off", Newport manager Michael Flynn told BT Sport after the game.
Premier League Wolverhampton Wanderers saw off a determined Shrewsbury Town to win 3-2 at home and will be joined in the fifth round by Queens Park Rangers, who overcame Portsmouth 2-0. I don't know what it is with them.
"They were the better team, they ran all over us. Forget the conditions, you have to match people and that's the real disappointment", Pulis said.
"He sprinted off, it's the quickest I've seen him move, and all our thoughts are with Joe and Lizzie and the arrival of the twins".
The club later tweeted: "Everyone connected with Newport County AFC would like to wish County's Number One Joe Day and his wife Lizzy all the very best!"
He added: "Lizzie backed me and made it an easy decision for me to play the game".