Neither side could convert their chances, and the game finished as a goalless draw, before City captain Andy Dawson netted the winner in the shootout.
Nicky Barmby nearly put Hull in front early on when Craig Fagan raced down the right wing and played a neat ball in, but the ex-England international's first-time shot from ten yards was well saved by Dimitrios Konstantopoulos.
Hartlepool then went forward and nearly broke the deadlock when Ben Clark found himself with room in the box, but his shot was palmed away by Boaz Myhill.
Mark Yeates had a clear opportunity in the second half, but Konstantopoulos comfortably saved his left-foot shot from 15 yards.
Fagan and Dawson then tested Konstantopoulos, but the Pools shot-stopper was equal to both efforts.
Darryl Duffy could have put the Tigers ahead just after coming off the bench, but he could not direct his header from five yards after a neat cross from Fagan.
Towards the end of the 90 minutes home substitute Jon Parkin found himself with just a defender to beat from five yards after Konstantopoulos failed to reach a cross, but the Tigers frontman couldn't convert.
Neither side could find the net in extra-time so the game went to penalties.
Hull finished as 3-2 victors with Dawson scoring the decisive kick, after Stuart Elliott and Duffy had missed for the Tigers, and Gary Liddle, Clark and James Brown had missed for the visitors.