Stafford was a Parade All-American quarterback who had a 46-5 record as a four-year starter at Temple and Belton. Stafford led Temple to regional playoff appearances in his sophomore and junior seasons, then took his talents to Belton when his father, Dick Stafford, accepted the head coaching job. The 1982 Texas Football Super Team quarterback promptly ended the Tigers’ long playoff drought with a district championship and a win in bi-district, the Tigers' first in eight years. As a senior, he set or tied five school records while amassing 2,344 yards in total offense (1,440 passing). Stafford was also a Parade All-American in baseball and all-state in both baseball and track, winning the 300-meter hurdle title as a junior. Although taken in the pro baseball draft, Stafford opted for football at Texas (1984-87), where he broke most of the Longhorns' passing marks and led UT to two bowl games, including a 32-27 win over Pittsburgh in the Bluebonnet Bowl.