In December 1951, on the recommendation of reserve team trainer Jimmy Gribben, Celtic bought him for £1,200. He was signed as a reserve but injuries incurred by first team players resulted in him being elevated to the first team. In 1952 he was appointed vice-captain; when captain Sean Fallon suffered a broken arm, the captaincy was passed to Stein.
Celtic finished eighth in the Scottish League during 1952–53 and were only invited to the pan-British Coronation Cup tournament due to their large support base. Celtic beat Arsenal, Manchester United and Hibernian to win the trophy. This success was built upon in 1953–54, as Stein captained the side that won a League and Scottish Cup double. It was their first League championship since 1938 and first League and Scottish Cup double since 1914. As a reward for their achievement, the club paid for all of the players to attend the 1954 FIFA World Cup in Switzerland. Celtic had also sent their players to watch the England v Hungary match in 1953. Stein was influenced by Scotland’s poor preparation and the impressive performance of Hungary. Stein received his only international recognition in 1954, when he was selected for the Scottish Football League XI. It was not a happy experience for Stein, as his direct opponent, Bedford Jezzard, scored two goals as The Football League XI won 4–0 at Stamford Bridge.
In 1954–55, Celtic finished second in the league and lost the 1955 Scottish Cup Final to Clyde. Stein was forced to retire from football after suffering persistent ankle injuries during the 1955–56 season. While playing in a friendly in Coleraine in May 1956, he landed awkwardly and rolled over on the ankle. Two weeks later he had an operation to remove a bone nodule, after which he was ordered to rest until fully recuperated. While on a holiday in Blackpool, the ankle irritated Stein, who found that it had become septic. He was no longer able to flex the joint and had to stop playing, officially retiring on 29 January 1957.