On 13 June 2013, Baldé signed a four-year contract with Scottish Premier League champions Celtic. On arrival, he stated that his new team were “one of the best clubs in Europe and they are going to play in the Champions League.” He made his official debut on 23 July, replacing James Forrest for the last 15 minutes of the second leg Champions League qualifier tie against Cliftonville.
Baldé scored his first competitive goal on 28 September 2013, coming in as a late substitute and netting the last goal in a 5–2 win at Kilmarnock. On 27 October, at Partick Thistle, he took to the pitch in the 67th minute and scored the 2–1 winner eight minutes later. However, he failed to establish himself as a regular in the team, and although he made 20 league appearances over the season, most were as a substitute. His only other goal came in a 4–0 league win over Kilmarnock, on 29 January 2014.