Toronto’s economic development committee voted unanimously on Monday to defer considering a bid for the 2024 Summer Olympics.