"Sirisena has broken his promise and taken executive powers into his hands. He has sidelined parliamentary power," UNP leader Wickremesinghe said, addressing the crowd.
The UNP has demanded that Parliament be convened immediately and democracy restored.
The ousted premier alleged that President Sirisena had assumed he would get his way.
However, he said the UNP and its partners in the United National Front will not give up and will continue to push for Parliament to convene immediately.
"People have spoken, the message is for early calling of parliament," said Champika Ranawaka, an ex-minister and a key organiser of the event.
Wickremesinghe said in his address that the movement calling for recalling Parliament would be taken to places outside the capital to ensure democratic rights of people.
At the rally, an activist held a placard that said 'I am not here for Ranil but I am here for democracy and good governance'.
Thousands gathered at the Liberty Plaza junction in central Colombo which included UNP activists and members of civil society groups and citizens.
The UNP claimed about 100,000 people thronged the rally, though police sources estimated the number at 25,000.
Sri Lanka, a Buddhist-majority nation in the Indian Ocean, was plunged into chaos on Friday when Sirisena sacked Wickremesinghe as prime minister in a surprise move. He also suspended Parliament in an apparent bid to shore up support for newly-appointed Prime Minister Mahinda Rajapaksa.
Sirisena is under increasing political and diplomatic pressure to reconvene Parliament and resolve the Constitutional crisis.
Mangala Samaraweera, the ex-finance minister under Wickremesinghe said, "This was a Constitutional coup and it is our duty to protect democracy and sovereignty of people".
Speaker Karu Jayasuriya has urged the president to let Wickremesinghe prove his majority support on the Parliament floor.
The supporters of Rajapaksa, a former president, are confident that he would be able to prove majority support in Parliament as they are sure that members of Wickremesinghe's UNP would defect.
"We are waiting for more UNP members to join us. We have the numbers," Lakshman Yapa Abeywardena, a Rajapaksa loyalist, said.
Wickremesinghe has said that he still commands a majority. However, he suffered a setback after four lawmakers from his party, who had pledged allegiance to him in public, made a U-turn and accepted ministerial positions in the Rajapaksa government.
"We have the majority despite four of them joining Rajapaksa," Ranjith Madduma Bandara, an ex-minister, said.
Speaker Jayasuriya plans to reconvene parliament on Friday after he meets with Sirisena.
Jayasuriya has got signatures from a majority to re-summon Parliament in spite of Sirisena's proclamation to suspend sittings.
The UNP claim that Sirisena had suspended Parliament to engineer crossovers from UNP and other groups to provide majority for Rajapaksa has been dismissed by the Rajapaksa camp.
However Rajapaksa was able to secure the fifth defection of a UNP member to his side this evening when Dunesh Gankanda, a UNP deputy minister was sworn in as state environment minister.
The Rajapaksa-Sirisena combine is now short of 17 MPs to form a simple majority of 113 in the 225-member assembly to confirm Rajapaksa as premier.
Jayasuriya has called for a meeting of all party leaders to assess the current political situation. At least 128 members had written to him calling for reconvening of Parliament. Jayasuriya has insisted that the issue needs to be settled within Parliament.
Both Wickremesinghe and Rajapaksa are working to secure their numbers in Parliament.
The leaders of the main Tamil party, Tamil National Alliance (TNA), met Rajapaksa this morning to discuss the current political situation. TNA sources said R Sampanthan, who is also the leader of the main Opposition, was asked by Rajapaska to stay neutral in case of a floor test.
The TNA has 16 MPs and can play a crucial role in deciding the legitimacy of the government.
Rajapaksa's tenure as president was marred by allegations of authoritarianism, corruption and human rights abuses, especially against the country's Tamil minority.
He was defeated in the 2015 presidential election when Wickremesinghe and Sirisena formed an unlikely coalition, and their government initiated several investigations into alleged Rajapaksa-era crimes.
Under Sri Lanka's Constitution, the president, who maintains executive powers, can appoint a new prime minister once the current premier has lost control of Parliament.
Wickremesinghe argues he cannot legally be dismissed until he loses the support of Parliament. His party was prevented from holding a vote when Sirisena abruptly suspended Parliament on Saturday until November 16.
Wickremesinghe claimed in a Facebook post on Monday that he had obtained the signatures of 126 MPs calling for Parliament to be reconvened immediately to end the political standoff.