I've seen several episodes where planets have petitioned the Federation for membership, and still others where the Federation is watching a planet's development in anticipation for future membership; but I can't remember if any planet has been kicked out of the Federation. Have there been any?
3 Answers
Within the TV series and films, there are no instances of existing UFP members being ejected due to their actions although there is a mention of Turkana IV withdrawing their membership after a bitter civil war.
Additionally, certain planets were annexed during the Dominion War (such as Betazed) but their status as members remained and they operated with "governments in exile".
In order to find situations where planetary systems have left the UFP you need to start looking at the weird and wonderful EU book canon;
Star Trek: Final Frontier - Vulcan has left the federation in order to explore the idea of reunification with the Romulan Empire.
Stone and Anvil - Selelvia was effectively ejected from the Federation after it became apparent that their delegates were using mind-control at Federation Council meetings.
Star Trek: Voyager: The Eternal Tide - Andoria 'left' the Federation for an unspecified reason.
Star Trek: Typhon Pact #4: Paths of Disharmony - Another scenario where Andor leaves the UFP, this time after a global referendum on whether to remain (prompted by the UFP's stance on Genetic Engineering).
Star Trek: The Original Series: Devil's Bargain - Vesbius is ejected from the UFP for refusing to give up Eugenics and Genetic Engineering.
Star Trek : New Frontier : Restoration - Danter is temporarily(?) ejected after it becomes apparent that their government is under external control..
Star Trek: The Next Generation – Survivors - New Paris leaves the Federation as they feel that they've been 'abandoned' after a series of disasters.
In the original plot for Deep Space 9's 'Homefront/Paradise Lost', back when it was going to be the season 3 finale, Vulcan was going to pull out of the Federation. http://en.memory-alpha.org/wiki/Homefront_(episode)#Background_Information (Just as I finished writing this, I just realised James Sheridan wrote this as a comment long before me, but to be honest, I think it will be every respondent's first thought on this question)
I could be mistaken but I think something like this happened with the Danteri from Star Trek: New Frontier. http://memory-beta.wikia.com/wiki/Danteri I remember them being a member of the Federation in the first book but apparently that is no longer the case, though I'm not sure if that was a retcon or such.
The Andorians were placed under an embargo by Baras Rodirya (posing as Ishan Anjar) during his term as Federation President. They withdrew from the Federation but rejoined after his involvement in a conspiracy to deny them a cure for a reproductive crisis was exposed. (The Fall, Peaceable Kingdoms)
Some worlds were denied membership in The Next Generation until they presumably improved.