Launch Stars appear to function based on a magnet-like force holding the inner star (which one's feet are placed on) roughly inside the outer star, albeit weakly enough that it can wobble about when its Launch Star is not in use. The force of Mario's Star Spin is enough to push the inner star out of alignment, which then tries to snap back, providing a catapult-like effect in the process.
Launch Stars are also known to have their own gravity. This would make them very dense, so perhaps Mario having enough force to shoot out of one at all means that he is pushed at escape velocity for most planetoids.
These details, which are rooted in the Launch Stars' games of origin, would explain how Launch Stars work while still falling under some physical laws of the Mario franchise. For instance, despite exerting gravity, Launch Stars are not affected by other gravity sources, in the same vein as coins and Question Blocks.
Super Mario Galaxy 2.5
Super Mario Galaxy DSi
Launch Stars reapear in Super Mario Galaxy DSi, and have the same role as in Super Mario Galaxy and Super Mario Galaxy 2.
RedYoshi's Universal Conquest
Launch Stars return in RedYoshi's Universal Conquest. They are used just as they always have been.