Ubuntu uses Rhythmbox as the default music player / podcast app. I used to use Amarok, but the new version is just bad - Laggy Buggy, Ugly, and generally lame. So I've gone back to Rhythmbox, which actually is pretty good.
There is one thing that is sort of a pain with it though. When you are using Firefox, and you stumble across a podcast you want to subscribe to, it is not one of the default options to use.
No problem. when you hit that firefox subscription screen, make sure that Rhythmbox is running. Hit the drop down menu, and select "Choose Application" and drop in:
you should only have to do it once, and then you can choose Rhythmbox from the drop down menu in the future.
Just a side note, Rhythmbox doesn't appear to like all feeds that much. sometimes you need to dig around a site to find one that works well for it. I think it may have a problem with Atom feeds but I'm not sure on that.