As Phoronix referred, UVD hardware decoding can work on AMD open source driver at this time, however, they just post how to use UVD on Ubuntu. I believe that it is not hard to port UVD to Fedora by myself in this weekend.
In the original post, UVD needs a drm-next branch kernel. Luckily, drm-next branch is merged into mainline by Dave Airlie. Since kernel-3.10.0.rc0.git15.fc20 on Fedora, the kernel could load UVD firmware normally. After installing new kernel, my machine looks laggy intermittently. I manage to rebuild the kernel at the local machine without debug option. At this moment, I do not see any lag again.
The second step is updating linux-firmware. Clone the latest firmware from git.kernel.org and put the new radeon firmware to /lib/firmware/radeon directory. Or you could install my RPMs. When you reboot your machine, you could find out UVD firmware is loaded in dmesg. Look for ‘[drm] UVD initialized successfully’.
The next is building mesa with VDPAU support. Only the latest mesa source code could support UVD decoding. It is quite easy to git clone the code. Please pay attention that Fedora has not LLVM-3.3 so that you must disable llvm option. Moreover you need to disable radeonsi driver because radeonsi driver relies on llvm. I think it loses some performance improvement for r600g driver. Another is dropping –enable-osmesa option. The reason is it fails to build osmesa component. Finally you could use my UVD repository too. If you want to try UVD decoding, don’t forget to yum install mesa-vdpau-drivers. And mesa-git requires libdrm-2.4.44, newer than official repository.
Something extra I done is I upgrade libvpdau and radeon DDX driver. libvdpau bumps to 0.6. And xorg-x11-drv-ati bumps to 7.1.0-git with glamor library. I could make sure whether glamor works fine or not. Generally we could log in system normally. These pacakges are not necessary, I think. Just try it.
All the things are done. You must reboot the machine and select the latest kernel to boot. After logging in Fedora, you could choose a movie player with VDPAU support. Then you could play the movie under hardware decoding. MPlayer and smplayer works fine. Maybe all mplayer-based movie players are OK. VLC could not get hardware decoding. Totem could not, either.
All my packages are built on my local machine. They are not tested on Koji. If you want to try them, you just need to download the repo file and put it into /etc/yum.repos.d directory. Both RPM and SRPM are here.