I've been running Serviio on an old (circa 2006) Windows XP box with a 2.5 GHz Celeron CPU and 512 MB of RAM. It has served me quite well. I posted about it on my blog (
http://mdb139.wordpress.com/?s=NAS) a while back, but I will concur that you're not likely to be too happy with any video transcoding (though I've found that audio transcoding works okay most of the time).
I recently (as in this week) upgraded the RAM to 2 GB (the max my box would allow), upgraded to Windows 7, and installed a new 3 TB hard drive. I'm in the process of moving my media files to the new drive as I type this. I thought about Linux and even buying a Synology box, but for the money ($150 for RAM and Windows upgrade) I thought it was a reasonable investment.
I don't even keep a monitor, keyboard, or mouse attached to it... In fact, I don't even own a proper monitor at this point it time, so I just connect it to my TV to do the initial setup, install TightVNC, and then do everything remotely. Works like a champ. I use the PC as a media server and backup station.
At the end of the day, I think your best bet will be to go with what you know (which is why I stuck with Windows).
Good luck!