Not sure what the minimum spec is but I've done it with an i5 and 8Gbps of RAM. No drama but as mentioned, it will use all available CPU resources while transcoding. CPU and disk seem the major factors, not so much with RAM (assuming 8G or above - you shouldn' run anything Windows with less

). I allocate all of my CPU cores to Serviio.
The bigger issue is what else is running. If the server has other demands then the CPU may get bogged down. As long as it can maintain transcoding at a rate faster than it streams there is no issue (assuming all other factors are good - such as your network).
Curious as to what advantage you perceive in transcoding on-the-fly to H264 rather than MPEG2 which uses less CPU to achieve and for streaming on a LAN wont in my experience produce any noticable difference. The only reasons I would do it to H264 is where the renderer doesnt support MPEG2 at the resolution required, or arguably, you are streaming across the internet (where bandwdith may become an issue and the higher compression of H264 comes into play). No drama with transcoding to H264 just wondering what the driver is if you are worried about minimum specs.
Transcoding to AC3 is not an issue for even basic hardware.