Serviio ships with ffmpeg that doesn't include h264 encoding. You should be able to remux h264 content (e.g. in mkv) into mpeg2ts, but according to this article it only plays h264 in mpeg4 container, which is not suitable for streaming (afaik), ie it has to be fully transcoded before being able to be played.
Petr Serviio developer / site admin Do not send me PM for support as the solution can't be shared with others.