After switching from libav to FFmpeg, the HandBrake developers quickly added NVENC encoder support to HandBrake. You can now select both NVENC for H.264/H.265 encoders in the drop down menu or with the command line interface:
$ HandBrakeCLI --help | grep -A12 "Select video encoder" -e, --encoder Select video encoder: x264 x264_10bit nvenc_h264 x265 x265_10bit x265_12bit nvenc_h265 mpeg4 mpeg2 VP8 VP9 theora
With most GPUs I tried, even setting the slowest and costly preset results in the video engine not being fully utilized. Encoding times are cut to ~25%.
Awesome! No more
ffmpeg command line black magic. You can now comfortably create your preset in the HandBrake gui and then use
HandBrakeCLI through SSH on your awesome Plex Media Server. The build is available for both CentOS/RHEL 7 and Fedora.