Intel Xeon E5-2680 v4 is processor manufactured by Intel and launched in Q2 2016 . It's using FCLGA2011-3 socket. It has 14 cores and 28 threads with clock of 2.4 GHz that can turbo clock itself to 3.3 GHz . This processor is used in Server devices and can consume up to 120 watts of power .
NVIDIA Quadro P5000 is graphic card manufactured by NVIDIA and launched in Q4 2016 . It's using PCIe 3.0 x16 bus type. It has core clock of 1607 MHz and 16384 MB (16GB) of memory . It supports DirectX 12.1 and OpenGL 4.5 . This graphic card is used in Workstation devices and can consume up to 180 watts of power .
This configuration has 0.2% of processor bottleneck . Everything less than 5% should not be concerned major bottleneck.
Processor bottleneck is worse, than graphic card bottleneck, because utilization of your processor will be on maximum and it will throttle other programs that are ran in background. Also you won't experience maximal performances that your graphic card offers.
You can resolve this by upgrading your processor or by downgrading your graphic card. Downgrading your graphic card doesn't make much sense and in that case is better to leave current graphic card in you computer.