So I discovered there is yet another minor flaw with Vesta CP. With the current release, phpMyAdmin does not work out of the box. Fortunately I found this: phpmyadmin-fixer
I took a quick peak at this script on the first page just to make sure that there was no malware and everything looked ok. I gave it a shot and now phpMyAdmin fully works as expected.