Virtual machines with GPUs are expensive. The cheapest I found so fare are from Paperspace. Nevertheless, it can be useful to quickly reproduce your setup without the disadvantages of snapshots or golden master images.
When you manage your hosts with Ansible, you can just run the below playbook against it to set it up from scratch. I use some additional roles to set up my home folder or firewall rules.
Before running it, you need to mirror the cuDNN installer package, since Nvidia doesn’t allow direct downloads. That part is marked in line 11.