Excuse the long title, it’s to help people search for this post – took me ages to find out how to make this work.
1) Download gnome-tweak-tool
2) Under the “typing” option
3) Set CAPS-LOCK as additional super key
Done. Everything works just fine, it’s lovely.
Still not sure about the keyboard, it sounds like a typewriter factory in here!