The current AZURE documentation is outdated.
I'm trying to figure out how to setup web app + sql and it's all different now.
I was eventually able to figure it out.
The documenation screenshots are the older Azure interface.
Basically - the steps now are to create the service plan, add a sql box, add a database, add an app service, connect and to migration / update-database, then in visual studio (for me anyway) publish and it will pick up everything on your Azure account.