For a total noob I would recommend to take it easy with simple things.
First you need learn networking, at least at basic level.
Second, you didn't say for what you want to build a website. Is just a blog, is an webapp, is and information website?

1. Buy a domain, it doesn't really matter from where, but at least look about to have easy setup DNS and stuff like that. Then to be cheap.
2. If you do not want to self hosted at home, buy a good VPS, with a decent space for your web and traffic. Some domain sellers are also offering hosting. Analyze the offers. But I would go more for self hosting first, to learn basic stuff.
3. Setup a simple machine PC with LinuxOS (it doesn't really matter the flavour) + web server. Ubuntu already come with webserver.
4. Setup a Wordpress web framework. It's very easy to start for a noob. Later you will have time to learn more types of webs.
5. Setup a static public IP from your ISP and open the port 80 and 443 on your router and forward the ports to your local hosting machine IP.
6. Point your domain and www host to your static public IP.
7. setup a free SSL certificate with letsencrypt for your wordpress site.

Done, start building your first web, install wordpress plugins as you need.
Easy peasy for a total noob.