One of the things that I didn't think about before I started my first startup were all the parts of the infrastructure that weren't product. They've also consistently been the things that other entrepreneurs forget about when they come to me with their "ohmygodIhavethisidea" ideas. They're also vitally important because they allow the founding team to actually execute the pivots you'll need to execute quickly, intelligently, and without sacrificing development of the actual product that you're trying to pivot.
The first part of this infrastructure is the admin panel. This admin panel is what will allow you to do troubleshoot for customer support, engage in administrative actions, and audit your payments in and out of your system without having to mess around with your production database. I cannot impress upon the first-time web entrepreneur just how important it is to have a robust and well-functioning administration system. When you're building your administration system, it is important to make sure that you can disable and delete users, view and change payment regimes, and reset user passwords (because they will invariably not be able to find your reset password form and they *will* e-mail you). If your product takes off and you find yourself bringing on board a business person, they're going to be tasked with support, and you do not want whoever is doing support interrupting engineering to complete basic Level 1 and 2 customer support. If your product really takes off, you may find yourself bringing on board a dedicated support person. If you wait until traction to build out your administration panel, you're going to be ceding ground to the competitors that spring up when they see your traction. Django's admin panel is so well developed that you may not have to build your own, and is a reason to strongly consider the framework if you're familiar with (or interested in) Python. Even as your product pivots, the administrative support tasks generally don't.
Make sure you've built a functional admin panel before you launch.
The second part of the infrastructure you're going to want to build (or implement with WordPress, Melody, or some simple bare-bones CMS plugin that someone has already released for your framework) is a content management system (CMS). It should not take a push of new code to change the static content on your website. Holding up basic changes and updates to your about section, marketing pages, terms of service, privacy policy, and help section because you need to wait on some guy to merge his branch into master (or check into trunk, or whatever your VCS calls it) is silly. You don't want to have your marketing person or intern create some hot new copy then have to hand it off to engineering to copy/paste it into code and markup each and every link, line break, and image to get it live on the site. It makes the business side frustrated with the pace of changes and the engineering side of the house gets frustrated at having to do such menial tasks when they could be building some crazy awesome technology that is your core long-term competitive advantage that you brought them on board to do. Again, some web frameworks have bare-bones CMSs or templating engines built into the system out of the box.
Make sure you've built (or implemented) a content management system before you launch.
The third part of the infrastructure you need to build is a good metrics dashboard and logger. The standard piece of advice is "pick a few metrics, but log everything". The idea of logging everything when you've decided to focus on just a few Key Performance Indicators (KPIs) can often cause heartburn. But it's relatively simple; build a separate table (or database) that automatically logs all the various things that users do. Don't worry about duplicating writes that you're doing elsewhere in your database. Having a separate table (or better, database) will save ou on reads and JOINs and whatever else when you need to dig in. The one thing the logger needs to do is be *auditable* - that is, you need to be able to say that user_id=8273 did x on page one, then y on page two, then z on page three. These loggers are so important that there are venture-backed startups that exist solely to help other startups collect these logs and make sense of the data they're collecting. The KPIs are, of course, actionable metrics that exist in the aggregate, but the ability to see what any individual user has done will help you troubleshoot, audit, and make hypotheses for new features and functions. Also, one quick note: Google Analytics aren't auditable at the user level nor actionable in helping you make forward-looking hypotheses. Everything in Google Analytics is backwards looking. The product is free as in beer and all-but-free from an engineering cost perspective, which is great. GA does give you the ability to find exception cases and the basic funnel visualizations can be useful for getting conversion metrics up and running quickly, but don't kid yourself - Google Analytics is mostly data porn.
Make sure you build a real metrics dashboard and implement a real logger before you launch.