Programming Language - Java, PHP, Ruby, Python
This is the first decision a developer has to make and might be the easiest choice - pick the one where you have the most expertise. However if you are well versed in more than one language - pick the one that will suit your startup - based on the funds you have ( cheapest to host will be php to Java hosting being expensive) , speed at which you want to release ( php/rails being fastest to Java being the slowest ), team's expertise ( java developers being easy to find and ruby developers being hard to find ) etc.
Framework Choices - Rails, ActiveRecord, CakePHP, Spring, Hibernate, Django..
In most of the cases the framework and language decision will go hand in hand. If you pick Rails and active record then you will be going with Ruby. If you pick Spring,Hibernate - then you will pick Java. In the end it all boils down to the passion and expertise you have with the languages and frameworks.
One advice I have here is - do not pick a language/framework to educate yourself. You can try to learn from a hobby project - but not with a startup. The honeymoon period is only till your launch date when you can learn/play/experiment. But after launch you cannot go back on any of your decisions - and you cannot afford to have downtime, bugs or incomplete features - customers these days are spoilt with high quality and feature rich applications like Gmail, Facebook etc. and they will expect the same out of your product too. So picking a language/framework where you are The Guru will save you from lot of trouble.
Database - MySQL,PostgreSQL
Tough choice here. Both are equally good. MySQL wins the popularity contest.
Development OS - OS X, Linux, Windows
Sometimes you are forced to work with an OS ( Windows because of Nokia PC software which will not work in linux). Some times you have no choice but to work with an OS ( linux for hosting / build machine ). Most of the time, for a startup, you will not have money for luxuries so you can only drool at OS X screenshots. Sigh.
Hosting - Shared or Dedicated
Avoid shared hosting. It will be the cheapest option - but you will have trouble later with ip address being blacklisted because a spammer was sitting next to you. Dedicated hosting costs a little more but it will be worth it and also you get full control.
HTML/CSS : Table vs DIV
I don't have enough expertise with DIVs so I built my layout with Tables - simple as that. Even though I hear lot of advantages of DIVs I decided to go with Tables - looks great even when browser is resized, resolution is changed, some content decides to grow out of hand etc.
Richeditor frameworks - YUI Rich Text editor, TinyMCE, NicEdit
No clear winner - each has its own quirks and strong points. This one you can easily change even after going live.
IDE - Eclipse, IDEA, Netbeans
Perhaps IDEA might be the first software license I purchased. There are open source alternatives for almost everything - Office, Photo editing etc. But IDEA burns all the open source alternatives to ash with its blazing speed and shortcuts.
I guess this covers most of the choices I think I made. Do point out any choices I might have missed and any better alternatives you have come across.