How do I begin and what should I collect ?

The fun part about collecting stamps or other philatelic material is you collect whatever interests you. Begin by collecting different things that appeal to you. No need to spend a lot of money to begin. Clubs like ours have collections of stamps for very little. Learn about the stamps you collected by looking in a catalog, researching on the internet or talking with others at the meetings. Chat with other members about their collections and you’ll discover how diverse the theme of a collection can be.  Check out our Exhibit section to see what others have collected. As you learn more you’ll find areas that you want to concentrate on.
Examples
1) a country, an era of a country
2) postcards or postal stationary
3) topical stamps such as trains, birds, Olympics, famous people
4) type of stamps such as overprints, air mails, occupation, postage due
5) different types of cancellations, cancellations for same city or postal route over time
6) variations of a single stamp

There are many good resources to help you.  Here are some internet suggestions.

Starting a Collection – American Philatelic Society

The ABCs of Stamp Collecting by Janet Klug (Smithsonian)

If you have an Android Tablet there is a free App about stamp collecting:  Stamp Collecting A to Z