There's like 50-60 cents a gallon in taxes, for starters.
Then you have refinery costs. Distribution/transportation costs. And special "summer blends" for environmental reasons also make gas significantly more expensive in the summer.
But take a look at Canada or Europe and then tell me gas companies are gouging us.