You don't have to bring that much with you, but there are a few things you should always bring along. Besides all the clothes and the bike, I always pack a small set of tools (not much unless you are heading out for a few weeks), some extra tubes, a bathing suit, and a good bike lock. A floor pump is always nice as well.
Finally, here's a quick list of the most commonly forgotten items that people forget: bike shoes, helmet, gloves, sunscreen, sunglasses, water bottle/hydration pack, and pedals. Bring all those and you won't end up missing too much
Time's running out for this season, so pack your bags and hit the road.