Illegal Charge to Credit Card?

I called a limo company to make a reservation for a limousine. The guy said that the reservation won't be confirmed until half the cost of the limousine is paid ($400). I cancelled the reservation because we found a limousine much cheaper. I was sent an authorization form for the credit card of $100 (the money i was initially going to put down) which i did not sign. Now the guy is trying to to charge me a $160 for a cancellation fee because the limo was "off the market" for 3-4days. What pisses me off tho is this guy told me that i had to hurry with this deposit because another person could put down more money on the car and whoever gets the reservation first will get the car. A friend of mine says to just have my parents cancel the fee on the credit card because it hadnt been authorized, but my parents are afraid of being sued. Would i have a solid defense?