Our holidays are usually in our caravan, so our little JRT comes with us and loves it! She knows when we are getting ready to go and gets excited, and she sulks on the day we return home!
If we go away other than to the caravan, she either stays at home with our daughter or goes in kennels. Bob is quite laid back - as long as she gets food and cuddles, she's happy, so we have no problems leaving her.
Maybe your little fella would settle better with his own bed and favourite toys in 'his' packing? Might be worth trying DAP spray to help to calm him too - Bob was travel sick due to anxiety in the car when she was young and the vet suggested trying DAP - it worked really well. It's not cheap, but is worth considering.