Caving or spelunking is dangerous. Unless you visit a cave that has been staged for site-seeing, you should respect the cave you visit. Raw caves are typically on private property and if anything happens to you, you will require a very expensive cave rescue because caves are not tourist attractions and you will require knowledge and skill to get through them. You'll be underground for hours so if you're claustrophobic, caving is not be a good idea.
Caves are muddy and regardless of the traction on your sole, you'll never have enough traction. Additionally, that mud never seems to get out, so any sturdy pair of boots, shoes, or sneakers that you were going to throw out anyway is fine.
2 flashlights/headlights with replacement batteries
Light cotton socks
Bug repellent (like DEET).
Extra change of clothes (including underwear)
2 large garbage bags
Dry bag and/or ziplocks for your phone, camera, valuables
Please note that you will have to thoroughly rinse everything off after you're done