Create your prop/brush/whatever you want to climb, place that in your desired location. Then create another brush parrallel to the entity/brush you just created, space it out as close as you can get it to the brush/entity, but try not to let them touch. Also make it the exact (or as close as you can get it to) length and width of your entity/brush. Apply the toolsnodraw texture to the brush that you just created and then hit ctrl + T (while selecting) and where it says "func_***" rename it func_ladder. Compile and test.
What this does is creates an invisible surface that is the actual climbing material. The closer you create this brush to the "ladder", the better it will look.