Accepted AnswerEveryone can benefit, but It seems mechanical is the way to go. If you search on ebay for "digital depth gauge" you will find many different digital depth gauges.
Most of them have a port for digital signal readout, so that means measurement can be used in micro controller that controls spiderbot.
For 25$ one can get some decent tool that reads up to 0.02mm accuracy, 0.01mm resolution.
Accepted AnswerI found a more recent version by another designer that scans filament in both directions
Also this uses arduino so programming the thing is more easy.