It's a completely bonkers idea which is why I love it and think it is genius!
Presumably you are going to soak it in PVA or something similar or the cars will rip it to shreds (which will also make handling difficult!)
This would also make it hard enough to buff it up with a wax based furniture polish to get rid of any last traces of grip
Don't forget to post your findings and some photos - this is one for my "to do" list I think
Just a thought if you use something like the anaglypta paper (do they still make this?) - this might raise the cars too far out of the slot so is there any way you could ease the rails up by a mm or two? You could really go to town on the unevenness of the surface and still run standard slot cars over it then.