There are many types of aluminum and the ones that are intended for bending operations don't like to be machined, the material tends to push rather than cut and it also wants to gum up on the tool. Before you worry about the cutting speed too much make sure you have a machineable grade of aluminum.