Mine was doing the same thing. I had to put a shim between the starter and bellhousing.
+1 Got the shim at NAPA. They have them for various applications. Just ask at the counter.
My '64 1/2 vert. Ordered May '64. D code 4 speed, handling package, caspian blue, accent group, Ford blue manual top.
'68 vert. driver. Owned since Apr '78. C code AT, AC, PS, P disc B, PT lime gold, standard black interior and top. NOS RF fender and left quarter.New top and folding glass.