I think there will only be a problem if the lens did not go right to infinity, going past is not a problem.

Some Olympus lenses I used many years ago were sent to be adjusted because they did not focus right at infinity. I never knew the cause but after adjustment they worked perfectly.