It's not hard, it's a challenge.
challenge #1: if you have a 1/2 round file it's easy, insert file and rotate the file against the sides of the hole. The file will cut sideways.
You could use the push-pull technique if you sandwich the board with some sort of ply or the like.