Thank you all for the excellent feedback - I ended up trying both sharpie and pencil, both worked without issue. Sharpie (the small, thin kind) in brown seemed to be best in that I could see what I wrote as I wrote it in the darkroom. Pencil worked as well, but I couldn't see what I was writing and the sharpie allows me to write accurately for some reason.
Funny end to this thank you note: after I carefully marked all of the paper and developed it, the pictures all came out exactly the same because I'd forgotten to turn put my filters in the light path. So, all that note taking could have been condensed into "this print is massively overexposed, bullocks."