When's the Best Time to Go to Yoshiwara? My Notes on Time Slots
Short answer: arrive at the slot you actually booked — don't change it yourself. High-end shops are easier to book from daytime to early evening; late at night, popular companions are often fully booked.

Photos are freely licensed images from Wikimedia Commons (PD/CC0/CC BY/CC BY-SA; credits on each file page), not shop ads.
Short answer: when to go depends on what time you booked. With reservations, lateness is hard to push back and may cut into your own time.
At high-end shops, start times are commonly from midday into the evening (a rhythm like 12, 14, 16 or 19:00 — not fixed at every shop). If you try to book a popular companion very late at night, your chances are usually lower.
Weekends and long holidays are busier. For a first visit I'd go for a weekday or an earlier slot — fewer people and more relaxed communication.
If a shop still offers pickup, the departure time is usually the booked time, so get to the meeting point early; now that most shops have dropped pickup, plan to go yourself and enter 10–20 minutes early.
Check the companions' schedule before choosing a slot, then set the time — not the other way round, hoping for luck.
Related: How to Book in Yoshiwara: What I Say and How I Confirm, Every Time · My First Time in Yoshiwara: From Booking to Leaving