The difference is that when we conjecture about the past, we will have anchored ourselves in some period of time. For example:
Ayer serían las ocho cuando llegué a casa.
The time frame is yesterday/when I got home, and since we speculating about something concurrent/contemporaneous with that timeframe, we use the simple conditional.
If we establish a time frame in the past, but we want to speculate about something happening before that time frame (that is, it has already finished), then we use the conditional perfect:
Ayer cuando llegué a casa, mi familia habría cenado porque vi los platos sucios en la mesa.
Here we anchored the time frame in yesterday/when I got home, but our speculation is about what happened before when I got home, and thus being a completed (or perfect) action, we use conditional perfect.
Notice that this is the same difference between preterite/imperfect and pluperfect:
Ayer eran las ocho cuando llegué a casa.
Ayer cuando llegué a casa, mi familia había cenado porque vi los platos sucios en la mesa.1
1. This one would sound more natural with the first two clauses swapped and a few other minor elements changed (mi familia ya había cenado cuando llegué a casa, ya que…) but I kept the other order to show the parallel with the conditional perfect.