You can read about the evolution from the pluperfect tenses from Latin to the current imperfect tenses in chapter 24.2 of the *Gramática española" by the RAE. I'll translate the most important parts.
24.2a [...] The -ra variant of the imperfect tense comes from the pluperfect indicative tense of the Latin language (amavĕram 'había amado'), while the -se variant comes from the pluperfect subjunctive (amavissem 'hubiera amado').
This just confirms what you said in the question, but let's move on...
24.2h The use of CANTARA as HABÍA CANTADO is etymological [...]. This is the main interpretation in medieval texts, in which it appears both in main sentences as in relative ones.
So in a sentence such as "el rey oyó bien lo que dixera su nieto", dixera must be understood as había dicho. The use of CANTARA as HABÍA CANTADO is still alive in the languages from the Nothwest part of the Iberian peninsula.
In the next paragraphs (24.2m and following) you can read more about the evolution of the tense.
24.2m The presence of the CANTARA form in the apodosis of conditional sentences is characteristic of the classical language, in which it must be understood as CANTARÍA: Si en encuentro me cayera, yo le diera tal golpe con mi bordón que nunca otra vez hiziera otro tanto.
And finally:
24.2n From the apodosis of conditional sentences, the CANTARA form passed to the protasis, now of general use [...]. The CANTARA form thus started to be used as the imperfect of subjunctive and to compete with the CANTASE form.
Maybe this is just the result of what the RAE states in paragraph 24.2a:
The imperfect (CANTARA or CANTASE) is the most complex tense of the subjunctive mood, both for the syntactic contexts in which it is used as for the variety of meanings it conveys.
More on this on paragraphs 47.8t and 47.8u:
47.8t [...] To express the unreality of a past tense in the symmetrical scheme Si {TUVIERA ~ TUVIESE}, {DIERA ~ DIESE}, the -se forms of the protasis corresponded to -ra forms in the apodosis, with the meaning of HABRÍA CANTADO. Thus the equivalent in the current language to Sabet bien que, si ellos le viessen, non escapara de muert would be Sabed bien que si ellos lo hubiesen visto, no habría escapado de la muerte. Since the Middle Age the CANTARA form extended to the protasis [...].
47.8u The progressive extension of imperfect subjunctive from apodosis to protasis occurred from the second half of the thirteenth century. The symmetrical scheme with both forms in -ra was generalized from the last quarter of the fourteenth century as a prototypical expression of past unreality: No vos quiero negar mi fazienda, que bien vos pudiera mentir si quisiera; También me vengara yo si pudiera, fuera o no fuera armado caballero, pero no pude.
Possibly the CANTARA form was just a complex enough tense for the people to ignore the proper way to use, but common enough to know that it must be used somewhere, making the tense do a trip among several positions in a sentence until the one it is used today.