¿Es correcta esta frase?
Coge la bolsa por el asa pequeño.
It is incorrect. The adjective pequeño must agree with the noun asa which is feminine.
The tricky part is that, since the word asa starts with an stressed "a" (/a/ tónica in Spanish), it won't use the feminine article "la," but rather the masculine "el" to avoid "la asa," which would sound awkward. You can see this happen with other feminine words that start with stressed "a":
even words that start with an "h" followed by stressed "a":
Note that you would not be doing this if the "a" were not stressed
La arena (stress is on the letter "e")
La actitud (stress is on the letter "u")
La agencia (stress is on the letter "e")
You will still use this "feminized" form of the article "el" for declinations of words that start with stressed "a," even if the declined word loses the stress on the "a" (/a/ átona in Spanish).
El agüita (not la agüita)
You won't be using it either word composed feminine words in which the first word starts with an stressed "a" but the composed word has a non-stressed "a":
La aguamarina (not el aguamarina. Stress in in letter "i")
The word asa has a stressed "a", so the correct form would be
Coge la bolsa por el asa pequeña.