The short answer would be
Que tengas un buen día
Which express your desire for the other person to actually
"have a nice day".
"Ten un buen día" is the literal translation but it doesn't express the sentiment behind
"have a nice day". It could be used to maybe cheer someone up; something like
"Hey you, go and have a nice day" or something like that.
"Que vaya bien" is not entirely correct. It would be more like
"Que te vaya bien" and it is most commonly used when someone is going to do something, maybe when someone is going to a job interview, you can tell him
"Que te vaya bien" to express your desire for the interview to go well.
But for saying goodbye the most common way is
"Que tengas un buen día",
"Que tengas buen día" or as mentioned in the comments
Also remember that, in some cultures and countries, when you want to speak with more respect (to older people, for example) you would say
"Que tenga un buen día" since
"tengas" is a more personal manner.