Is there any difference? What's their usage? When should one be used instead of the other one?
Examples:
El niño debe de hacer su tarea.
El niño debe hacer su tarea.
The difference is simple:
Deber + infinitive is used to express obligation:
You must do it as soon as possible.
Deber de + infinitive is used to express probability or supposition:
It must be very early.
Sometimes, in the second case the preposition "de" is omitted, so it might be confusing. So the first case could also be interpretated as the second one, but the context of the sentence would give you the right meaning.
In your example I'd say that the correct way is "El niño debe hacer su tarea", because it's likely that the meaning is obligation. But for example you mean something like:
The test is answered, the child must have done his homework
then you would use "deber de" (but with another tense) because this is a supposition:
El test está contestado, el niño debe de haber hecho sus tareas.
You can find on the RAE the official reference about the topic.
El niño debe hacer su tarea.
This sentence means obligation: The kid must do his job.
A similar sentence could be "El niño tiene que hacer su tarea."
El niño debe de hacer su tarea.
The construction "deber de" means supposition or possibility. I'm not sure this usage applies to your example as is. But another example can be:
Deben de ser las ocho y media. - It must be half past eight.
If you want to read more, check the "deber" vs "deber de" comparison on the RAE, provided by Gonzalo Medina.
Others have explained the theoretical distinction. However, in practice, "deber de" can be used as a slightly softer equivalent for "deber."
Examples found by googling the exact phrase "debes de":
7 aspectos que debes de mantener en secreto siempre según la sabiduría Hindú
Tú debes de averiguarlo todo por ti mismo
Debes de aceptar que eres una canalla
¿Qué información debes tener en tu página, y cómo lo debes de demostrar? [...] ¿Cuáles páginas debe de tener?
It's true that some of this is imperfect translating (primarily from English), but that's not the whole story, in my opinion.
Explanation of softening: Unsoftened, we hear a strong expression of what must be done; softened, we hear something that sounds more like guidance than an order. Or perhaps I should say, the person who adds "de" can at least hope that by doing so, he is coming across a bit less heavy-handed.
(And then there's deberías de....)