Is there any difference? What's their usage? When should one be used instead of the other one?
El niño debe de hacer su tarea.
El niño debe hacer su tarea.
The difference is simple:
Sometimes, in the second case the preposition "de" is omitted, so it might be confusing. So the first case could also be interpretated as the second one, but the context of the sentence would give you the right meaning.
In your example I'd say that the correct way is "El niño debe hacer su tarea", because it's likely that the meaning is obligation. But for example you mean something like:
then you would use "deber de" (but with another tense) because this is a supposition:
You can find on the RAE the official reference about the topic.
If you want to read more, check the "deber" vs "deber de" comparison on the RAE, provided by Gonzalo Medina.