It doesn't matter whether they are Muslim or not Muslim.
All children, regardless of their families' religions or what religion they identify as, are closer to the pure/uncorrupted/sinless nature known as the fitrah that recognizes and acknowledges the truth, the unseen, and good and evil. (Perhaps this is why children are often quite wise!)
At the same time, all children are still developing the psychological capabilities that are necessary to make solid moral judgments and act on them (for instance, empathy, the ability to put the good of the group above the good of the individual, self-restraint, an understanding of cause and effect, an understanding of the permanency of death, etc).
This is not to say that children cannot make good and bad ethical decisions, or that children do not have authentic religious experiences and beliefs or even conversions. All of that happens; it depends on the child and the age. However, they are not held responsible in the same way that mature people are.
So, children go to barzakh (the intermediary realm after death before the final judgment) and then to paradise. It is said that they skip some things that they are not mature enough for, such as some of the questioning.
The experience of children in the afterlife will likely be different from that of mature adults since part of our job in this life is to learn more about spiritual and other realities. So, they may have some catching up to do, or maybe their experience will just be different, just as different adults will also have different experiences.
On the other hand, we should not underestimate children's souls (in the same way that people often underestimate children and confine them to things that are "childish"). The soul is a noble and honourable creation of Allah and this is the same for children or adults.
