You bring up some interesting points:
You suggest that improperly managed time travel could result in chaos, and so it is denied to those who do not have sufficient knowledge of its impact on the world.
I would suggest that given what little we already know about chaos theory, that even the shortest and minimally impacting time travel event could result in massive changes to the historical timeline as time moves forward (or backward?) from the event. This was hinted at in Ray Bradbury's short story about the hunters who go back in time to shoot dinosaurs that are just about to die anyway. One guy gets frightened, jumps off the hovering track and accidentally squishes a single butterfly. When they get back to the future, a fascistic presidential candidate who was about to lose the next election ends up winning. Also, the letter 'c' is replaced by 'k' and other small changes have occured because of the one dead butterfly.
Even though this was only a short story, I believe that Bradbury had it right. If I just flash on in the past for a second and then go back to the present, if someone happened to see me, that could cause a whole bunch of things to happen, especially if the person undergoes a religious conversion because he believed I was a saint, or goes crazy because he believed I was a devil.
Thus I think that if knowledge is a criteria for time travel, then one would need to have COMPLETE knowledge, i.e. one would have to be God. Or at least an angel or other being in communion with God.
The slightest error of judgment could lead to horrible consequences unless every combination and permutation can be calculated and certified.
The Bible may be the complete and inerrant word of God, but unfortunately we are finite beings who cannot fully fathom all that the Bible has to say. That is one of the reasons there are so many schisms and denominations within EVERY religion. There are at least four major variants of Judaism that I am aware of, and at least two versions of Islam (though there seem to be progressive and militant branches of each of those.) So no religion or religious text seems to be clear enough to us finite people to lead to only one set of adherents.
So if you are right, then I imagine that only God and those in communion with him will be allowed to time travel.
In the meantime, although it may sound contradictory, it might be best if each of us did his or her "own thing" so that as time goes forward we will try out all the possible combinations and permutations, and hopefully through some form of cultural evolution we will slowly but surely hit upon the best set of laws/behaviors that we as finite beings can come up with. Even if we never succeed in traveling through time, we will certainly have a happier time as we whiz through space!