Suppose a materialistic universe, i.e. the time traveler is merely the sum of the molecules and chemical reactions that make him up.
Suppose, also, that space-time is like space in that we can imagine being at one point in space-time and traveling to any other point in space-time.
Now imagine one molecule in the brain of a time traveler which holds one part of a memory. That molecule traces out a continuous line in four dimensions through space-time.
Now suppose the time traveler goes back in time. Instead of tracing out a continuous line, there is now a gap as the molecule is transported back near one of the points it had occupied previously in the life of the time traveler.
As the time traveler thinks and perceives, the state of the molecule changes to reflect the changes in his brain. There is no reason to believe that the molecule in the time traveler's "original" brain need be in the same state as the molecule transported back in time. Even if the time traveler should move in such a way as to cause the molecule in his current brain to overlap at the exact same space-time coordinate as his original brain molecule. There is no reason to believe that the function has to be a relation where there can be only one unique dependent value.
The problem is that now the time traveler can cause a change in the path of the "original" molecule. If the time traveler just gives his original self a little shove and then goes back to the future, he has altered the space-time path of his original self. What does this mean?
If you believe that space-time is just a background upon which actions occur, then altering a path in space-time is no different than altering a path in space. However, if you believe that the space-time continuum is inexhorably linked with the movements of particles, etc. then once a path has been established in space-time it must be fixed. So there could then not be any time travel, or at least only extremely limited versions such as the ability to look back in time without affecting it.
So when you as a time traveler go back to the future, you wouldn't necessarily have to catch up with the modified "original" self. When you got back to the future, you might experience the same thing as when you went into the past, i.e. another version of you staring back at you.
I'm not suggesting a multiverse, but it might be that once you stray from a continuous space-time path, you may necessarily create copies of yourself. This would be similar to what happened to Riker on Star Trek when the transporter accidentally made two copies of him.
This is actually a method that might work to transport people in the far distant future: store a bunch of biological protoplasm at some remote location, scan the heck out of a person, send the scanned info to the blob of goo, and use it to turn the goo into the original person. The first person is still back at home, while there is a new person with all the same qualities (down to memories) walking around on Pluto.
Time travel may work the same way: as soon as you walk through the wormhole to go back into the past, the "original" time traveler, now affected by the returned time traveler, appears around the corner.