Yes technically speaking,moving to a previous location in time may lead your body to be in that position but the rest of the world not,if things were not planned correctly youd end up in outer space.
This is easy to counter,we know that the earth will be in the exact position 24 hours from now,so you can counter by going back in time at equal position times,
but as far as i know quantum displacement could occur if the plan was like teleportation,in otherwords if the plan is to end up back in the device(time machine)when you get back then you could have problems
unless you could be 100% accurate.
Obviously in time teleportation,the machine is built and you can only travel between the time it is built and the life of the machine i.e if it is built today you can only go back to today,cant go before the date of the machines completion.