There's a famous example in philosophy the problem of identity that's pretty relevant here, although I forget its name.
Basically, a wooden ship is built, but after some time, it starts rotting. Over the years the rotting planks are replaced by new planks, one by one, and thrown into a pile, until none of the original planks are left. Is it still the same ship?
The plank-ship relationship is similar to the thought-thinker relationship. I think the answer is that the "ship" or "self" are only abstract concepts. Unlike an atom of a certain isotope and element, a ship is not the same as another ship. Atoms are inherently atoms, ships are only the relationship between the atoms in the planks of wood.
There were some experiments done with people whose two halves of the brain were separated in order to stop seizures. Each eye is controlled by one half of the brain. Different forms of communication are also controlled by different halfs of the brain. When participants were asked to indicate a certain picture, the answer varied depending on which eye they saw it with, and which communication method was used.
tl;dr - The idea of a cohesive self is a misconception.