I think the trouble with Wittgenstein's argument is that it assumes the 'I' is looking only at a single world, but it isn't - at least at a naive level. My I isn't a point but a surface that separates my inner world from the outer one. Each of these worlds are not-me, but are disjoint. I have far more confidence in the existence of my inner world, because I apprehend it directly - it's only through an act of faith that I believe there is an outer world because I can only access it through the metaphors that my mind creates in order to express it in terms that I can grasp. Even so, there are different orders of doubt - although i am far more sure of the reality of my inner world than my outer, that doesn't provide certainty. In the end, my experience of my I-ishness is just that - an experience. That experience is itself a sort of illusion, because I don't have any real knowledge of what I am as seen from outside myself so what I experience as 'I' is a projection - I don't think the source of the projection is much like the projected image, but that is an intuition that I can only express and not justify.
I don't think this deep reflection leads away from the OP made by
@just me. My own encounter with solipsism is through experience, not conceptualisation, and may well not actually fit its philosophical definition. It was horrible, there is no other description for it - an experience of hell. I was alone, there was no time, no space - everything was just an extension of myself and illusory. There was no-one else, just me, now .... without future or past. The ultimate bad trip, but without any chemical trigger. It sounds silly when written down, but it wasn't.
The way out is really a vindication of Just Me's OP. I was thrown back inside myself, and several years after this experience, I was rescued by someone deep inside me who was not me. The experience of everything being me was turned inside out - it's still profoundly subjective, and there is still only 'I'. But it's not my 'I' any more but Our 'I'.
All I can say is that the conventional materialistic perspective is profoundly deluded. There is an intimate relationship between all things that exist that means I share their reality with them, and they with me. Together we are an amazing reality.