Endure the Silence?

At work recently, we were put into different rooms. I was put in with a colleague, who I'll imaginatively refer to as Colleague X. Now, Colleague X is man of few words, or more specifically, a man of no words. Yes, entire days pass where the only conversation is a single, abbreviated "see you later". Other than that it's complete silence all day. Now, up until now my attitude is that if Colleague X isn't going to make any conversation then I'm not going to bother either. As a result of this I'm now as quiet as Colleague X. I've been called quiet in the past and often am, but Colleague X makes me look like Graham Norton/Lily Savage/Dame Edna rolled into one (in conversation terms, not dress sense).
The dilemma is this: either I make more of an effort to talk to Colleague X and maybe he'll come out of his shell, or I continue as I am and take the attitude "well if he's not making any effort then he can fuck right off!"
The latter, I think. Dilemma resolved.