Telegram isn't the best choice if you're looking for security and privacy. I suggest either #signal or #wire.

Telegram isn't really secure, just search for news about it. https://diasp.org/posts/9429057

You put it yourself: Why would a chat program need your phone number? You can use e.g. an xmpp client with omemo. It won't need your phone number.

XMPP + Conversations (with OMEMO) is much more secure and private. Telegram is not bad, but there are better options

Conversations or Gajim + Omemo
or
Riot.im (matrix)
both are far more secure Than telegram.

Sorry, answering your question, Telegram uses your phone number to identify you, and they need that to make their cloud messaging work, so that you can use it to manage your account and verify your identity in case of need

I agree with Criptos, it's the method they chose to identify you, and to verify every time you set up an account in a new device (they will send a text to that number with a PIN code). Of course they could use that number in many other ways if they wanted to, like any other app that does the same - Signal also is asking for you number, I think.
Telegram will also access your contact list to see who else in your contacts has Telegram installed, much like Signal and WhatsApp do.
Regarding security, despite what others say, I have yet to see how Telegram is more easy to break than Singal/WhatsApp. You have to actively select Private Chat (it's not default) and then the communication is encrypted and is not saved on the cloud. Normal conversations are, and you can access them from any device.
Same with calls, which are by default encrypted, and it shows you the encryption keys to compare with the other party, to ensure they are the same, and the encryption is secure. Quite conveniently, it converts the encryption keys to four emojis, so it is easier to compare them over the call.

Overall, I would say it is as secure as any (it was blocked in Russia exactly because of that), and in terms of privacy as private as WhatsApp and Signal. If you want total anonymity, so no one can link your account to you via a phone number, then chose any of the other suggestions.
But it is quite convenient otherwise, and its bot API is fun to play with and setup your own bots.

@vazelas99@nerdpol.ch thanks for the explanation. I was all happy because they finally fixed the gentoo package, but then I learned it wants my phone number. Phone numbers are tied to people's ID card, so I found it annoying that a chat that self-advertises as privacy aware relies on that and not on an username like, say, XMPP. So in spite of what you said, I still feel uneasy about giving them my phone number. Either way, whatsapp also claims to be securely encrypted, so why one should trust Telegram over whazzap? They work the same, and Telegram's server is also closed source afaict.

I also got put off by their claim that they stopped 78 terrorists - if chats are private as they claim, how did they know those users were terrorists?

@CitizenZero I tried the other programs you suggested, but unfortunately they don't work. Signal (and wired too) installs a ton of dependencies and that electron thing should just die. It's slow, bloated, kills my laptop's battery for no advantage at all. Unfortunately the tone of developers on the github page is quite dismissive in that regard. Signal, anyways, asks me to bind to my phone or something, and why a chat program would need an Android or Apple phone to work on Linux is beyond me. It's just irritating, what's with this phone fetish that everyone seems to have?

About Ring, the gentoo package's been broken since 2017 at least, I opened an issue on their github on november 2017 but it's still not fixed, so I just can't install it.

Last option is Tox, but their team seems to lure catastrophes every other week. Not sure if it's still a thing even.

About XMPP, maybe it's still my best option at this point (I do have a jabber.at account already). Anybody knows how secure voice calls over XMPP are? Do the OTR plugins affect those too?

OMG, do NOT use telegram. Not secure at all. Use Signal. While it may still require a phone number you can register with a google voice number or even a land line.
If you absolutely cannot give a phone number and require complete anonymity, try Riot(matrix.org) with encryption turned on or Tox, or Ricochet.

Voice calls over XMPP are a bit painful.. you can try with Jitsi: it supports zrtp encripted calls.

For encrypted message including audio, video and images the best free option is OMEMO encryption. Whatsapp, and signal both use this protocol OMEMO is the free implementation. OTR is a bit obsolete and does not support multiple devices.
for me the best android xmpp client that supports OMEMO is Conversations, for desktop i use Gajim with OMEMO plugin.