How is it possible that there is no need to have an intermediary between me and God?

You will not find Muslims prostrating before idols and openly worshipping idols. However, you will find Muslims who have put between themselves and Allah, intermediaries.  The intermediaries might be given different names. In some cultures, they call them saints, auliya. In other cultures, they call them imams (the 12 imams). But these intermediaries have become objects of worship though people are saying we are only praying to them because they are closer to Allah but that was the same thing which was said during the times of the Prophet (ﷺ) when he came with the message of worshipping Allah directly. As Allah said,

“Call on Me in prayer and I will answer you (in prayer)” [1]

But his time, people were saying that they were only worshipping these other idols/gods in order to bring them closer to Allah. As people would say how can you, covered in sin, turn to Allah who is so pure? You need to turn to somebody who is purer and they can help get your message to Allah. If you want to try to speak to the ruler of Saudi Arabia, you cannot just go to his palace, knock on his door and talk to him. You have to talk to this one who will talk to that one until eventually, what you are asking for can get to the ruler.

Allah did not tell us to call on Muhammad (ﷺ) and I will answer you. Instead, He said call on Me and I will answer you. That is the moral way. It is immoral to call on others besides Allah. In this way, immorality, with regards to our relationship with Allah, has crept in. We need to get back to the moral message of Islam about which Prophet Muhammad (ﷺ) had said:

“I was only sent to perfect for you the highest of moral character traits.”[2]

References:

[1] [Qur’an 40:60]

[2] [Narrated by al-Bukhaari in al-Adab al-Mufrad (273)]