Hello Vishnu,
As per my knowledge you need whitelist all the IPs, Because the message flow can happen through any server nodes from IP.
but I guess even if you whitelist with host it should work, Since host will be unique.
Why don't you try it out.