Multicast traffic specially the one generated by Microsoft's NLB Cluster can flood the network even if it is switched network. Network engineers will often feel the switches acting like a classic hub completely defeating the purpose of switched networks. Microsoft intentially did this in their NLB Cluster by design to achieve the load balancing or express delivery or response to network traffic dealt by their load balancers.
In order to track or detect symptoms you will have to trace network packets across network from any port. This can be done using Wireshark on Windows and Snoop on Solaris Unix systems.
Packet trace will usually be carried out on one of the network interface connected to VLAN or network in question. In below example we are using Unix snoop command but we would also use other commands to determine what is required.
$ bash # access the Bash Shell
$ ifconfig -a # list the interface details and write down the interface you want to investigate.
Now we are going to run the snoop on one of the interfaces we wanted to investigate.
snoop -c e1000g -v arp 2>&1 > /tmp/e1000g.txt
If you have more than one interface you may have to run below at the same time.
snoop -c e1000g1 -v arp 2>&1 > /tmp/e1000g1.txt & snoop -c e1000g2 -v arp 2>&1 > /tmp/e1000g2.txt
Break the connection using Ctrl+C or Break and inspect the output files to find any multicast packet.
You can also run below command to see the live traffic with word multicast in filter.
snoop -c e1000g -v arp | grep multicast
Initially you will have to ensure that suspected IP Addresses are resolving ARP entries while you do the trace. If they are already learned they may not appear in arp trace.
List the arp entries by using following command.
arp -an | egrep "192.168.0.1|192.168.1|etc"
In order to delete specific arp entries uses below.
arp -d 192.168.0.10 || arp -d "another ip address"
Now issue a ping while you are tracing the packets.
ping 192.168.0.1 && ping 192.168.0.2 && etc
More to come, however above should be sufficient for you ....