This is probably because of Yahoo/BT's mail deferral policies that have been discussed at length in other recent forum threads in respect of SPF records, for example:
Slow email?
Does your domain have an SPF record and (optionally) is set up for DomainKeys?