The "best" way to do what I think (based on the code you provided, I suppose you want pairs (source IP address, destination IP address), not IP packets), you want:
>>> set((p[IP].src, p[IP].dst) for p in PcapReader('file.pcap') if IP in p)
PCAP, set .
, Scapy IP, IP:
>>> IP.payload_guess = []
.