Key ingredients to achieve effective 1/O asyne caching c/e Impact of I/O latency Typical I/O pattern Source Destination ●send a packet to First bit sent destination sending time ●wait for ack Last bit sent ping time Ack ●go to next block First bit sent time sending time efficiency sending time+ping time latency ze mg 6/57 S.Ponce-CERN
Key ingredients to achieve effective I/O 6 / 57 S. Ponce - CERN async IOopt struct caching c/c latency async msgQ Impact of I/O latency Typical I/O pattern send a packet to destination wait for ack go to next block Source Destination time First bit sent Last bit sent Ack First bit sent sending time ping time efficiency = sending time sending time + ping time
Key ingredients to achieve effective 1/O asyne Copt stnct caching c/e Some mathematics Definitions efficiency= sending time (1) sending time+ping time data size sending time = speed (2) ping size speed ping time (3) Gives 1 efficiency (4) 1+ ping size data size efficiency data size= 1-efficiency ping size (5) latency ac mig 7/57 S.Ponce-CERN
Key ingredients to achieve effective I/O 7 / 57 S. Ponce - CERN async IOopt struct caching c/c latency async msgQ Some mathematics Definitions efficiency = sending time sending time + ping time (1) sending time = data size speed (2) ping size = speed ∗ ping time (3) Gives efficiency = 1 1 + ping size data size (4) data size = efficiency 1 − efficiency ∗ ping size (5)
Key ingredients to achieve effective 1/O asyne Copt stnct caching c/e Some (bad)numbers Consequences for 10KB blocks Usage Speed Latency Ping Size Efficiency CC 1GBs-1 100μs 10kB 50% cc 10GBs-1 100μs 100kB 9% WAN 1GBs-1 10ms 1MB 1% WAN 10GBs-1 10ms 10MB 1%0 UK-JP 10GBs-1 250ms 250MB 0.04%0 latency ac mig 8/57 S.Ponce-CERN
Key ingredients to achieve effective I/O 8 / 57 S. Ponce - CERN async IOopt struct caching c/c latency async msgQ Some (bad) numbers Consequences for 10KB blocks Usage Speed Latency Ping Size Efficiency CC 1 GB s−1 100 ➭s 10 kB 50% CC 10 GB s−1 100 ➭s 100 kB 9% WAN 1 GB s−1 10 ms 1 MB 1% WAN 10 GB s−1 10 ms 10 MB 1h UK-JP 10 GB s−1 250 ms 250 MB 0.04h
Key ingredients to achieve effective 1/O asyne IOopt snact caching c/c More bad numbers Data size for decent efficiency Usage Speed Latency 50%efficiency 91%efficiency CC 1GBs-1 100μs 10kB 100kB CC 10GBs-1 100μs 100kB 1MB WAN 1GBs-1 10ms 1MB 10 MB WAN 10GBs-1 10ms 10MB 100MB UK-JP 10GBs-1 250ms 250MB 2.5GB Remember maximum TCP packet size is 64 KiB latency ac mig 9/57 S.Ponce-CERN
Key ingredients to achieve effective I/O 9 / 57 S. Ponce - CERN async IOopt struct caching c/c latency async msgQ More bad numbers Data size for decent efficiency Usage Speed Latency 50% efficiency 91% efficiency CC 1 GB s−1 100 ➭s 10 kB 100 kB CC 10 GB s−1 100 ➭s 100 kB 1 MB WAN 1 GB s−1 10 ms 1 MB 10 MB WAN 10 GB s−1 10 ms 10 MB 100 MB UK-JP 10 GB s−1 250 ms 250 MB 2.5 GB Remember maximum TCP packet size is 64 KiB
Key ingredients to achieve effective 1/O 4aync0o时natt cachinc The solution asynchronous I/O Do not wait the acknowledgment Source Destination ocall an API to express what First bit 1 should be transfered o this immediately returns 电 without doing much ●get called back when transfer has been done/has failed 超093 Ack 3 time latency asyne mig 10/57 S.Ponce-CERN
Key ingredients to achieve effective I/O 10 / 57 S. Ponce - CERN async IOopt struct caching c/c latency async msgQ The solution : asynchronous I/O Do not wait the acknowledgment ! call an API to express what should be transfered this immediately returns without doing much get called back when transfer has been done/has failed Source Destination time First bit 1 Last bit 1 Ack 1 First bit 2 Last bit 2 Ack 2 First bit 3 Last bit 3 Ack 3 First bit 2 Last bit 2 Ack 2 First bit 3 Last bit 3 Ack 3 First bit 4 Last bit 4 Ack 4 First bit 5 Last bit 5 Ack 5