CSC 2018 Data Technologies Exercises Exercises Link Andreas-Joachim Peters CERN IT-ST CERN CERN CSC 2018 Data Technology Exercises
CSC 2018 Data Technology Exercises CSC 2018 Data Technologies Exercises Andreas-Joachim Peters CERN IT-ST Exercises Link
CERN Exercises Overview 1.IO system What do you know already? IOPS,bandwidth,latency blocksize media and their characteristics cache 1O optimisation strategies 1st hour how to debug Io problems 2.Redundancy Technology 2nd hour Parity for RAID technology 3.Cloud Storage Technology 3rd 4th hour Scalability,Hashing,Indexing,,Deduplication CERN CSC 2018 Data Technology Exercises
CSC 2018 Data Technology Exercises Exercises Overview 1. IO system • What do you know already? • IOPS, bandwidth, latency & blocksize • media and their characteristics • cache & IO optimisation strategies • how to debug IO problems 2. Redundancy Technology • Parity for RAID technology 3. Cloud Storage Technology • Scalability, Hashing, Indexing,, Deduplication 1st hour 2nd hour 3rd + 4th hour
CERN lutorial Exercise 1 A common user experience:"My IO intensive application does not run fast enough -why?" Three important questions to answer what performance should we expect of our IO system? how can we measure limitations? how can we inspect the IO of our application? To answer this,we need a basic understanding of the IO system, some measurement and debugging tools CERN CSC 2018 Data Technology Exercises
CSC 2018 Data Technology Exercises Tutorial - Exercise 1 • A common user experience: “My IO intensive application does not run fast enough - why?” • Three important questions to answer • what performance should we expect of our IO system? • how can we measure limitations? • how can we inspect the IO of our application? • To answer this, we need a basic understanding of the IO system, some measurement and debugging tools
CERN Interlude before we start Let's see what you already know .. 光S16H片 Please open this anonymous online poll with your phone or laptop .. http://etc.ch/WAb5 We will repeat this poll in the end of the exercises and discuss the correct answers! CERN） CSC 2018 Data Technology Exercises
CSC 2018 Data Technology Exercises Interlude before we start … Let’s see what you already know … http://etc.ch/WAb5 Please open this anonymous online poll with your phone or laptop … We will repeat this poll in the end of the exercises and discuss the correct answers!
CERN Linux 10 System in non-virtual machines Local 1O Since we measure here, Measurement Tools User read.write 244 GLIBC strace System-call Interface SCI ere房he meta data Cache implemented Virtual Filesystem Switch VFS (not important for the 已X日1Cs XFS EXT4 FS(x) Here is the data cache Skip imalementod important for the Block Layer CxICSES using vmstat iostat Device Drivers KERNEL CERN） CSC 2018 Data Technology Exercises
CSC 2018 Data Technology Exercises Linux IO System Skip caching using direct IO Measurement Tools in non-virtual machines Local IO