Join the discussion p2p.wror.com Wrox Programmer to Programmer Wrox A Wiry Brand Professional CUDA C Programming Foreword by Dr.Barbara Chapman,Center for Advanced Computing Data Systems,University of Houston ②nVIDIA John Cheng,Max Grossman,Ty McKercher w.it-ebooks.itif色
www.it-ebooks.info
Professional CUDAC Programming Published by John Wiley Sons,Inc. 10475 Crosspoint Boulevard Indianapolis,IN 46256 www.wiley.com Copyright 2014 by John Wiley Sons,Inc.,Indianapolis,Indiana Published simultaneously in Canada ISBN:978-1-118-73932-7 ISBN:978-1-118-73927-3(ebk) IsBN:978-1-118-73931-0(ebk) Manufactured in the United States of America 10987654321 No part of this publication may be reproduced,stored in a retrieval system or transmitted in any form or by any means, electronic,mechanical,photocopying,recording,scanning or otherwise,except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act,without either the prior written permission of the Publisher,or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center,222 Rosewood Drive,Danvers, MA 01923,(978)750-8400,fax(978)646-8600.Requests to the Publisher for permission should be addressed to the Permissions Department,John Wiley Sons,Inc.,111 River Street,Hoboken,NJ 07030,(201)748-6011,fax(201)748- 6008,or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty:The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties,including without limitation warranties of fitness for a particular purpose.No warranty may be created or extended by sales or pro- motional materials.The advice and strategies contained herein may not be suitable for every situation.This work is sold with the understanding that the publisher is not engaged in rendering legal,accounting,or other professional services. If professional assistance is required,the services of a competent professional person should be sought.Neither the pub- lisher nor the author shall be liable for damages arising herefrom.The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Web site may provide or recommendations it may make.Further,readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services please contact our Customer Care Department within the United States at(877)762-2974,outside the United States at(317)572-3993 or fax (317)572-4002. Wiley publishes in a variety of print and electronic formats and by print-on-demand.Some material included with stan- dard print versions of this book may not be included in e-books or in print-on-demand.If this book refers to media such as a CD or DVD that is not included in the version you purchased,you may download this material at http://book- support.wiley.com.For more information about Wiley products,visit www.wiley.com. Library of Congress Control Number:2014937184 Trademarks:Wiley,Wrox,the Wrox logo,Programmer to Programmer,and related trade dress are trademarks or regis- tered trademarks of John Wiley Sons,Inc.and/or its affiliates,in the United States and other countries,and may not be used without written permission.CUDA is a registered trademark of NVIDIA Corporation.All other trademarks are the property of their respective owners.John Wiley Sons,Inc.,is not associated with any product or vendor mentioned in this book. www.it-ebooks.info
ffi rs.indd 08/07/2014 Page ii Professional CUDA® C Programming Published by John Wiley & Sons, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2014 by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-1-118-73932-7 ISBN: 978-1-118-73927-3 (ebk) ISBN: 978-1-118-73931-0 (ebk) Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748- 6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifi cally disclaim all warranties, including without limitation warranties of fi tness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Web site may provide or recommendations it may make. Further, readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com. Library of Congress Control Number: 2014937184 Trademarks: Wiley, Wrox, the Wrox logo, Programmer to Programmer, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affi liates, in the United States and other countries, and may not be used without written permission. CUDA is a registered trademark of NVIDIA Corporation. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc., is not associated with any product or vendor mentioned in this book. www.it-ebooks.info
CREDITS ACQUISITIONS EDITOR BUSINESS MANAGER Mary James Amy Knies PROJECT EDITOR VICE PRESIDENT AND EXECUTIVE GROUP Martin V.Minner PUBLISHER Richard Swadley TECHNICAL EDITORS Wei Zhang ASSOCIATE PUBLISHER Chao Zhao Jim Minatel PRODUCTION MANAGER PROJECT COORDINATOR,COVER Kathleen Wisor Patrick Redmond COPY EDITOR PROOFREADER Katherine Burt Nancy Carrasco MANAGER OF CONTENT DEVELOPMENT AND INDEXER ASSEMBLY Johnna VanHoose Dinse Mary Beth Wakefield COVER DESIGNER DIRECTOR OF COMMUNITY MARKETING Wiley David Mayhew COVER IMAGE MARKETING MANAGER ©iStock.com/fatido Carrie Sherrill www.it-ebooks.info
ffi rs.indd 08/07/2014 Page iii ACQUISITIONS EDITOR Mary James PROJECT EDITOR Martin V. Minner TECHNICAL EDITORS Wei Zhang Chao Zhao PRODUCTION MANAGER Kathleen Wisor COPY EDITOR Katherine Burt MANAGER OF CONTENT DEVELOPMENT AND ASSEMBLY Mary Beth Wakefi eld DIRECTOR OF COMMUNITY MARKETING David Mayhew MARKETING MANAGER Carrie Sherrill BUSINESS MANAGER Amy Knies VICE PRESIDENT AND EXECUTIVE GROUP PUBLISHER Richard Swadley ASSOCIATE PUBLISHER Jim Minatel PROJECT COORDINATOR, COVER Patrick Redmond PROOFREADER Nancy Carrasco INDEXER Johnna VanHoose Dinse COVER DESIGNER Wiley COVER IMAGE © iStock.com/fatido CREDITS www.it-ebooks.info
ABOUT THE AUTHORS JOHN(RUNWEI)CHENG is a research scientist with extensive industry experience in high-performance computing on heterogeneous computing platforms.Before joining the oil and gas industry,John worked in the finance industry for more than ten years as an expert in computational intelligence,providing advanced solutions based on genetic algorithms hybridized with data mining and statistical learning to solve real world business challenges.As an internationally recognized researcher in the field of genetic algorithms and their application to industrial engineering,John has co-authored three books.John's first book,Genetic Algorithms and Engineering Design,published by John Wiley and Sons in 1997,is still used as a textbook in universities worldwide.John has a wide range of experience in both academic research and industry development,and is gifted in making complex subjects accessible to readers with a concise,illustrative,and edifying approach.John earned his doctoral degree in computational intelligence from the Tokyo Institute of Technology. MAX GROSSMAN has been working as a developer with various GPU programming models for nearly a decade.His experience is focused in developing novel GPU pro- gramming models and implementing scientific algorithms on GPU hardware.Max has applied GPUs to a wide range of domains,including geoscience,plasma phys- ics,medical imaging,and machine learning,and enjoys understanding the compu- tational patterns of new domains and finding new and unusual ways to apply GPUs to them.Lessons learned from these domains help to guide Max's work in programming models and frameworks.Max earned his degree in computer science from Rice University with a focus in paral- lel computing TY MCKERCHER is a Principal Solution Architect with NVIDIA,leading a team that specializes in visual computing systems architecture across multiple industries. He often serves as a liaison between customer and product engineering teams dur- ing emerging technology evaluations.He has been engaged in CUDA-based projects since he participated in the first CUDA kitchen training session held at NVIDIA headquarters in 2006.Since then,Ty has helped architect GPU-based supercomput- ing environments at some of the largest and most demanding production datacenters in the world. Ty earned his mathematics degree with emphasis in geophysics and computer science from the Colorado School of Mines. www.it-ebooks.info
ffi rs.indd 08/07/2014 Page v ABOUT THE AUTHORS JOHN (RUNWEI) CHENG is a research scientist with extensive industry experience in high-performance computing on heterogeneous computing platforms. Before joining the oil and gas industry, John worked in the fi nance industry for more than ten years as an expert in computational intelligence, providing advanced solutions based on genetic algorithms hybridized with data mining and statistical learning to solve real world business challenges. As an internationally recognized researcher in the fi eld of genetic algorithms and their application to industrial engineering, John has co-authored three books. John’s fi rst book, Genetic Algorithms and Engineering Design, published by John Wiley and Sons in 1997, is still used as a textbook in universities worldwide. John has a wide range of experience in both academic research and industry development, and is gifted in making complex subjects accessible to readers with a concise, illustrative, and edifying approach. John earned his doctoral degree in computational intelligence from the Tokyo Institute of Technology. MAX GROSSMAN has been working as a developer with various GPU programming models for nearly a decade. His experience is focused in developing novel GPU programming models and implementing scientifi c algorithms on GPU hardware. Max has applied GPUs to a wide range of domains, including geoscience, plasma physics, medical imaging, and machine learning, and enjoys understanding the computational patterns of new domains and fi nding new and unusual ways to apply GPUs to them. Lessons learned from these domains help to guide Max’s work in programming models and frameworks. Max earned his degree in computer science from Rice University with a focus in parallel computing. TY MCKERCHER is a Principal Solution Architect with NVIDIA, leading a team that specializes in visual computing systems architecture across multiple industries. He often serves as a liaison between customer and product engineering teams during emerging technology evaluations. He has been engaged in CUDA-based projects since he participated in the fi rst CUDA kitchen training session held at NVIDIA headquarters in 2006. Since then, Ty has helped architect GPU-based supercomputing environments at some of the largest and most demanding production datacenters in the world. Ty earned his mathematics degree with emphasis in geophysics and computer science from the Colorado School of Mines. www.it-ebooks.info
ABOUT THE TECHNICAL EDITORS WEI ZHANG is a scientific programmer and has been working in the high-performance computing area for 15 years.He has developed or co-developed many scientific software packages for molecu- lar simulation,computer-aided drug design,EM structure reconstruction,and seismic depth imag- ing.He is now focusing his effort on improving the performance of seismic data processing using new technologies such as CUDA. CHAO ZHAO joined Chevron in 2008 and currently serves as Geophysical Application Software Development Specialist.In this role,Chao is responsible for designing and developing software products for geoscientists.Prior to joining Chevron,Chao was a software developer for Knowledge Systems Inc.and Seismic Micro Technology Inc.With more than 13 years of software developing experience in the exploration and production industry,Chao has gained rich knowledge in the fields of geology and geophysics.Having a broad education in science,Chao likes to see CUDA program- ming used widely in scientific research and enjoys contributing to it as much as he can.He holds a Bachelor of Science degree in chemistry from Peking University and a Master of Science in computer science from the University of Rhode Island. www.it-ebooks.info
ffi rs.indd 08/07/2014 Page vi ABOUT THE TECHNICAL EDITORS WEI ZHANG is a scientifi c programmer and has been working in the high-performance computing area for 15 years. He has developed or co-developed many scientifi c software packages for molecular simulation, computer-aided drug design, EM structure reconstruction, and seismic depth imaging. He is now focusing his effort on improving the performance of seismic data processing using new technologies such as CUDA. CHAO ZHAO joined Chevron in 2008 and currently serves as Geophysical Application Software Development Specialist. In this role, Chao is responsible for designing and developing software products for geoscientists. Prior to joining Chevron, Chao was a software developer for Knowledge Systems Inc. and Seismic Micro Technology Inc. With more than 13 years of software developing experience in the exploration and production industry, Chao has gained rich knowledge in the fi elds of geology and geophysics. Having a broad education in science, Chao likes to see CUDA programming used widely in scientifi c research and enjoys contributing to it as much as he can. He holds a Bachelor of Science degree in chemistry from Peking University and a Master of Science in computer science from the University of Rhode Island. www.it-ebooks.info