Dryad and dryadLINQ Presented by yin zhu Aoi22,2013 Slides taken from DryadlINQ project page http://research.microsoft.com/en-us/ projects/dryadling/default. aspx
Dryad and DryadLINQ Presented by Yin Zhu April 22, 2013 Slides taken from DryadLINQ project page: http://research.microsoft.com/en-us/ projects/dryadlinq/default.aspx
Distributed data-Parallel Programming using Dryad EuroSysO7 Andrew birrell. Mihai budiu Dennis Fetterly, Michael Isard, Yuan Yu Microsoft Research Silicon valley
Distributed Data-Parallel Programming using Dryad, EuroSys’07 Andrew Birrell, Mihai Budiu, Dennis Fetterly, Michael Isard, Yuan Yu Microsoft Research Silicon Valley
Dryad goals General-purpose execution environment for distributed, data-parallel applications Concentrates on throughput not latency Assumes private data center Automatic management of scheduling distribution fault tolerance etc
Dryad goals • General-purpose execution environment for distributed, data-parallel applications – Concentrates on throughput not latency – Assumes private data center • Automatic management of scheduling, distribution, fault tolerance, etc
Talk outline Computational model · Dryad architecture Some case studies DryadLINQ overview · Summary
Talk outline • Computational model • Dryad architecture • Some case studies • DryadLINQ overview • Summary
A typical data-intensive query var logentries= from line in logs where!line Starts With ( #" select new Log Entry(line); var user from access in logentries here access user Endswith(@ ulfar") Ulfar's most select access frequently visited var accesses web pages from access in user group access by access page into pages select new User Page Count("ulfar" pages. Key, pages. Count() var htmaccesses= from access in accesses where access page Endswith ".htm") orderby access. count descending select access
A typical data-intensive query var logentries = from line in logs where !line.StartsWith("#") select new LogEntry(line); var user = from access in logentries where access.user.EndsWith(@"\ulfar") select access; var accesses = from access in user group access by access.page into pages select new UserPageCount("ulfar", pages.Key, pages.Count()); var htmAccesses = from access in accesses where access.page.EndsWith(".htm") orderby access.count descending select access; Ulfar’s most frequently visited web pages