P2P tutorial (ESEC 2001) Napster:The Protocol [Drschollo1] The protocol was never published openly and is rather complex and inconsistent OpenNap have reverse engineered the protocol and published their findings TCP is used for C/S communication Messages to/from the server have the following format: length type data Byte offset 0 12 34 -length specifies the length of the data portion type defines the message type data:the transferred data plain ASCII,in many cases enclosed in double quotes (e.g., filenames such as"song.mp3"or client ids such as"nap vo.8" 82001 Karl Abe ESEC/FSE 2001 31 Sample Messages -1 Type C/S Description Format 0 Error message <message> Login <nick><pwd><port><client info><link type> 3 Login ack <user's email> 5S Auto-upgrade <new version><http-hostname:filename> 6 New user login <nick><pwd><port><client info><speed> <email address> 100 Client notification <filename>"<md5><size><bitrate> of shared file <frequency><time> 200 C Search request [FILENAME CONTAINS "artist name"] MAX RESULTS <max>[FILENAME CONTAINS <song][LINESPEED <comp><link type>] [BITRATE <comp>"bit rate"][FREQ <comp> "freg][WMA-FILE][LOCAL ONLY] 201 N Search response "<filename>"<md5><size><bit rate> <frequency><length><nick><ip address> 202 End of search (empty) response 2001 Karl Abe Manfred Hauswirth ESEC/FSE 2001 32 (c)2001 Karl Aberer,Manfred Hauswirth 16
P2P tutorial (ESEC 2001) (c) 2001 Karl Aberer, Manfred Hauswirth 16 © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 31 Napster: The Protocol [Drscholl01] • The protocol was never published openly and is rather complex and inconsistent • OpenNap have reverse engineered the protocol and published their findings • TCP is used for C/S communication • Messages to/from the server have the following format: – length specifies the length of the data portion – type defines the message type – data: the transferred data • plain ASCII, in many cases enclosed in double quotes (e.g., filenames such as “song.mp3” or client ids such as “nap v0.8” length type data Byte offset 0 1 2 3 4 ..... n © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 32 Sample Messages - 1 Type C/S Description Format 0 S Error message <message> 2 C Login <nick><pwd><port><client info><link type> 3 S Login ack <user’s email> 5 S Auto-upgrade <new version><http-hostname:filename> 6 C New user login <nick><pwd><port><client info><speed> <email address> 100 C Client notification of shared file “<filename>”<md5><size><bitrate> <frequency><time> 200 C Search request [FILENAME CONTAINS “artist name”] MAX_RESULTS <max> [FILENAME CONTAINS <song] [LINESPEED <comp> <link type>] [BITRATE <comp> “bit rate”] [FREQ <comp> “freq”] [WMA-FILE] [LOCAL_ONLY] 201 S Search response “<filename>”<md5><size><bit rate> <frequency><length><nick><ip address> 202 S End of search response (empty)
P2P tutorial (ESEC 2001) Sample Messages-2 Type C/S Description Format 203 Download request <nick>"<filename>" 204 S Download ack <nick><ip><port>"<filename>"<md5> <linespeed> 206 Peer to download not <nick>"<filename>" available 209s Hotlist user signed on<user><speed> 211 C Browse a user's files <nick> 212 Browse response <nick>"<filename>"<md5><size> <bit rate><frequency><time> 213 SEnd of browse list <nick>[<ip address>] 500 Push file to me <nick>"<filename>" (firewall problem) 501 Push ack(to other <nick><ip address><port>"<filename> client) <md5><speed≥ 2001 Karl Aberer,Manfred Hauswirth ESEC/FSE 2001 33 Client-Client Communication -1 Normal download (A downloads from B): -A connects to B's IP address/port as specified in the 204 message returned by the server (response to 203) B sends the ASCII character "1" A sends the string "GET" -A sends <mynick>"<filename>"<offset> -B returns the file size(not terminated by any special character!)or an error message such as "FILE NOT SHARED" A notifies the server that the download is ongoing via a 218 message;likewise B informs the server with a 220 message Upon successful completion A notifies the server with a 219 message;likewise B informs the server with a 221 message 2001 Karl Aberer,Manfred Hauswirth ESEC/FSE 2001 (c)2001 Karl Aberer,Manfred Hauswirth 17
P2P tutorial (ESEC 2001) (c) 2001 Karl Aberer, Manfred Hauswirth 17 © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 33 Sample Messages - 2 Type C/S Description Format 203 C Download request <nick> “<filename>” 204 S Download ack <nick><ip><port> “<filename>” <md5> <linespeed> 206 S Peer to download not available <nick> “<filename>” 209 S Hotlist user signed on <user><speed> 211 C Browse a user’s files <nick> 212 S Browse response <nick> “<filename>”<md5><size> <bit rate><frequency><time> 213 S End of browse list <nick>[<ip address>] 500 C Push file to me (firewall problem) <nick> “<filename>” 501 S Push ack (to other client) <nick><ip address><port> “<filename>” <md5><speed> © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 34 Client-Client Communication - 1 • Normal download (A downloads from B): – A connects to B’s IP address/port as specified in the 204 message returned by the server (response to 203) – B sends the ASCII character “1” – A sends the string “GET” – A sends <mynick> “<filename>” <offset> – B returns the file size (not terminated by any special character!) or an error message such as “FILE NOT SHARED” – A notifies the server that the download is ongoing via a 218 message; likewise B informs the server with a 220 message – Upon successful completion A notifies the server with a 219 message; likewise B informs the server with a 221 message
P2P tutorial (ESEC 2001) Client-Client Communication -2 Firewalled download (A wants to download from B who is behind a firewall): -A sends a 500 message to the server which in turn sends a 501 message (holding A's IP address and data port)to B -B connects A according to the 501 message -A sends the ASCII character"1" B sends the string "SEND" -B sends <mynick>"<filename>"<size> -A returns the byte offset at which the transfer should start (plain ASCII characters)or an error message such as INVALID REQUEST" A notifies the server that the download is ongoing via a 218 message;likewise B informs the server with a 220 message Upon successful completion A notifies the server with a 219 message;likewise B informs the server with a 221 message B 2001 Karl Abe Manfred Hauswirth ESEC/FSE 2001 35 Napster:Further Services Additionally to its search/transfer features the Napster client offers: -A chat program that allows users to chat with each others in forums based on music genre,etc. -A audio player to play MP3 files from inside Napster A tracking program to support users in keeping track of their favorite MP3s for later browsing -Instant messaging service Most of the message types in the protocol deal with hotlist,chat room,and instant messages 2001 Karl Aberer,Manfred Hauswirth ESEC/FSE 2001 36 (c)2001 Karl Aberer,Manfred Hauswirth 18
P2P tutorial (ESEC 2001) (c) 2001 Karl Aberer, Manfred Hauswirth 18 © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 35 Client-Client Communication - 2 • Firewalled download (A wants to download from B who is behind a firewall): – A sends a 500 message to the server which in turn sends a 501 message (holding A’s IP address and data port) to B – B connects A according to the 501 message – A sends the ASCII character “1” – B sends the string “SEND” – B sends <mynick> “<filename>” <size> – A returns the byte offset at which the transfer should start (plain ASCII characters) or an error message such as “INVALID REQUEST” – A notifies the server that the download is ongoing via a 218 message; likewise B informs the server with a 220 message – Upon successful completion A notifies the server with a 219 message; likewise B informs the server with a 221 message © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 36 Napster: Further Services • Additionally to its search/transfer features the Napster client offers: – A chat program that allows users to chat with each others in forums based on music genre, etc. – A audio player to play MP3 files from inside Napster – A tracking program to support users in keeping track of their favorite MP3s for later browsing – Instant messaging service • Most of the message types in the protocol deal with hotlist, chat room, and instant messages
P2P tutorial (ESEC 2001) 和 Napster:Summary ·(Virtually)）centralized system -single point of failure limited fault tolerance limited scalability (server farms with load balancing) Protocol is complicated and inconsistent Querying is fast and upper bound for the duration can be given ·"Topology is known" Reputation of peers is not addressed Many add-on services users like 82001 Karl Aberer,Manfred Hauswirth ESEC/FSE 2001 Gnutella:A brief History Developed in a 14 days "quick hack"by Nullsoft (winamp) ● Originally intended for exchange of recipes 。Timeline: Published under GNU General Public License on the Nullsoft web server Taken off after a couple of hours by AOL(owner of Nullsoft) This was enough to "infect"the Internet -Gnutella protocol was reverse engineered from downloaded versions of the original Gnutella software Third-party clients were published and Gnutella started to spread 2001 Karl Aberer,Manfred Hauswirth ESEC/FSE 2001 38 (c)2001 Karl Aberer,Manfred Hauswirth 19
P2P tutorial (ESEC 2001) (c) 2001 Karl Aberer, Manfred Hauswirth 19 © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 37 Napster: Summary • (Virtually) centralized system – single point of failure ⇒ limited fault tolerance – limited scalability (server farms with load balancing) • Protocol is complicated and inconsistent • Querying is fast and upper bound for the duration can be given • “Topology is known” • Reputation of peers is not addressed • Many add-on services users like © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 38 Gnutella: A brief History • Developed in a 14 days “quick hack” by Nullsoft (winamp) • Originally intended for exchange of recipes • Timeline: – Published under GNU General Public License on the Nullsoft web server – Taken off after a couple of hours by AOL (owner of Nullsoft) – This was enough to “infect” the Internet – Gnutella protocol was reverse engineered from downloaded versions of the original Gnutella software – Third-party clients were published and Gnutella started to spread
P2P tutorial (ESEC 2001) Gnutella:System Architecture 。No central server cannot be sued (Napster) Constrained broadcast -Every peer sends packets it receives to all of its peers (typically 4) Life-time of packets limited by time-to-live(typically set to 7) Packets have unique ids to detect loops Hooking up to the Gnutella systems requires that a new peer knows at least one Gnutella host -gnutellahosts.com:6346 -Outside the Gnutella protocol specification 82001 Karl Aberer Manfred Haus ESEC/FSE 2001 39 Gnutella:Protocol Message Types Type Description Contained Information Ping Announce availability and None probe for other servents Pong Response to a ping IP address and port#of responding servent;number and total kb of files shared Query Search request Minimum network bandwidth of responding servent;search criteria QueryHit Returned by servents IP address,port#and network that have the requested bandwidth of responding servent; file number of results and result set Push File download requests Servent identifier;index of for servents behind a requested file;IP address and firewall port to send file to 2001 Karl Aberer ed Hauswirth ESEC/FSE 2001 (c)2001 Karl Aberer,Manfred Hauswirth 20
P2P tutorial (ESEC 2001) (c) 2001 Karl Aberer, Manfred Hauswirth 20 © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 39 Gnutella: System Architecture • No central server – cannot be sued (Napster) • Constrained broadcast – Every peer sends packets it receives to all of its peers (typically 4) – Life-time of packets limited by time-to-live (typically set to 7) – Packets have unique ids to detect loops • Hooking up to the Gnutella systems requires that a new peer knows at least one Gnutella host – gnutellahosts.com:6346 – Outside the Gnutella protocol specification © 2001 Karl Aberer, Manfred Hauswirth ESEC/FSE 2001 40 Gnutella: Protocol Message Types Type Description Contained Information Ping Announce availability and probe for other servents None Pong Response to a ping IP address and port# of responding servent; number and total kb of files shared Query Search request Minimum network bandwidth of responding servent; search criteria QueryHit Returned by servents that have the requested file IP address, port# and network bandwidth of responding servent; number of results and result set Push File download requests for servents behind a firewall Servent identifier; index of requested file; IP address and port to send file to