JS010862817B2 d2United States Patent ao)Patent No.:US 10,862,817 B2 Xue et al. (45)Date of Patent: Dec.8,2020 (54)CONGESTION CONTROL METHOD, (56) References Cited APPARATUS,AND SYSTEM U.S.PATENT DOCUMENTS (71)Applicants:Huawei Technologies Co.,Ltd. Shenzhen (CN):University of Science 9,391,911B1* 7/2016 Anderson …H04L1/1628 2010/0246400A1*9/20100 nishi H04L47/28 and Technology of China,Hefei (CN) 370/235 (72)Inventors:Kaiping Xue.Hefei (CN):Wenjia Wei. (Continued) Hefei (CN);Qing Xu,Shanghai (CN) FOREIGN PATENT DOCUMENTS (73)Assignees:Huawei Technologies Co.,Ltd., Shenzhen (CN);University of Science 贺 102318312A 1/2012 103581035A 22014 and Technology of China,Hefei(CN) (Continued) (*Notice: Subject to any disclaimer,the term of this OTHER PUBLICATIONS patent is extended or adjusted under 35 U.S.C.154(b)by 2 days. Ferlin et al.,"Revisiting Congestion Control for MultipathTCP with Shared Bottleneck Detection,"IEEE Infocom 2016-The 35th (21)Appl.No.:16/357,564 Annual IEEE International Conference on Computer Communica- tions,XP032930233,pp.1-9,Institute of Electrical and Electronics (22)Filed: Mar.19,2019 Engineers,New York,New York (Apr.2016). (Continued) (65) Prior Publication Data US2019/0215276A1Jul.11.2019 Primary Examiner-Phong La (74)Attorney,Agent,or Firm-Leydig,Voit Mayer, Related U.S.Application Data Ltd. (63)Continuation of application No. (57) ABSTRACT PCT/CN2016/099466,filed on Sep.20,2016. Embodiments of the present disclosure provide a congestion control method,an apparatus,and a system.The method (51)Int.Cl. includes receiving,by a first device,an acknowledgement H04L.12891 (2013.01) packet that is from a second device and that includes an ECE H04L12801 (2013.01) identifier.Based on an indication of the ECE identifier the (Continued) first device starts monitoring a plurality of sub streams on a (52)U.S.C. plurality of transmission paths between the first device and CPC… H04L47/41(2013.01);H04L45/24 the second device,and grouping,into one set,sub streams (2013.01):Ⅱ04L47/11(2013.01):I04L that meet a congestion control condition.Additionally,intra- 47/115(2013.01)月 set congestion control is performed on the sub streams in the (Continued) set.Separately congestion control is performed on a sub stream that is not in the set.The congestion control method (58)Field of Classification Search is applied to a scenario in which data transmission is CP℃，.H04L47/41;H04L47/193;H04L47/1; performed between the first device and the second device by H04L47/2482:H04L47/115:H04L using the MPTCP. 45/24:H04L43/0829:H04L43/0852 (Continued) 6 Claims,9 Drawing Sheets Routing device 12 Transmit Receive end 10 end 11
US010862817B2 ( 12 ) United States Patent ( 10 ) Patent No .: US 10,862,817 B2 Xue et al . ( 45 ) Date of Patent : Dec. 8 , 2020 ( 54 ) CONGESTION CONTROL METHOD , ( 56 ) References Cited APPARATUS , AND SYSTEM U.S. PATENT DOCUMENTS ( 71 ) Applicants : Huawei Technologies Co. , Ltd. , Shenzhen ( CN ) ; University of Science and Technology of China , Hefei ( CN ) 9,391,911 B1 * 2010/0246400 A1 * 7/2016 Anderson 9/2010 Onishi HO4L 1/1628 HO4L 47/28 370/235 ( Continued ) ( 72 ) Inventors : Kaiping Xue , Hefei ( CN ) ; Wenjia Wei , Hefei ( CN ) ; Qing Xu , Shanghai ( CN ) FOREIGN PATENT DOCUMENTS ( 73 ) Assignees : Huawei Technologies Co. , Ltd. , Shenzhen ( CN ) ; University of Science and Technology of China , Hefei ( CN ) CN CN 102318312 A 1/2012 103581035 A 2/2014 ( Continued ) ( * ) Notice : OTHER PUBLICATIONS Subject to any disclaimer , the term of this patent is extended or adjusted under 35 U.S.C. 154 ( b ) by 2 days . ( 21 ) Appl . No .: 16 / 357,564 Ferlin et al . , “ Revisiting Congestion Control for MultipathTCP with Shared Bottleneck Detection , ” IEEE Infocom 2016 — The 35th Annual IEEE International Conference on Computer Communica tions , XP032930233 , pp . 1-9 , Institute of Electrical and Electronics Engineers , New York , New York ( Apr. 2016 ) . ( Continued ) ( 22 ) Filed : Mar. 19 , 2019 ( 65 ) Prior Publication Data US 2019/0215276 A1 Jul . 11 , 2019 Primary Examiner - Phong La ( 74 ) Attorney , Agent , or Firm - Leydig , Voit & Mayer , Ltd. Related U.S. Application Data ( 63 ) Continuation of application PCT / CN2016 / 099466 , filed on Sep. 20 , 2016 . No. ( 51 ) Int . Ci . H04L 12/891 ( 2013.01 ) H04L 12/801 ( 2013.01 ) ( Continued ) ( 52 ) U.S. Ci . CPC H04L 47/41 ( 2013.01 ) ; H04L 45/24 ( 2013.01 ) ; H04L 47/11 ( 2013.01 ) ; H04L 47/115 ( 2013.01 ) ; ( Continued ) ( 58 ) Field of Classification Search CPC HO4L 47/41 ; H04L 47/193 ; H04L 47/11 ; H04L 47/2482 ; H04L 47/115 ; HO4L 45/24 ; H04L 43/0829 ; H04L 43/0852 ( Continued ) ( 57 ) ABSTRACT Embodiments of the present disclosure provide a congestion control method , an apparatus , and a system . The method includes receiving , by a first device , an acknowledgement packet that is from a second device and that includes an ECE identifier . Based on an indication of the ECE identifier the first device starts monitoring a plurality of sub streams on a plurality of transmission paths between the first device and the second device , and grouping , into one set , sub streams that meet a congestion control condition . Additionally , intra set congestion control is performed on the sub streams in the set . Separately congestion control is performed on a sub stream that is not in the set . The congestion control method is applied to a scenario in which data transmission is performed between the first device and the second device by using the MPTCP . 6 Claims , 9 Drawing Sheets Routing device 12 000 000 000 Transmit end 10 Receive end 11
US10,862,817B2 Page 2 (51)Int.C. FOREIGN PATENT DOCUMENTS H04L12/707 (2013.01) H04L12807 (2013.01) 103973587A 8/2014 H04L12/851 (2013.01) 104796350A 72015 wo 2015174901A1 11/2015 H04L12/26 (2006.01) (52)U.S.C1. OTHER PUBLICATIONS CPC....... H04L47193(2013.01):H04L472483 2013.01):H04L47/27(2013.01:H04L Braden et al.."Recommendations on Queue Management and 43/0829(2013.01H04L43/0852(2013.01) Congestion Avoidance in the Internet,"RFC 2309,The Internet (58)Field of Classification Search Society,XP055604783,Internet Engineering Task Force.Reston, USPC 370/236 Virginia (Apr.1998). See application file for complete search history. Floyd et al.."Random Early Detection Gateways for Congestion Avoidance,"IEEE/ACM Transactions on Networking vol.I.No.4. pp.1-22,Institute of Electrical and Electronics Engineers,New (56) References Cited York,New York (Aug.1993). Hassayoun et al.,"Dynamic Window Coupling for Multipath Con- U.S.PATENT DOCUMENTS gestion Control,"Proceedings of the 19th IEEE International Con- ference on Network Protocols (ICNP).pp.1-12.Institute of Elec- 2011/0292801A112/2011 Turnbull et al. trical and Electronics Engineers.New York,New York (2011). 2012/0054583A1◆3/2012Pak. H03M13/6306 Xue et al.,"Migrating Unfairness Among Subflows in MPTCP With 714/776 Network Coding for Wired-Wireless Networks",IEEE Transactions 2013/0182567A1◆ 7/2013 Scheffenegger .......H04L 47/12 370/23 on Vehicular Technology,vol.66,No.1,pp.798-809,Institute of Electrical and Electronics Engineers.New York.New York (Jan. 2014/0164641A1.6/2014Ye H04L47/11 2017)- 709/235 2015/0043345A12/2015 Testicioglu Ramakrishnan et al.,"The Addition of Explicit Congestion Notifi- .H04L47/24 370/232 cation (ECN)to IP,"Request for Comments:3168,pp.1-63 (Sep. 2015/0085665A13/2015 Kompella H04L47/20 2001). Xu et al.,"Delay-based Congestion Control for MPTCP:draft-xu- 370/236 2015/0138978A1◆ 5/2015 Ishikawa......... H04L45/74 mptcp-congestion-control-04.txt,"Multipath TCP.Internet Draft. Intended status:Standard Track (Jul.3,2016). 370)39 2016/0014030A1◆ 1/2016 Thyni...H04L47/115 Alizadeh et al.,"Data Center TCP(DCTCP),"SIGCOMM ACM. 370/237 New Delhi.India (Aug.30-Sep.3.2010). 2016/0088645A1◆ 3/2016 Burhan H04W7202 Raiciu et al."Coupled Congestion Control for Multipath Transport 370/349 Protocols,"Request for Comments:6356.ISSN:2070-1721.Inter- 2017/0180261A1◆6/2017Ma H04L.5/0055 net Engineering Task Force,Reston,Virginia (Oct.2011). 2017/0366374A1◆1220170sug4H04L47/14 20180220325A1◆8/2018Lcc H04W7210 cited by examiner
US 10,862,817 B2 Page 2 FOREIGN PATENT DOCUMENTS CN CN WO 103973587 A 104796350 A 2015174901 A1 8/2014 7/2015 11/2015 ( 51 ) Int . Ci . H04L 12/707 ( 2013.01 ) H04L 12/807 ( 2013.01 ) H04L 12/851 ( 2013.01 ) H04L 12/26 ( 2006.01 ) ( 52 ) U.S. CI . CPC H04L 47/193 ( 2013.01 ) ; H04L 47/2483 ( 2013.01 ) ; H04L 47/27 ( 2013.01 ) ; HO4L 43/0829 ( 2013.01 ) ; H04L 43/0852 ( 2013.01 ) ( 58 ) Field of Classification Search USPC 370/236 See application file for complete search history . OTHER PUBLICATIONS ( 56 ) References Cited U.S. PATENT DOCUMENTS 2011/0292801 Al 2012/0054583 A1 * 12/2011 Turnbull et al . 3/2012 Park 2013/0182567 A1 * 7/2013 Scheffenegger Braden et al . , “ Recommendations on Queue Management and Congestion Avoidance in the Internet , ” RFC , The Internet Society , XP055604783 , Internet Engineering Task Force , Reston , Virginia ( Apr. 1998 ) . Floyd et al , “ Random Early Detection Gateways for Congestion Avoidance , ” IEEE / ACM Transactions on Networking vol . 1 , No. 4 , pp . 1-22 , Institute of Electrical and Electronics Engineers , New York , New York ( Aug. 1993 ) . Hassayoun et al . , “ Dynamic Window Coupling for Multipath Con gestion Control , ” Proceedings of the 19th IEEE International Con ference on Network Protocols ( ICNP ) , pp . 1-12 , Institute of Elec trical and Electronics Engineers , New York , New York ( 2011 ) . Xueetal , “ Migrating UnfairnessAmong Subflowsin MPTCPWith Network Coding for Wired - Wireless Networks ” , IEEE Transactions on Vehicular Technology , vol . 66 , No. 1 , pp . 798-809 , Institute of Electrical and Electronics Engineers , New York , New York ( Jan. 2017 ) . Ramakrishnan et al . , “ The Addition of Explicit Congestion Notific cation ( ECN ) to IP , ” Request for Comments : 3168 , pp . 1-63 ( Sep. 2001 ) . Xuet al . , « Delay - based Congestion Control for MPTCP ; draft - xu . mptcp - congestion - control - 04.txt ” Multipath TCP , Internet Draft , Intended status : Standard Track ( Jul . 3 , 2016 ) . Alizadeh et al . , “ Data Center TCP ( DCTCP ) , ” SIGCOMM ACM , New Delhi , India ( Aug. 30 - Sep . 3 , 2010 ) . Raiciu et al . , “ Coupled Congestion Control for Multipath Transport Protocols , ” Request for Comments : 6356 , ISSN : 2070-1721 , Inter net Engineering Task Force , Reston , Virginia ( Oct. 2011 ) . 2014/0164641 A1 * 6/2014 Ye 2015/0043345 A1 * 2/2015 Testicioglu 2015/0085665 A1 * HO3M 13/6306 714/776 HO4L 47/12 370/231 HO4L 47/11 709/235 H04L 47/24 370/232 HO4L 47/20 370/236 H04L 45/74 370/235 HO4L 47/115 370/237 HO4W 72/02 370/349 HO4L 5/0055 H04L 47/14 HO4W 72/10 3/2015 Kompella 2015/0138978 A1 * 5/2015 Ishikawa 2016/0014030 A1 * 1/2016 Thyni 2016/0088645 A1 * 3/2016 Burhan 2017/0180261 A1 * 2017/0366374 A1 * 2018/0220325 A1 * 6/2017 Ma 12/2017 Osuga 8/2018 Lee * cited by examiner
U.S.Patent Dec.8,2020 Sheet 1 of 9 US10,862,817B2 Routing device 12 Transmit Receive end 10 end 11 FIG.1 Radio frequency Mobile phone circuit 110 Audio -il Speaker 151 Memory circuit 150 Microphone 152 120 Processor 160 Power supply 170 Input unit 130 Display unit 140 Touch panel 131 Display panel Another input 141 device 132 FIG 2
U.S. Patent Dec. 8 , 2 2020 Sheet 1 of 9 US 10,862,817 B2 Routing device 12 Transmit end 10 Receive end 11 FIG . 1 Radio frequency Mobile phone circuit 110 Speaker 151 Audio Memory circuit 150 120 + Microphone 152 Processor 160 Power supply 170 Input unit 130 Display unit 140 Touch panel 131 Another input device 132 Display panel 141 FIG . 2
U.S.Patent Dec.8,2020 Sheet 2 of 9 US10,862,817B2 Router Memory Processor Interface 220 210 230 FIG.3
U.S. Patent Dec. 8 , 2 2020 Sheet 2 of 9 US 10,862,817 B2 Router Memory 220 Processor 210 Interface 230 FIG . 3
U.S.Patent Dec.8,2020 Sheet 3 of 9 US10,862,817B2 First device Routing device Second device S101.A first device sends a first data packet to a routing device S102.The routing device receives the first data packet S103.The routing device determines that an average queue length of a data packet queue is greater than or equal to a first preset threshold and is less than or equal to a second preset threshold 工 S104.The routing device makes a congestion mark for the first data packet,and adds the first data packet to the data packet queue S105.The routing device sends the first data packet to a second device S106.The second device receives the first data packet S107.The second device makes an ECE mark for a first acknowledgement packet S108.The second device sends the first acknowledgement packet to the first device S109.The first device receives the first acknowledgement packet S110.The first device determines a first sub stream set S111.The first device performs congestion control on sub streams in the first sub stream set FIG.4
U.S. Patent Dec. 8 , 2 2020 Sheet 3 of 9 US 10,862,817 B2 First device Routing device Second device S101 . A first device sends a first data packet to a routing device S102 . The routing device receives the first data packet S103 . The routing device determines that an average queue length of a data packet queue is greater than or equal to a first preset threshold and is less than or equal to a second preset threshold S104 . The routing device makes a congestion mark for the first data packet , and adds the first data packet to the data packet queue $ 105 . The routing device sends the first data packet to a second device S106 . The second device receives the first data packet S107 . The second device makes an ECE mark for a first acknowledgement packet S108 . The second device sends the first acknowledgement packet to the first device S109 . The first device receives the first acknowledgement packet S110 . The first device determines a first sub stream set S111 . The first device performs congestion control on sub streams in the first sub stream set FIG . 4