Takeaways from an experimental evaluation of eXpress Data Path (XDP) and Data Plane Development Kit (DPDK) under a Cloud Computing environment
DOI:
https://doi.org/10.33448/rsd-v11i12.34200Keywords:
XDP; DPDK; Linux Kernel; Fast Packet Processing Frameworks; Cloud computing.Abstract
The popularization of the Linux kernel at data center servers became a significant factor to consider when developing or deploying network applications. New “packet processing frameworks” emerged to ensure fast packet processing inside these servers that rely on the Linux kernel, XDP and DPDK being the two main ones. Alongside this, Cloud Computing environments dominated the data center scenario, bringing virtualization to the center of these servers. This research compares DPDK and XDP packet processors when subject to a cloud computing environment with I/O, CPU, and network load. We first describe an architecture that implements frameworks and emulates a cloud environment. We measure throughput and packet loss while varying the number of competing VMs, CPU usage by the frameworks, and packet size and present our results in the form of takeaways. Furthermore, we conclude that the virtual environment can impact the performance of both frameworks depending on the received packet size and the type of workload the cloud environment performs. A significant case to highlight is when the cloud environment performs network load, and the servers receive small-sized packets, creating both throughput degradation and increasing packet loss. We conclude our experiments by conducting statistical non-parametric hypothesis tests to enhance our conclusions and have a closer look at the results.
References
Cilium. (2022). Bpf and xdp reference guide. https://docs.cilium.io/en/latest/bpf/
Incubator, F. (2022). Katran - a high performance layer 4 load balancer. https://github.com/facebookincubator/katran
Cilium. (2022). What is cilium? https://cilium.io/get-started/
Belay, A., Prekas, G., Klimovic, A., Grossman, S., Kozyrakis, C., & Bugnion, E. (2014). Ix: A protected dataplane operating system for high throughput and low latency. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Imple-mentation, OSDI’14, page 49–65, USA. USENIX Association.
Bruijn, W. D. & Dumazet, E. (2017). sendmsg copy avoidance with msg zerocopy. In NetDev, The Technical Conference on Linux Networking, Netdev 2.1, Montreal, Canada. Netdev.
Corder, G. W. & Foreman, D. I. (2011). Nonparametric statistics for non-statisticians.
Dantas, R., Sadok, D., Flinta, C., & Johnsson, A. (2015). Kvm virtualization impact on active round-trip time measurements. In 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM), pages 810–813.
Oliveira Filho, A. T., Freitas, E., do Carmo, P. R., Sadok, D. H., & Kelner, J. (2022). An experimental investigation of round-trip time and virtualization. Computer Communications, 184:73–85.
Emmerich, P., Gallenmüller, S., Raumer, D., Wohlfart, F., & Carle, G. (2015). Moongen: A scriptable high-speed packet generator. In Proceedings of the 2015 Internet Measurement Conference, IMC ’15, page 275–287, New York, NY, USA. Association for Computing Machinery.
Fabre, A. (2018). L4drop: Xdp ddos mitigations. https://blog.cloudflare.com/l4drop-xdp-ebpf-based-ddos-mitigations/
Foundation, L. (2018). Data plane development kit. https://www.dpdk.org/
Freitas, E. (2021). Experimental evaluation on packet processing frameworks under virtual environments. Master’s thesis, Universidade Federal de Pernambuco, Recife.
Gallenmüller, S., Emmerich, P., Wohlfart, F., Raumer, D., and Carle, G. (2015). Comparison of frameworks for high-performance packet io. In Proceedings of the Eleventh ACM/IEEE Symposium on Architectures for Networking and Communications Systems, ANCS ’15, page 29–38, USA. IEEE Computer Society.
Group, D. P. D. K. (2017a). Dpdk docs: Poll mode driver. http://doc.dpdk.org/guides/prog_guide/poll_mode_drv.html
Group, D. P. D. K. (2017b). Dpdk docs: Programmer’s guide, mempool library. https://doc.dpdk.org/guides/prog_guide/mempool_lib.html
Group, D. P. D. K. (2017c). Dpdk docs: Programmer’s guide, overview. http://doc.dpdk.org/guides/prog_guide/overview.html
Group, D. P. D. K. (2017d). Dpdk docs: Programmer’s guide, ring library. https://doc.dpdk.org/guides/prog_guide/ring_lib.html
Han, S., Jang, K., Park, K., & Moon, S. (2010). Packetshader: A gpu-accelerated software router. In Proceedings of the ACM SIGCOMM 2010 Conference, SIGCOMM ’10, page 195–206, New York, NY, USA. Association for Computing Machinery.
Hohlfeld, O., Krude, J., Reelfs, J. H., R¨uth, J., and Wehrle, K. (2019). Demystifying the performance of xdp bpf. In 2019 IEEE Conference on Network Softwarization (NetSoft), pages 208–212.
-Jørgensen, T., Brouer, J. D., Borkmann, D., Fastabend, J., Herbert, T., Ahern, D., & Miller, D. (2018). The express data path: Fast programmable packet processing in the operating system kernel. In Proceedings of the 14th International Conference on Emerging Networking EXperiments and Technologies, CoNEXT ’18, page 54–66, New York, NY, USA. Association for Com-puting Machinery.
IDG (2020). Cloud computing survey. Executive summary, IDG. https://resources.foundryco.com/download/cloud-computing-executive-summary
Jeong, E., Wood, S., Jamshed, M., Jeong, H., Ihm, S., Han, D., & Park, K. (2014). mtcp: a highly scalable user-level TCP stack for multicore systems. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), pages 489– 502, Seattle, WA. USENIX Association.
Kourtis, M.-A., Xilouris, G., Riccobene, V., Mc-Grath, M. J., Petralia, G., Koumaras, H., Gardikis, G., & Liberal, F. (2015). Enhancing vnf performance by exploiting sr-iov and dpdk packet processing acceleration. In 2015 IEEE Conference on Network Function Virtualization and Software Defined Network (NFV-SDN), pages 74–78.
Liu, J. (2010). Evaluating standard-based self-virtualizing devices: A performance study on 10 gbe nics with sr-iov support. In 2010 IEEE International Symposium on Parallel Distributed Processing (IPDPS), pages 1–12.
Liu, M., Cui, T., Schuh, H., Krishnamurthy, A., Peter, S., & Gupta, K. (2019). Offloading distributed applications onto smartnics using ipipe. In Proceedings of the ACM Special Interest Group on Data Communication, SIGCOMM ’19, page 318–333, New York, NY, USA. Association for Computing Machinery.
Miano, S., Doriguzzi-Corin, R., Risso, F., Siracusa, D., & Sommese, R. (2019). Introducing smartnics in server-based data plane processing: The ddos mitigation use case. IEEE Access, 7:107161–107170.
Monnet, Q. (2016). Dive into bpf: a list of reading material. https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/#what-is-bpf
Rizzo, L. (2012). netmap: A novel framework for fast packet i/o. In 2012 USENIX Annual Technical Conference (USENIX ATC 12), pages 101–112, Boston, MA. USENIX Association.
Rybczyńska, M. (2019). Bounded loops in bpf for the 5.3 kernel. https://lwn.net/Articles/794934/
Scholz, D., Raumer, D., Emmerich, P., Kurtz, A., Lesiak, K., & Carle, G. (2018). Performance implications of packet filtering with linux ebpf. In2018 30th International Teletraffic Congress (ITC 30), volume 01, pages 209–217.
Soares, L. & Stumm, M. (2010). Flexsc: Flexible system call scheduling with exception-less system calls. In Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation, OSDI’10, page 33–46, USA. USENIX Association.
Surveys, W. W. W. W. T. (2021). Usage of operating systems broken down by data center providers. https://w3techs.com/technologies/cross/operating_system/data_center
Tsuna (2010). How long does it take to make a context switch? https://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context.html
Van Tu, N., Yoo, J.-H., & Hong, J. W.-K. (2019). evnf - hybrid virtual network functions with linux express data path. In 2019 20th Asia-Pacific Network Operations and Management Sym-posium (APNOMS), pages 1–6.
Vieira, M. A. M., Castanho, M. S., Pacífico, R. D. G., Santos, E. R. S., Júnior, E. P. M. C., & Vieira, L. F. M. (2020). Fast packet processing with ebpf and xdp: Concepts, code, challenges, and applications. ACM Comput. Surv., 53(1).
Wu, W., Crawford, M., & Bowden, M. (2007). The performance analysis of linux networking – packet receiving. Computer Communications, 30(5):1044 – 1057. Advances in Computer Commu-nications Networks.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Eduardo Freitas; Assis T. de Oliveira Filho; Pedro R. X. do Carmo; Djamel F. H. Sadok; Judith Kelner
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
1) Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2) Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3) Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
