Improving Coverage of Internet Outage Detection in Sparse Blocks
- 55 Downloads
There is a growing interest in carefully observing the reliability of the Internet’s edge. Outage information can inform our understanding of Internet reliability and planning, and it can help guide operations. Active outage detection methods provide results for more than 3M blocks, and passive methods more than 2M, but both are challenged by sparse blocks where few addresses respond or send traffic. We propose a new Full Block Scanning (FBS) algorithm to improve coverage for active scanning by providing reliable results for sparse blocks by gathering more information before making a decision. FBS identifies sparse blocks and takes additional time before making decisions about their outages, thereby addressing previous concerns about false outages while preserving strict limits on probe rates. We show that FBS can improve coverage by correcting 1.2M blocks that would otherwise be too sparse to correctly report, and potentially adding 1.7M additional blocks. FBS can be applied retroactively to existing datasets to improve prior coverage and accuracy.
We thank Yuri Pradkin for his input on the algorithms and paper.
We thank Philipp Richter and Arthur Berger for discussions about their work, and Philipp for re-running his comparison with CDN data.
The work is supported in part by the National Science Foundation, CISE Directorate, award CNS-1806785; by the Department of Homeland Security (DHS) Science and Technology Directorate, Cyber Security Division (DHS S&T/CSD) via contract number 70RSAT18CB0000014; and by Air Force Research Laboratory under agreement number FA8750-18-2-0280. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon.
- 1.IODA: Internet outage detection & analysis. https://ioda.caida.org
- 2.Baltra, G., Heidemann, J.: Improving the optics of active outage detection (extended). Technical report ISI-TR-733, May 2019. https://www.isi.edu/%7ejohnh/PAPERS/Baltra19a.html
- 4.Dainotti, A., et al.: Analysis of country-wide Internet outages caused by censorship. In: Proceedings of the ACM Internet Measurement Conference, Berlin, Germany, pp. 1–18. ACM, November 2011. https://doi.org/10.1145/2068816.2068818
- 5.Madory, D.: Iraq downs internet to combat cheating...again! (2017). https://dyn.com/blog/iraq-downs-internet-to-combat-cheating-again/. Accessed 01 Aug 2019
- 6.Guillot, A., et al.: Chocolatine: outage detection for internet background radiation. In: Proceedings of the IFIP International Workshop on Traffic Monitoring and Analysis. IFIP, Paris, France, June 2019. https://clarinet.u-strasbg.fr/~pelsser/publications/Guillot-chocolatine-TMA2019.pdf
- 7.Heidemann, J., Pradkin, Y., Govindan, R., Papadopoulos, C., Bartlett, G., Bannister, J.: Census and survey of the visible Internet. In: Proceedings of the ACM Internet Measurement Conference, Vouliagmeni, Greece, pp. 169–182. ACM, October 2008. https://doi.org/10.1145/1452520.1452542
- 8.Internet Addresses Survey dataset, PREDICT ID: USC-LANDER/internet-address-survey-reprobing-it75w-20170427Google Scholar
- 9.MaxMind: GeoIP Geolocation Products (2017). http://www.maxmind.com/en/city
- 10.Padmanabhan, R., Dhamdhere, A., Aben, E., Claffy, K.C., Spring, N.: Reasons dynamic addresses change. In: Proceedings of the ACM Internet Measurement Conference, Santa Monica, CA, USA. ACM, November 2016. https://doi.org/10.1145/2987443.2987461
- 11.Padmanabhan, R., Schulman, A., Levin, D., Spring, N.: Residential links under the weather. In: Proceedings of the ACM Special Interest Group on Data Communication, pp. 145–158. ACM (2019)Google Scholar
- 12.Quan, L., Heidemann, J., Pradkin, Y.: Trinocular: understanding Internet reliability through adaptive probing. In: Proceedings of the ACM SIGCOMM Conference, Hong Kong, China, pp. 255–266. ACM, August 2013. https://doi.org/10.1145/2486001.2486017
- 13.Quan, L., Heidemann, J., Pradkin, Y.: When the Internet sleeps: correlating diurnal networks with external factors. In: Proceedings of the ACM Internet Measurement Conference, Vancouver, BC, Canada, pp. 87–100. ACM, November 2014. https://doi.org/10.1145/2663716.2663721
- 14.Richter, P., Padmanabhan, R., Spring, N., Berger, A., Clark, D.: Advancing the art of Internet edge outage detection. In: Proceedings of the ACM Internet Measurement Conference, Boston, Massachusetts, USA. ACM, October 2018. https://doi.org/10.1145/3278532.3278563
- 15.Schulman, A., Spring, N.: Pingin’ in the rain. In: Proceedings of the ACM Internet Measurement Conference, pp. 19–25. Berlin, Germany. ACM, November 2011. https://doi.org/10.1145/2068816.2068819
- 16.Shah, A., Fontugne, R., Aben, E., Pelsser, C., Bush, R.: Disco: fast, good, and cheap outage detection. In: Proceedings of the IEEE International Conference on Traffic Monitoring and Analysis, Dublin, Ireland, pp. 1–9. IEEE, June 2017. https://doi.org/10.23919/TMA.2017.8002902
- 17.USC/ISI ANT Project. https://ant.isi.edu/datasets/outage/index.html