Overview

Together with the graduate and undergraduate students and postdoc fellow at the University of Victoria (UVic) and research collaborators around the world, we have been putting together a low-earth-orbit (LEO) satellite network (LSN) testbed, with access to SpaceX’s Starlink and Eutelsat’s OneWeb user terminals (UT, also known as “dish”) and in contact with Amazon’s Project Kuiper and Telesat (Canada)’s Lightspeed, four current leading LSN service providers covering both consumer and business customers.

We first explored the Starlink access network performance near Seattle and other point-of-presence (PoP) locations and revealed the Starlink backbone network around the world, with publications that attracted considerable academic and industry attention. We are among the first to identify and confirm the Starlink UT-satellite handover behaviors every 15 seconds globally synchronized off each minute, and proposed techniques to mitigate the handover packet loss and delay spikes and their impact in the transport and application layers of the network protocol stack.

The work focusing on low-latency live video streaming over LSNs received the Dynamic Adaptive Streaming over HTTP (DASH, a standard powering almost all the videos we see over the Internet nowadays) Industry Forum (DASH-IF) Prize at ACM Multimedia Systems 2024. We also open sourced our code and released our datasets to the research community, under the Low Earth Network of Satellites (LENS) initiative. The dataset contains the packet delay and loss measurement data in terabytes from more than a dozen dishes around the world and is updated monthly, and has been downloaded more than 400 times in less than a year, enabling trace-driven network simulation and emulation research worldwide.

We further made our research testbed accessible through RIPE Atlas (European Internet operators), LEOScope (an initiative led by the University of Surrey and Microsoft Research India) and Starlink Status (a user-contributed measurement platform), and our research development is disseminated to the broader public through Reddit Starlink Engineering, in addition to traditional means such as academic conferences, research journals and invited talks. We also hosted indigenous teenager’s summer camp (VJKF 2024 program) on bridging the digital divide and participated in the projects to bring connectivity to indigenous kids attending schools on daily extended commutes.

Some of our research tools and bug fixes have been adopted by Starlink in its mobile app and backbone network and used by its more than 3 million users in more than 100 countries and regions. Most recently, we are comparing the performance of Starlink and OneWeb at different locations around the world given their different technologies, LEO parameters and targeted customers, and correlating UT, communicating satellites and landing ground stations without UT revealing such information to improve the handover performance and behaviors.

A list of our work, papers, datasets and impact can be found below.

Publications

13. Trajectory-based Serving Satellite Identification with User Terminal’s Field-of-View, [PDF]
Ali Ahangarpour, Jinwei Zhao, Jianping Pan
Accepted by 2024 ACM 2nd Workshop on LEO Networking and Communication (LEO-NET’24)

Abstract

Low-Earth-Orbit (LEO) satellite networks, such as SpaceX’s Starlink, achieved global broadband Internet coverage with significantly lower latency and higher throughput than traditional satellite Internet service providers utilizing geostationary satellites. Despite the substantial advancements, the research community lacks detailed insights into the internal mechanisms of these networks. This paper presents the first systematic study of Starlink’s obstruction map and serving satellite identification. Our method achieves almost unambiguous satellite identification by incorporating satellite trajectories and proposing an accurate Field-of-View (FOV) estimation approach. We validate our methodology using multiple Starlink dishes with varying alignment parameters and latitudes across different continents. We utilize Two-Line Element data to identify the available satellites within the user terminal’s FOV and examine their characteristics, in comparison to those of the serving satellites. Our approach revealed a correlation between the satellite selection strategy and the user terminal to gateway latency. The findings contribute to the broader understanding of the architecture of LEO satellite networks and their impact on user experience.


12. StarQUIC: Tuning Congestion Control Algorithms for QUIC over LEO Satellite Networks, [PDF]
Victor Kamel, Jinwei Zhao, Daoping Li, Jianping Pan
Accepted by 2024 ACM 2nd Workshop on LEO Networking and Communication (LEO-NET’24)

Abstract

With the deployment of mega constellations of Low-Earth-Orbit (LEO) satellites, low latency and high throughput Internet coverage is extended globally. Latency-sensitive applications can benefit from the inherent lower transmission delay of LEO satellite networks compared to traditional Geostationary-Earth-Orbit (GEO) satellite networks. Starlink employs a globally time-synchronized controller to manage the association of satellite-to-ground communication links with an interval of 15 seconds, at fixed 12-27-42-57 seconds of every minute. Latency spikes and packet losses can occur during the handover period which can degrade the performance of transport layer protocols including TCP and QUIC, which rely on similar congestion control algorithms for fair data transmission. In this paper, we investigate the impact of the frequent Starlink handover events on QUIC performance. By leveraging the predictable handover patterns to avoid unnecessary congestion window reduction, we improved the performance of QUIC by up to 35% in terms of completion time in both network emulation and real-world experiments over Starlink networks. Our approach is independent of specific loss-sensitive congestion control algorithms and can be easily generalized.


11. An eBPF-Based Trace-Driven Emulation Method for Satellite Networks, [arXiv], [Code] (to be updated)
Weibiao Tian, Ye Li, Jinwei Zhao, Sheng Wu, Jianping Pan
(In submission)

Abstract

System-level performance evaluation over satellite networks often requires a simulated or emulated environment for reproducibility and low cost. However, the existing tools may not meet the needs for scenarios such as the low-earth orbit (LEO) satellite networks. To address the problem, this paper proposes and implements a trace-driven emulation method based on Linux’s eBPF technology. Building a Starlink traces collection system, we demonstrate that the method can effectively and efficiently emulate the connection conditions, and therefore provides a means for evaluating applications on local hosts.


10. Adaptive Multi-Link Data Allocation for LEO Satellite Networks
Jinkai Zheng, Tom H. H. Luan, Jinwei Zhao, Guanjie Li, Yao Zhang, Jianping Pan, Nan Cheng
Accepted by 2024 IEEE 43rd Global Communications Conference (GLOBECOM’24)

Abstract

The rapid development of Low Earth Orbit (LEO) satellite networks has provided ubiquitous Internet access to users around the world, especially in areas where there are no terrestrial networks. However, a dish can only communicate with one of the available satellites when uploading data in the current framework, resulting in low communication efficiency. As the number of satellites continues to increase, the current framework cannot make full use of communication resources. In this paper, we first conduct a comprehensive measurement of Starlink’s network performance and report some unique features. Then, we propose an adaptive multi-link data allocation framework for LEO satellite networks where a dish can communicate with multiple satellites at the same time to improve data transmission efficiency. With this framework, data can be split into chunks and uploaded simultaneously over multiple links. Our goal is to determine the data allocation strategies to jointly optimize the transmission latency and data processing costs. To this end, we propose a deep reinforcement learning-based algorithm integrated with the traffic prediction module to determine the optimal data allocation strategies in a dynamic network environment. Through extensive simulations, we demonstrate the effectiveness of our approach compared with baselines.


9. Mobility-Aware Congestion Control for Multipath QUIC in Integrated Terrestrial Satellite Networks
Wenjun Yang, Lin Cai, Shengjie Shu, Jianping Pan
IEEE Transactions on Mobile Computing, doi: 10.1109/TMC.2024.3397164

Abstract

The Integrated Terrestrial and LEO Satellite Network (ITSN) has a high bandwidth-delay product (BDP) and high-speed movement, which makes congestion control difficult. We develop a M obility- A ware CO ngestion control (MACO) algorithm for multipath QUIC (MPQUIC) in ITSN. MACO models the dynamic interactions between MPQUIC subflows and LEO networks, including handovers and outages triggered by satellite movement, and changes in network topology and link conditions. With the knowledge of network dynamics influenced by mobility, MACO can estimate changes in path BDP without solely relying on lengthy network probing. It employs a quick start (QS) and an effective congestion avoidance (CA) mechanism based on a multipath fluid model. The QS sets an appropriate initial cwnd to shorten the slow start duration. The CA applies a square root function to quickly increase the cwnd to the equilibrium and conservatively increase when approaching the BDP. We conduct a series of experiments to evaluate MACO using network simulator 3 (ns-3) based on collected data traces on Starlink. Simulation results demonstrate that MACO can achieve upto three times higher throughput and improve the convergence performance by 70.67% against benchmark algorithms.


8. LENS: A LEO Satellite Network Measurement Dataset, [Dataset][Poster]
Jinwei Zhao, Jianping Pan
2024 ACM 15th Multimedia Systems Conference Open-Source software & Datasets track (MMSys’24 ODS), doi: 10.1145/3625468.3652170

Abstract

Low-Earth-Orbit (LEO) satellite constellations are narrowing the performance gap between satellite networks and the terrestrial Internet. Low-latency satellite Internet offered by Starlink enables functionalities that are otherwise unachievable with the traditional geosynchronous equatorial orbit (GEO) satellite networks, including low-latency live video streaming, cloud gaming and real-time video conferencing. The absence of a comprehensive and long-term network measurement dataset with a global perspective poses significant challenges for researchers to evaluate the application performance over Starlink networks. In this paper, we introduce LENS, which is a LEO satellite network measurement dataset, collected from 13 Starlink dishes, associated with 7 Point-of-Presence (PoP) locations across 3 continents. The dataset currently consists of network latency traces from Starlink dishes with different hardware revisions, various service subscriptions and distinct sky obstruction ratios. We provide a high-level overview and analysis of the latency performance using the dataset and discuss various use cases. This dataset is useful for researchers who wish to understand the long-term network performance of Starlink and to evaluate and optimize the performance of multimedia applications over satellite networks.


7. Low Latency Live Video Streaming over a Low-Earth-Orbit Satellite Network with DASH, [Code][Slides]
Jinwei Zhao, Jianping Pan
2024 ACM 15th Multimedia Systems Conference (MMSys’24), doi: 10.1145/3625468.3647616
DASH-IF Excellence in DASH Award Third Place

Abstract

In light of Starlink’s recent rapid growth in constructing a global low-Earth-orbit satellite constellation and offering high-speed, low-latency Internet services, the implications of utilizing Starlink for low-latency live video streaming, particularly in the context of its fluctuating latency and regular satellite handovers events, remain insufficiently explored. In this paper, we conducted a thorough measurement study on the Starlink access network, examining its performance across different protocol layers and at multiple geographical installations, including locations where laser intersatellite links are utilized in practice. We performed a comprehensive latency target-based analysis of low-latency live video streaming with three state-of-the-art adaptive bitrate (ABR) algorithms in dash.js over Starlink. We presented a novel ABR algorithm designed for low-latency live video streaming over Starlink networks which leverages satellite handover patterns observed from measurements to dynamically adjust video bitrate and playback speed. The performance evaluation of the proposed algorithm was conducted using both a purpose-built network emulator and actual Starlink networks. The results demonstrate that the proposed algorithm effectively delivers a better quality of experience for low-latency live video streaming over Starlink networks, characterized by low live latency, high average bitrate, minimal rebuffering events and reduced visual quality fluctuation.


6. Measuring the Satellite Links of a LEO Network, [PDF][Slides]
Jianping Pan, Jinwei Zhao, Lin Cai
2024 IEEE 59th International Conference on Communications (ICC’24), doi: 10.1109/ICC51166.2024.10623111

Abstract

Low-earth-orbit (LEO) satellite networks have become very popular in recent years, exemplified by Starlink, OneWeb, Kuiper and others, due to the dramatically reduced launch cost and increased demand for connectivity anytime, anywhere. After an exploration of Starlink access, core and backbone networks, in this paper we focus on the satellite access network (SAN) of Starlink around the world. Particularly, we measure the access performance in terms of one-way delay and round-trip time from user terminal (UT) to ground station (GS) and point-of-presence (PoP), both inside-out and outside-in, and even on inactive dishes. It reveals the unique characteristics of Starlink SAN in terms of satellite-GS scheduling, media access control and user contention, and sheds light on the challenges and opportunities for network protocols and applications. The paper will be complemented by public dataset release and conference on-site demo for the research and industry community.


5. Measuring a Low-Earth-Orbit Satellite Network, [arXiv][Slides]
Jianping Pan, Jinwei Zhao, Lin Cai
2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC’23), doi: 10.1109/PIMRC56721.2023.10294034

Abstract

Starlink and alike have attracted a lot of attention recently, however, the inner working of these low-earth-orbit (LEO) satellite networks is still largely unknown. This paper presents an ongoing measurement campaign focusing on Starlink, including its satellite access networks, gateway and point-of-presence structures, and backbone and Internet connections, revealing insights applicable to other LEO satellite providers. It also highlights the challenges and research opportunities of the integrated space-air-ground-aqua network envisioned by 6G mobile communication systems, and calls for a concerted community effort from practical and experimentation aspects.

Updated backbone map in the US and Asia Pacific region in 2024 (Click to enlarge).

This work partially contributes to the creation of the map of Unofficial Starlink Global Gateways & PoPs.

Our work also partially inspired Hammas’s work on Making Sense of Constellations: Methodologies for Understanding Starlink’s Scheduling Algorithms.


4. High-Reliability, Low-Latency, and Load-Balancing Multipath Routing for LEO Satellite Networks
Yufei Wang, Lin Cai, Jun Liu
2023 Biennial Symposium on Communications (BSC), doi: 10.1109/BSC57238.2023.10201829

Abstract

Being a critical part of the sixth generation mobile networks (6G) infrastructure, satellite networks have rapidly developed in recent years. With the increasing number of satellites and high mobility, the challenges of Ultra-Reliable and Low-Latency (URLL) services are increasingly prominent. The regular topology and orbital movement of low earth orbit (LEO) satellites present a new opportunity for the design of network routing for URLL services. In this paper, we propose a High-Reliability, Low-Latency, and Load-Balancing Multipath Routing (HLLMR) to support URLL services for LEO satellite networks. To ensure the reliability of satellite network transmission, a packet is transmitted through multiple paths. The path and link selection strategy avoids hotspots through load balancing to ensure end-to-end reliability and delay and minimize the link cost. Using the Starlink constellation, we illustrate the advantages of HLLMR routing in terms of delay and reliability.


3. Distance-Based Back-Pressure Routing for Load-Balancing LEO Satellite Networks
Xia Deng, Le Chang, Shouyuan Zeng, Lin Cai, Jianping Pan
IEEE Transactions on Vehicular Technology, doi: 10.1109/TVT.2022.3206616

Abstract

Featuring wide coverage and high data rate, LEO satellite networks will be an important supplement to the traditional terrestrial networks, enabling the space-air-ground integrated network service. However, effective load balancing routing strategies for LEO satellite networks need to be designed, due to the bursty characteristic of the network traffic and imbalanced regional communication load. To achieve that, we propose a Distance-based Back-Pressure Routing (DBPR) strategy for LEO satellite networks. DBPR calculates the link weights based on a novel distance-based metric, which can select uncongested short-distance paths to the destinations and distribute network traffic dynamically with low delay. To control the number of forwardings in the network, we restrict the transmission range to a rectangle region between each source-destination pair. We design DBPR in the distributed fashion without collecting the global network load information, which is suitable for LEO satellite networks with limited resources, long propagation delay, dynamic topology, etc. We analyze the network stability and prove the throughput optimality of DBPR. Simulation results demonstrate that DBPR can achieve higher throughput and lower delay, compared with the state-of-the-art strategies, especially in the environments with limited cache resource.


2. MM-QUIC: Mobility-aware Multipath QUIC for Satellite Networks
Wenjun Yang, Shengjie Shu, Lin Cai, Jianping Pan
2021 17th International Conference on Mobility, Sensing and Networking (MSN’21), doi: 10.1109/MSN53354.2021.00093

Abstract

The Integrated Terrestrial and LEO Satellite Network (ITSN) is promising for providing ubiquitous communication services, which attracts attention but also brings new challenges. In this regard, a new transport layer protocol, Multipath QUIC (MPQUIC) appears salient advantages in tackling with the challenging environment (e.g., large propagation delays, high-speed mobility, etc.). However, the standard congestion control algorithm of MPQUIC, Opportunistic Linked Increases Algorithm (OLIA), still encounters great challenges such as congestion window (cwnd) overshooting whenever handoff, which motivates our proposal, a Mobility-aware Multipath QUIC (MM-QUIC) congestion control algorithm. MM-QUIC leverages the periodical changes of path capacity and good similarity among disjoint subflows to quickly start a new round of transmission, and employs a multipath-based fluid model to determine the cwnd adjustment in the congestion avoidance phase. Finally, simulation results on NS-3 demonstrate that MM-QUIC can offer up to 50% throughput improvement compared to OLIA in ITSN.


1. Directed Percolation Routing for Ultra-Reliable and Low-Latency Services in Low Earth Orbit (LEO) Satellite Networks
Junhao Hu, Lin Cai, Chengcheng Zhao, Jianping Pan
2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall), doi: 10.1109/VTC2020-Fall49728.2020.9348676

Abstract

With tens of thousands Low Earth Orbit (LEO) satellites covering Earth, LEO satellite networks can provide coverage and services that are otherwise not possible using terrestrial communication systems. The regular and dense LEO satellite constellation also provides new opportunities and challenges for network architecture and protocol design. In this paper, we propose a new routing strategy named Directed Percolation Routing (DPR), aiming to provide Ultra-Reliable and Low-Latency Communication (URLLC) services over long distances. Given the long propagation delay and uncertainty of LEO communication links, using DPR, each satellite routes a packet over several Inter-Satellite-Links (ISLs) towards the destination, without relying on link-layer retransmissions. Considering the link redundancy overhead and delay/reliability tradeoff, DPR can control the size of percolation. Using the Starlink as an example, we demonstrate that with the proposed DPR, the inter-continent propagation delay can be reduced by about 4 to 21 ms, while the reliability can be several orders higher than single-path optimal routing.

Technical Reports

1. Analysing the Performance of Cloud Gaming over a Low-Earth Orbit Satellite Network, [Report], [Poster]
Pouria Tolouei

Presentations

3. Research on Low-Earth-Orbit (LEO) Satellite Networks @PanLab, [Slides]
Jinwei Zhao
Kanazawa University & Internet Initiative Japan (IIJ) Research Laboratory


2. Accessible and Affordable Agricultural Technologies: A Space-Air-Ground Integrated Network Approach
Jianping Pan
AraFest’24 Lightning Talks


1. Global Access to the Internet for All, Anywhere, [Slides]
Jianping Pan
IETF 120 gaia: Global Access to the Internet for All, Vancouver

Datasets

2. LENS: A LEO Satellite Network Measurement Dataset, [GitHub]
Jinwei Zhao, Jianping Pan

1. Starlink Latency and Downlink Throughput Measurement Dataset[Zenodo]
Jinwei Zhao, Jianping Pan, doi: 10.5281/zenodo.10020034

Community Contributions

4. starlinkstatus.space, #1656

3. RIPE Atlas Probe Starlink-Denver, #60287

2. RIPE Atlas Probe Starlink-Victoria, #62390

1. LEOScope

Tools

3. starlink-lens, [GitHub], [Ubuntu PPA]

starlink-lens is a tool to collect Starlink user terminal (UT) to gateway latency with ping and irtt, for the LENS dataset.

2. Starlink GeoIP Map, [Data]

Starlink GeoIP Map is a map to visualize the planned naming and addressing of Starlink ISP, representing the association between user IP and Starlink’s point-of-presence (PoP).

1. Starlink Exporter, [GitHub]

Starlink Exporter is a set of tools to collect metrics from Starlink user terminal (UT)’s gRPC interface and visualize them in a Grafana dashboard.