Telecom and Software Insights

Testing NTN Apps Without Waiting for a Satellite Pass

hyavari26@gmail.com (Hossein Yavari) — Tue, 14 Jul 2026 22:00:52 -0400

I have been thinking about NTN from a very practical angle: if an app is supposed to work over satellite, how do we test it before the real network is available?

That sounds simple, but it is not. A satellite link is not just “slow internet”. It can be available for a short window, become worse near the edge of coverage, disappear completely, and come back later. If we only test with normal Wi-Fi or a basic offline toggle, we miss a lot of the behavior that matters.

That is why I started NTN-in-a-Box.

The goal is not to simulate the radio layer. There are already good projects for PHY and RAN research. I wanted something closer to the developer workflow: run an app, put it behind NTN-like conditions, and see what breaks.

What it does

NTN-in-a-Box creates a local sandbox where the network changes over time. A profile can describe things like:

coverage windows
latency and jitter
packet loss
bandwidth limits
when the next window starts or ends

On Linux, it can run a command inside a shaped network namespace. That means the application does not need to know it is being tested. Requests get delayed, dropped, or timed out by the sandbox.

For example:

go build -o ntnbox ./cmd/ntnbox/
go build -o poller ./cmd/poller/

sudo ./ntnbox run \
  --profile testdata/profiles/leo_pass_90s.yaml \
  -- ./poller

During the pass, the poller starts with a usable but not perfect link. The link improves, then degrades, then disappears. When the next window opens, it recovers. This is the part I care about most: not just “does the app work”, but “does the app recover cleanly after a weird network period”.

Why this matters

Most apps still treat connectivity as a yes/no state. NTN is more annoying than that. The app may technically be online, but the next two minutes might be a terrible time to upload a large file. Or the app may be offline now, but a short window is coming soon and queued work should be ready.

Those details change application behavior. I want to test questions like:

Does retry logic back off, or does it hammer the link?
Does the app queue messages locally when coverage is gone?
Does it flush the queue when the window returns?
Does it avoid starting a large transfer when the window is almost over?
Can the UI explain what is happening instead of showing a generic failure?

These are not only telecom problems. They are product and UX problems too.

The project includes small sample apps for things like adaptive polling, offline queues, store-and-forward messaging, Android emulator testing, and CI replay. They are intentionally simple because I wanted the failure mode to be visible.

TLE support

Static profiles are good for repeatable tests. For example, “give me a 90 second LEO-style pass every 10 minutes” is perfect for CI or demos.

But I also wanted to play with real satellite orbit data, so NTN-in-a-Box supports TLE files. You can generate a profile from a TLE and an observer location:

./ntnbox tle generate \
  --file testdata/tle/iss.tle \
  --lat 37.7749 \
  --lon -122.4194 \
  --output iss-pass.yaml

Or you can run from predicted passes directly:

sudo ./ntnbox run \
  --tle testdata/tle/iss.tle \
  --lat 37.7749 \
  --lon -122.4194 \
  --start-at next-pass \
  --speed 10 \
  -- ./poller

The GUI shows the orbit, the observer, and the coverage beam. It is not meant to be a mission planning tool, but it makes the pass feel much more real than a YAML file.

This is useful when the scenario is tied to a location or a specific satellite, not just a synthetic “good link / bad link” timeline.

API-first testing

Not every test needs traffic shaping. Sometimes the application just needs to know what the satellite state looks like and make a better decision.

NTN-in-a-Box exposes a local API for that:

./ntnbox serve --profile testdata/profiles/leo_pass_90s.yaml

curl http://localhost:8080/devices/sandbox-0/condition
curl http://localhost:8080/devices/sandbox-0/lookahead
curl http://localhost:8080/devices/sandbox-0/capabilities
curl -N http://localhost:8080/events

An app can ask:

Am I in coverage right now?
How long until the next transition?
What is the current delay, jitter, loss, and bandwidth?
Is a usable window coming soon?

This is where the developer workflow becomes interesting. You can build a countdown, delay a sync job, show a better message to the user, or test store-and-forward behavior without needing access to a commercial NTN environment.

Android and mobile workflows

Android already has NTN-related platform work, but most developers cannot easily test the behavior end to end. NTN-in-a-Box has two paths for this.

On Linux or WSL2, you can shape emulator traffic directly. On macOS, you can still expose the API to the emulator with adb reverse and test how the app responds to coverage changes.

I like keeping those two modes separate:

traffic shaping tells you whether the network path actually fails and recovers
the API path tells you whether the app understands NTN state and reacts properly

Both are useful. They answer different questions.

CI and replay

One feature I like is record and replay. If a test fails during a simulated pass, I do not want to explain it with “well, the network was weird”. I want to replay the same weirdness.

1
2

./scripts/demo.sh --record session.jsonl
./ntnbox replay --file session.jsonl --speed 10 -- ./my-app

There is also a GitHub Action so a team can wrap normal tests in an NTN profile:

- uses: hyavari/ntn-in-a-box@v1
  with:
    profile: leo_pass_90s
    command: npm test

The command still exits like a normal test command. The difference is that it ran through a network condition that looks more like the real world we are trying to prepare for.

Where it fits

I see NTN-in-a-Box as a small bridge between telecom concepts and everyday software testing.

For telco engineers, it is a way to explain coverage windows and constrained links without needing a full network setup. For app developers, it is a way to test ugly network behavior before users find it. For platform teams, it can help design APIs and UX around intermittent connectivity.

The project is still early, but that is the direction: make NTN conditions easy to run locally, easy to observe, and easy to repeat.

You can find the project here:

https://github.com/hyavari/ntn-in-a-box

Anonymize SIP Traffic in PCAP Files

hyavari26@gmail.com (Hossein Yavari) — Tue, 25 Nov 2025 08:00:00 -0800

When working with SIP traffic captures for debugging, analysis, or sharing with colleagues, you often need to protect sensitive information like phone numbers, domains, and IP addresses. However, you still want to maintain the ability to follow call flows and understand the signaling behavior. This is where anonymization becomes crucial.

I’ve created a simple Python tool called anonymize_sip that helps you anonymize SIP traffic in PCAP/PCAPNG files while preserving the structure and relationships in your captures.

Why Anonymize SIP Captures?

There are several scenarios where you need to anonymize SIP traffic:

Sharing captures for troubleshooting: When you need to share a PCAP file with a vendor or colleague, you want to protect customer privacy
Documentation and training: Creating examples or training materials without exposing real phone numbers or network infrastructure
Compliance: Meeting privacy requirements when storing or analyzing call data
Testing: Creating sanitized test data from production captures

The challenge is that simple find-and-replace doesn’t work well because:

The same phone number appears in multiple headers (From, To, Contact, P-Asserted-Identity, etc.)
You need consistent mapping so +11234567890 in one packet maps to the same anonymized value in all packets
Call-IDs need to be anonymized but still allow you to correlate messages within the same dialog
Phone numbers can appear in different formats (+11234567890, 11234567890, 1234567890) but should map to the same anonymized value

Features

The tool provides several key features:

Consistent Anonymization

The same original value always maps to the same anonymized value throughout the entire capture. This means if +11234567890 appears in the From header of an INVITE and later in the Contact header of a 200 OK, both will be replaced with the same anonymized identifier (e.g., user_001).

Phone Number Normalization

Phone numbers are normalized before anonymization, so +11234567890, 11234567890, and 1234567890 all map to the same anonymized value. This is especially important for North American numbers where the country code +1 might be included or omitted.

Comprehensive Header Support

The tool anonymizes all common SIP headers that contain user identities:

From, To, Contact
P-Asserted-Identity, Remote-Party-ID
Diversion, P-Called-Party-ID
History-Info, P-Charge-Info

Call-ID Correlation

Call-IDs are anonymized consistently, so you can still follow a call flow from INVITE through BYE, even though the Call-ID itself is anonymized.

Optional IP Anonymization

You can optionally anonymize IP addresses at both the packet layer and within SIP/SDP content. When enabled, it anonymizes:

Packet source and destination IPs
IPs in Via, Route, Record-Route, Path headers
IPs in SDP c= and o= lines

Memory Efficient

The tool streams packets instead of loading the entire file into memory, making it suitable for large capture files.

Usage

Basic usage is straightforward:

`1`	`python anonymize_sip.py input.pcap output.pcap`

To also anonymize IP addresses:

`1`	`python anonymize_sip.py --anonymize-ips input.pcap output.pcap`

To see the mapping table after processing:

`1`	`python anonymize_sip.py --show-mapping input.pcap output.pcap`

What Gets Anonymized

Always Anonymized

Field	Example Before	Example After
From header	`sip:+11234567890@example1.com`	`sip:user_001@domain_001.example`
To header	`sip:+10987654321@example2.com`	`sip:user_002@domain_002.example`
Contact	`sip:+11234567890@10.0.0.1:5060`	`sip:user_001@domain_003.example`
P-Asserted-Identity	`tel:+11234567890`	`tel:user_001`
Call-ID	`abc123@host.example.com`	`call-id-0001@anonymous.example`
Request-URI	`INVITE sip:+10987654321@...`	`INVITE sip:user_002@...`

With `--anonymize-ips`

Field	Example Before	Example After
Packet source IP	`192.168.1.100`	`10.0.0.1`
Packet dest IP	`192.168.1.200`	`10.0.0.2`
Via header	`Via: SIP/2.0/UDP 192.168.1.100:5060`	`Via: SIP/2.0/UDP 10.0.0.1:5060`
SDP c= line	`c=IN IP4 192.168.1.100`	`c=IN IP4 10.0.0.1`
SDP o= line	`o=- 123 456 IN IP4 192.168.1.100`	`o=- 123 456 IN IP4 10.0.0.1`

How It Works

The tool uses Scapy to read and write PCAP files. It processes each packet by:

Identifying SIP packets: Quick signature check for SIP indicators (SIP/2.0, INVITE, REGISTER, etc.)
Extracting SIP content: Decoding the UDP/TCP payload as UTF-8
Anonymizing SIP headers: Using regex patterns to find and replace URIs, phone numbers, and Call-IDs
Maintaining mappings: Keeping dictionaries that map original values to anonymized values
Reconstructing packets: Rebuilding the packet with anonymized content and recalculating checksums

The anonymization maintains consistency by using the same mapping dictionaries throughout the entire file processing.

Example Output

When you run the tool with --show-mapping, you get a summary of all anonymizations:

$ python anonymize_sip.py --anonymize-ips --show-mapping capture.pcap anon.pcap

Processing: capture.pcap
IP anonymization: enabled
Processed 1500 packets...
Done! Processed 1523 packets.
Anonymized: 5 users, 3 domains, 12 Call-IDs, 8 IPv4
Output: anon.pcap

============================================================
ANONYMIZATION MAPPING SUMMARY
============================================================

User Mappings (phone numbers normalized):
  user_001 <- 1234567890
  user_002 <- 0987654321
  user_003 <- 1112223333

Domain Mappings:
  domain_001.example <- example1.com
  domain_002.example <- example2.com
  domain_003.example <- ims.mnc000.mcc000.3gppnetwork.org

IPv4 Mappings:
  10.0.0.1 <- 192.168.1.100
  10.0.0.2 <- 192.168.1.200
  10.0.0.3 <- 10.10.44.131

Call-ID Mappings: 12 unique Call-IDs
============================================================

Extending the Tool

The tool is designed to be easily extensible. You can add support for additional SIP headers by modifying the IDENTITY_HEADERS or IP_HEADERS lists in the code:

# Headers containing user identities (URIs with phone numbers)
IDENTITY_HEADERS = [
    'From', 'To', 'Contact', 'P-Asserted-Identity', ...
    'Your-Custom-Header',  # Add your header here
]

# Headers containing IP addresses (used with --anonymize-ips)
IP_HEADERS = ['Via', 'Contact', 'Route', 'Record-Route', 'Path']

Important Notes

The output file has recalculated checksums, so it’s a valid PCAP file
Ports are preserved (not anonymized)
The mapping is deterministic within a single run but changes between runs
Keep the --show-mapping output private if you need to correlate back to original values
The tool only processes SIP packets; other traffic passes through unchanged

Installation

The tool requires only Scapy:

`1`	`pip install scapy`

You can find the source code and more details in the anonymize_sip repository.

This tool has been useful for me when sharing captures for troubleshooting or creating documentation. If you find it helpful or have suggestions for improvements, feel free to contribute!

Take care and stay safe! ;)

Useful Resources for LLMs

hyavari26@gmail.com (Hossein Yavari) — Tue, 04 Mar 2025 11:37:39 -0500

LLMs and Agentic AI are evolving fast, and like many in tech, I’ve been diving into them. Here’s a collection of resources I’ve found useful—hope they help you too!

Community-driven Evaluation for the Best LLM and AI Chatbots

📌 Llama Arena – A platform that lets users compare and vote on AI chatbot performance.

SEAL LLM Leaderboards for Precise and Reliable LLM Rankings

📌 Scale SEAL Leaderboard – A benchmark for evaluating and ranking AI models across multiple dimensions.

Tokenizer

📌 Tiktokenizer – A handy tool for visualizing how text is tokenized before being processed by LLMs.

Local Chatbot with Different Models

📌 Nomic AI – A platform for running open-source LLMs locally on your machine.

Personalized AI Assistant from Google

📌 NotebookLM – A research tool from Google that helps summarize, analyze, and interact with your personal notes using AI.

Building AI Agents with a Powerful Toolset

📌 LlamaIndex – A framework for connecting LLMs with private or structured data to create intelligent agents.

Extracting Metadata and Text from Sources

📌 Apache Tika - A toolkit detects and extracts metadata and text from over a thousand different file types.

Embeddings, Rerankers and Readers

📌 Jina AI – Different tools/products for building LMs.

Working with Complex Workflow

📌 LangChain – Build context-aware and reasoning applications.

Build and Manage MLOps

📌 METAFLOW – A Python library that makes it straightforward to develop data-intensive applications.

LLM Testing and Evaluation

📌 LLM Studio - A platform for testing and evaluating LLMs.

🚀 To be continued…

Testing RTT Feature with iPhone

hyavari26@gmail.com (Hossein Yavari) — Sat, 22 Feb 2025 10:42:08 -0500

Real-Time Text (RTT) is a feature designed to improve communication in situations where talking or making a voice call is not possible—such as emergencies or for individuals who are deaf, hard of hearing, or rely on TTY for phone calls. In this post, I’ve put together a quick overview of RTT and how it works: Real-Time Text (RTT) in IMS.

RTT has been available in the U.S. since around 2021, and in Canada, the CRTC mandated its implementation in 2020. However, as of now (at the time of writing), Canadian providers are being urged to deploy it as soon as possible, especially for NG911 and emergency calls.

I recently implemented RTT support in our IMS core and tested it using two iPhones. Below, I’ll share some screenshots and a video to give a clearer picture of how RTT works and how it enhances communication.

Let’s see what we have in RTT Settings: (Settings > Accessibility > RTT/TTY)

First, you need to enable the RTT feature. There are three interesting settings you can adjust:

Send Immediately: When enabled, the phone sends each character as you type, rather than waiting for you to finish your message.
Answer RTT Calls as Muted: This setting caught me off guard at first—I couldn’t figure out why my calls had no audio. If enabled, RTT calls start with the microphone muted, so you’ll need to manually unmute if you want to speak.
Live Captions: A great accessibility feature for individuals with hearing disabilities. It allows users to read messages before answering and respond via text.

Let’s have a quick demo:

Your browser does not support the video tag.

As you can see, one phone has the Send Immediately option enabled, while the other requires pressing Enter to send messages. You can also mute and unmute the audio during the call.

I hope this helps you understand how to enable and use the RTT feature, especially in emergency situations where you might not have been aware of it before.

Take care ;)

Reading Logs

hyavari26@gmail.com (Hossein Yavari) — Mon, 13 Jan 2025 14:08:24 -0500

Generated by AI

Welcome to My Reading Logs

Books are more than just stories; they’re journeys, lessons, and companions through the different phases of life. This page is a curated collection of the books I’ve read, each holding a special place in my personal reading journey. While I prefer not to write detailed opinions about the books, I do provide a personal rating (1-10 stars) to reflect how enjoyable and impactful I found them overall. I update this list regularly, adding each book as I finish it, so it helps me to remember what I read ;)

I hope it inspires you to explore some of these titles yourself!

Stop Overthinking: 23 Techniques to Relieve Stress, Stop Negative Spirals, Declutter Your Mind, and Focus on the Present

by Nick Trenton

Rate: (8/10)

Amazon Link

Reading Date: Jan/2025

The 10X Rule: The Only Difference Between Success and Failure

by Grant Cardone

Rate: (7/10)

Amazon Link

Reading Date: Feb/2025

No Rules Rules: Netflix and the Culture of Reinvention

by Reed Hastings and Erin Meyer

Rate: (9/10)

Amazon Link

Reading Date: March/2025

Think Again: The Power of Knowing What You Don’t Know

by Adam Grant

Rate: (5/10)

Amazon Link

Reading Date: March/2025

The Psychology of Money: Timeless lessons on wealth, greed, and happiness

by Adam Grant

Rate: (9/10)

Amazon Link

Reading Date: August/2025

Real-Time Text (RTT) in IMS

hyavari26@gmail.com (Hossein Yavari) — Thu, 07 Nov 2024 15:46:50 -0500

In today’s digital landscape, communication is increasingly inclusive, allowing for real-time, text-based exchanges that cater to users with different needs and preferences. Real-Time Text (RTT) is an essential technology within the IP Multimedia Subsystem (IMS), designed to facilitate text conversations as they happen, character by character. Originally developed to replace legacy TTY systems, RTT offers a universal solution for the deaf, hard of hearing, and anyone who prefers real-time text over traditional voice calls.

Download PDF Here

IMS Training: Part Two

hyavari26@gmail.com (Hossein Yavari) — Fri, 25 Oct 2024 12:42:13 -0400

I spent a few hours last weekend finalizing my notes on IMS and VoLTE architecture. I’m sharing a section here, hoping it helps anyone interested in reviewing key aspects of IMS networks. This part covers essential topics such as registration processes, Quality of Service (QoS) mechanisms, and SIP routing. It provides a concise summary of these core areas to assist those seeking a quick review or clarification on IMS fundamentals.

Download Part One PDF

Let me know your feadback! Take care ;)

IMS Training: Part One

hyavari26@gmail.com (Hossein Yavari) — Sun, 13 Oct 2024 10:09:11 -0400

One of the challenges with reading 3GPP documents is that it’s easy to forget the majority of details unless you work with them on a daily basis. For support or maintenance teams, this might be feasible, but for developers, it’s much harder. As a developer, you implement, deploy, debug, and often don’t touch the system again for a long time unless an issue arises or a new feature is requested. This is particularly true for telecom core elements like the MME, HSS, etc., which you rarely need to interact with.

With that in mind, I’ve started reviewing my old notes and highlighted 3GPP documents. I’m consolidating them into a single document that I can refer to whenever needed. It’s not focused on deep protocol flows, since those can easily be found when necessary.

I’ve been working on this project in my spare time, and this week I completed part one. I’ll continue summarizing all of my documents from the past three years related to IMS/VoLTE, and hope to finish soon.

Download Part One PDF

Let me know your feadback! Take care ;)

AWS ECS and Chime Voice Connector

hyavari26@gmail.com (Hossein Yavari) — Sat, 12 Oct 2024 09:20:52 -0400

AWS offers a wide range of services and tools that simplify the work of developers and solution architects. However, there are instances where certain limitations or missing features can be frustrating or challenging. In this post, I’ll share some workarounds I’ve implemented to make SIP trunks function smoothly on ECS Fargate with AWS Chime Voice Connector.

First, if you’re not familiar with Voice Connector, it’s a SIP trunking service for call origination and termination, part of the Chime SDK. It can be used to build conferencing solutions and other voice-enabled applications. For more information, you can explore the details here: AWS Voice Cconnectors

Problem: An important point about Voice Connector is that its proxy or edge SIP servers are only accessible via the internet using public IPs. While AWS offers a solution using Direct Connect, it doesn’t work if your servers are hosted within AWS itself. This creates the first limitation: your SIP servers must connect to Voice Connector through the internet, meaning they either need a public IP or must be behind a NAT (although NAT doesn’t work in this case). To address this limitation and the security concerns around call origination, AWS provides an Allowed Hosts List in the termination settings. You must add your server’s or NAT’s IPs to this list, which allows your servers to send SIP traffic to Voice Connector.

If your SIP servers are running on EC2, you’re in luck—it’s easier to assign Elastic IPs to your instances. You simply need to add those IPs to the allowed hosts list and make a few adjustments to your security groups to allow incoming traffic from Chime. You can find the AWS IP ranges here: network-config

But what if you want to utilizing the benefits of containers and cost efficiency and use ECS Fargate (NOT ECS with EC2), you will be in trouble. You can enable your Fargate task to get dynamic public IP but how should you deal with that Allowed Hosts List? Maybe hiring someone to just sit there and whenever a task goes down or new task comes up, update IPs list :))

This is where AWS doesn’t currently offer a viable solution. I spoke with support, and their suggestion was to use EC2 or NAT. Recommending EC2 to a team that’s looking for a Fargate-based solution feels more like avoiding the problem than solving it. So, what’s the issue with NAT? NAT works when your server is in a subnet behind a NAT gateway and only needs to initiate requests to Chime. However, for incoming requests—like a BYE message at the end of a call—your server won’t receive it because it’s behind NAT.

What about using a network load balancer (NLB)? While it’s another potential approach, it doesn’t work either. If you put an NLB in front of your ECS cluster and use its IP in the SIP Contact header, you can receive incoming SIP requests from Chime. However, you still need to register your tasks’ public IPs with Chime to allow tasks to send SIP requests—complicating things further. Sound complex? Don’t worry—stay happy! 😉

The great thing about AWS is that there’s almost always a way to work around limitations, thanks to the extensive APIs available for most services. This flexibility allows you to build custom solutions to tackle challenges—just like the approach that worked for me in this scenario.

Solution: One solution I implemented was delegating the task of updating the IP whitelist for Voice Connectors to a simple service. When an ECS task starts, it calls this service and provides key information about itself. The service then takes care of updating the Voice Connector’s termination settings and allowed IP list automatically.

Step 1: Integrating a call to an API within your container’s entry point is straightforward. You can send any relevant information that the service needs to understand details about the ECS cluster, service, region, IPs, and more regarding that task.

In our scenario, our SIP servers already maintain a WebSocket connection to the API Gateway, so we leveraged that same connection. This approach eliminates the need for additional authentication steps typically required for calling a REST API, although managing authentication isn’t overly complicated.

Step 2: Behind the API Gateway, we have a Lambda function that receives the notifications and processes them. This Lambda function is responsible for updating the Voice Connector’s termination settings and managing the IP list.

Let’s include some snippets here to illustrate what this Lambda function does. Also I didn’t mention here how you should find the Voice Connector’s ID. Consider that Lambda function has it:

We can get settings of Voice Connector like this:

    const {
        ChimeSDKVoiceClient,
        GetVoiceConnectorTerminationCommand,
        PutVoiceConnectorTerminationCommand,
    } = require("@aws-sdk/client-chime-sdk-voice");

    const voiceConnectorId = "";
    const voiceClient = new ChimeSDKVoiceClient({ region: REGION });
    const params = {
        VoiceConnectorId: voiceConnectorId,
    };

    const result = await voiceClient.send(
        new GetVoiceConnectorTerminationCommand(params)
    );

    const terminationSettings = result.Termination;
    const cidrList = terminationSettings.CidrAllowedList;

    // we update cidrList with new IPs and remove old IPs here
    const cleanedCidrs = [];

In this way, so updating is simple. Assume you updated the IP CIDR list:

    terminationSettings.CidrAllowedList = cleanedCidrs;
    const putParams = {
        VoiceConnectorId: voiceConnectorId,
        Termination: terminationSettings,
    };

    const updateResult = await voiceClient.send(
        new PutVoiceConnectorTerminationCommand(putParams)
    );

Let’s assume the event we receive from the Fargate task includes its public IP, which we can easily add to the allowed CIDR list. But what about removing invalid IPs that are no longer in use by our tasks? To handle this, we can retrieve the eni (Elastic Network Interface) IDs for all tasks in the service. Then, we can get the public IPs assigned to these ENIs and remove any IPs that are no longer present in the task list from the Voice Connector’s CidrAllowedList.

    const tasksPublicIps = [];

    const tasks = await ecsClient.send(
        new ListTasksCommand({ cluster: clusterName })
    );

    for (const taskArn of tasks.taskArns) {
        const task = await ecsClient.send(
            new DescribeTasksCommand({ cluster: clusterName, tasks: [taskArn] })
        );

        const enis = task.tasks[0].attachments[0].details.filter(
            (detail) => detail.name === "networkInterfaceId"
        );

        for (const interface of enis) {
            const niId = interface.value;
            const ec2Client = new EC2Client({ region: REGION });
            const niDetails = await ec2Client.send(
                new DescribeNetworkInterfacesCommand({
                    NetworkInterfaceIds: [niId],
                })
            );

            const publicIp = niDetails.NetworkInterfaces[0].Association?.PublicIp;

            if (publicIp) {
                tasksPublicIps.push(publicIp);
            }
        }
    }

Now that you have an updated list of valid public IPs for your tasks (SIP servers), the final step is simply updating the appropriate Voice Connector they’re connected to. And that’s it—you’re done!

I hope this post gave you some useful insights for handling similar scenarios in the future.

Take care! ;)

Trust: A Hidden Treasure Among Teammates

hyavari26@gmail.com (Hossein Yavari) — Tue, 08 Oct 2024 10:27:12 -0400

“The best way to find out if you can trust somebody is to trust them.” – Ernest Hemingway

It’s been a long time since my last blog post, but today, in Ottawa, I want to dive into a topic that has taken me over 12 years to fully grasp: trust—an essential element in any long-lasting relationship, whether personal or professional.

In this post, I want to share insights into how trust operates within teams, particularly in the workplace. Trust isn’t built overnight; it requires time and effort. However, once established, it becomes a hidden treasure, unlocking the potential for creativity, collaboration, and success within a team.

Note: These are just a few thoughts that might help guide you, especially if you’re feeling uncertain about your role or the environment you’re working in.

Let’s start with the foundation of any job: the interview process. In my experience, outside of major companies like FANG (or similarly large corporations), the seeds of trust are planted as early as the interview stage. You trust the company enough to commit your most valuable resource—your time—because you believe in its mission and future.

This trust fosters a sense of security, allowing you to take risks, be more creative, and engage deeply with your work. When you feel safe within your role and team, you’re likely to be more productive and innovative. There’s no shortage of articles that highlight the importance of building trust in teams and organizations—this is no coincidence. Trust is a key to unlocking the best in people.

But how exactly do you establish trust? Someone joins your team or works alongside you, maybe as a co-founder, and they trust you. But why? Perhaps you offered them something compelling enough to earn their trust, or maybe they convinced themselves you’re trustworthy based on external factors—like income or a fancy job title—without fully knowing you.

I personally don’t favor the latter. Trust that’s rooted in shallow factors, like a paycheck or an impressive title, can be short-lived. Sooner or later, people who rely on these external motivators realize they’ve made a mistake. Before they fully understand, they may leave, placing blind trust elsewhere, which eventually leads them down a path of disappointment.

Now, let’s focus on the first group—those who trust you because you’ve given them genuine reasons to. If you’re authentic and the things you value align with what you’ve communicated, this opens up great opportunities for both sides. This type of relationship fosters the highest levels of creativity and productivity, leading to success for the entire team. As mentioned at begining, this is the kind of trust that leads to great outcomes.

On the other hand, if you’re just playing a role or putting on a facade, even if you’re skilled at it, your business or team will inevitably suffer in the long run. You’ll lose valuable people, and nothing is more essential to a business than its people. The worst scenario occurs when someone exploits trust for personal gain. This doesn’t just harm the team or the business—it deeply impacts the individuals who placed their trust in you.

What worries me most isn’t the damage to the business or the team, but the effect on those who trusted you. When they realize they made a mistake by placing their trust in you, the sense of betrayal and neglect can be overwhelming. Often, they don’t dwell on how you deceived them; instead, they reflect on all the moments they dedicated themselves to the team’s success, only to realize the reward wasn’t what they expected. The reality hits them that they trusted someone who merely wore a mask, prioritizing personal gain over the well-being of the team.

To those people: this is not your fault. You did what any reasonable person would do—you gave your trust and did your best in an environment where you believed everyone shared the same goals. You were dedicated to the team’s success, and that effort is something to be proud of, regardless of the outcome.

Unfortunately, this story is one I’ve experienced firsthand. After more than 10 years of hard work, dedication, and passion, I realized that I had placed my trust in the wrong entity. In long business relationships, friendships often develop as well, and that made the situation even harder to digest. At first, it was difficult for me to accept, but once I did, moving forward and making tough decisions became easier.

This experience taught me to think more broadly in similar situations and to prioritize myself and my family above everything else. Even in environments that seem trustworthy, it’s important to remember that you should always be your top priority. If someone or something isn’t respecting that, it’s a clear sign that it’s not the right place for you.

Closing Thoughts

In the end, trust is the foundation of any successful team or business. It’s what allows people to take risks, to be creative, and to fully invest themselves in the shared goals of the team. While building trust takes time, the reward is a team that’s not only productive but also resilient and capable of weathering challenges together.

If you’re leading a team or starting a business, remember that trust isn’t just something to ask for—it’s something to actively cultivate through transparency, empathy, and consistency. And if you’ve ever been in a position where your trust was misplaced, don’t let it discourage you from trusting again. Instead, use that experience as a lesson in recognizing the signs of authenticity, and remember that real trust leads to growth, both for you and those around you.

IMS Network Announcements

hyavari26@gmail.com (Hossein Yavari) — Fri, 17 May 2024 10:17:34 -0700

A chorus of howler monkeys of the species Alouatta caraya. Mariana Raño.

In telephony services, providing voice announcements to call parties is standard practice. This feature is essential for mobile operators to ensure that subscribers receive information about incomplete calls or dialing restrictions. It enables customers to understand the reason behind any issues they encounter and facilitates follow-up with customer service for further assistance.

In IMS networks, this task is typically handled by the Media Resource Function (MRF), which plays network announcements in appropriate situations.

There are various options available for implementing an MRF. I’d like to focus on one specifically designed for cloud-native IMS core solutions like ours. Our preference lies in solutions that require minimal server infrastructure and maintenance, streamlining integration without adding additional overhead.

For those who may have missed my previous posts, I’m excited to share that my team successfully implemented an IMS core on AWS, primarily utilizing Lambda functions! If you’re curious about how we achieved this feat, feel free to reach out. I’d be happy to discuss further details ;)

Before, I embarked on an intriguing journey using AWS Chime to implement the IMS call conference or Ad-hoc multi-party conference feature for our service. After overcoming various challenges, we successfully launched this feature, entirely leveraging Chime’s capabilities. The best part? No additional servers or containers were required, saving us from ongoing maintenance headaches. You can read more about our experience and the implementation process in detail on my blog here.

I’m contemplating replicating the same strategy for network announcements with AWS services, aiming to leverage existing solutions rather than reinventing the wheel. In the past, I implemented an IVR for incoming calls on AWS Voice Connector trunk using the SIP Media Application, which was both effective and efficient. I’m confident that this approach can be seamlessly adapted for network announcements as well.

You’ve probably grasped the concept: setting up a trunk to the Voice Connector and defining a specific SIP rule to direct calls over that trunk to your SIP Media Application, triggering a Lambda function in the process. The flow would be like this:

For more please check this link:

https://docs.aws.amazon.com/chime-sdk/latest/ag/understand-sip-data-models.html

For example, you could implement a Lambda function like this, utilizing Amazon Polly to perform text-to-speech (TTS) and play the configured announcement for incoming calls.

This solution, in addition to maintenance and cost, is very dynamic and you can support different language announcements without handling pre-recorded voice prompts. You can update the content and language of announcements on the fly.

Hint: You can add additional SIP headers to your INVITE starting with ‘x-{}’, and carry necessary information for using in Lambda logic. Read more here:

https://docs.aws.amazon.com/chime-sdk/latest/dg/sip-headers.html

CloudWatch Log Metric Filters

hyavari26@gmail.com (Hossein Yavari) — Fri, 16 Feb 2024 15:39:58 -0800

Yesterday, I successfully configured distinct metric filters for specific CloudWatch log groups within our AWS lab environment. After verifying their effectiveness, I realized the need to replicate these filters across both our AWS staging and production environments.

** I understand that AWS CloudWatch offers cross-account capabilities, but let’s assume for a moment that utilizing them isn’t feasible or recommended, as was the case in my situation. Additionally, there are instances, like mine, where swiftly integrating your CDK and deployment process to accommodate these requirements may not be feasible. While it’s true that log group metric filters can be added through AWS CDK during the deployment phase, implementing such changes entails additional steps and approvals for staging and production updates. Nevertheless, I wanted to highlight that my solution could potentially offer a time-saving alternative in certain scenarios.

To expedite this process, I leveraged the AWS CLI tool. This involved exporting designated log group metric filters from one AWS account and subsequently transferring them to another account. Additionally, I utilized the tool to import metric filters from a JSON file, which had been previously exported from log metric filters.

You can find this script here in Gist:

https://gist.github.com/hyavari/9d55f1fe4e0f608eaa8f5c415cc6a73a

Usage is not complicated. You only need SSO profiles (that in my case, I regularly use them.)

for example:

for importing a JSON file including filters you can do like this:

`1`	`./cloudwatch_metric_importer.sh -i --input metrics.json --profile lab001 --region us-east-1`

or if you want to export (copy) from one account then import to another account:

`1`	`./cloudwatch_metric_importer.sh -e --profile lab001 --region us-east-1 --log-group groupName001 --dst-profile staging001 --dst-region us-east-2`

I hope this helps you save time!

Always there is place for improvement, so please do not hesitate to contribute them.

Take care ;)

Usefule Links:

[1] CloudWatch Metrics - https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/MonitoringLogData.html

[2] CloudWatch Metric Filters Regex - https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html

ECXCapture: AWS ECS/EC2 Easy Tcpdump Tool

hyavari26@gmail.com (Hossein Yavari) — Sat, 06 Jan 2024 23:09:47 -0800

I’ve recently developed a straightforward yet, in my opinion, a quite handy tool for network troubleshooting. This tool simplifies the process of obtaining tcpdump from your AWS ECS tasks or EC2 instances directly from your desktop, all in one place!

While it may not boast the latest buzzwords like LLM and AI that seem to be popping up everywhere these days, it serves a practical purpose for tech enthusiasts who want a quick glance at the tcpdump of their servers with just a few simple steps.

You can access this Python tool on GitHub at:

ECXCapture GitHub Repository

All the necessary details are included in the repository’s README, but feel free to reach out if you encounter any issues.

Do not forget to star it :)

Give it a go and kick off the troubleshooting journey! 😊

iPhone Live Voicemail

hyavari26@gmail.com (Hossein Yavari) — Wed, 20 Sep 2023 19:38:45 -0700

Probably you have updated your iPhone iOS recently to version 17 and the only thing that in the first place you have recognized, it was the phone screen when someone calls you. You can find a Voicemail icon that represents the new Live Voicemail feature.

This feature lets you get real-time transcription of the caller’s message and if you want to answer the call or reject it. In other words, your voicemail would be on your phone not on your carrier side anymore. It somehow reminds me Google Pixel Call screening.

Anyway, I was testing this feature with our IMS core to see how it might affect conditional call forwarding. As you know when Live Voicemail is enabled, for call forwarding busy and unanswered, the device answers the call and records it ( and transcribes it).

So Apple has considered that the default call forwarding destination is voicemail. However, if subscribers change the forwarding destination to another number, they need to disable Live Voicemail and let the carrier handle call forwarding to the desired number. Otherwise, your calls will be answered by Live Voicemail.

For call forwarding unconditional (CFU) and unreachable (CFNRC), as the operator decides, it follows the subscriber’s settings and the carrier handles the subscriber’s voicemail.

In conclusion, technically carrier doesn’t need to do anything with this feature, just might push some knowledge to their customer service about it if the subscriber claims that their conditional settings do not work properly.

Reminder: You can change CF* settings on iPhone with these codes:

Forwarding no answer: **61*1+PhoneNumber#
Forwarding unreachable: **62*1+PhoneNumber#
Forwarding on busy/reject: **67*1+PhoneNumber#

For Android, it is doable via Supplementary settings on Call Settings menu.

X2/X3 Lawful Interception Dissector

hyavari26@gmail.com (Hossein Yavari) — Thu, 24 Aug 2023 23:24:53 -0700

Caption: Photo courtesy of John Downer Productions

Lawful Interception (LI) standard play a crucial role in enabling lawful interception activities by authorized entities for security and intelligence purposes. These protocols allow the interception and monitoring of communication services while adhering to legal requirements. If you’re involved in IMS/mobile core or other telephony networks development, or communication analysis and security, you’ll find a dissector besides Wireshark be an invaluable addition to your toolkit. In this blog post, I’ll introduce you to x2x3PduDissector, a powerful Lua-based Wireshark dissector for analyzing LI (X2/X3 PDU Format) packets.

What is x2x3PduDissector?

x2x3PduDissector is an open-source project available on GitHub that provides a Wireshark Lua script for dissecting LI packets. The script is designed to decode various fields within the X2/X3 PDU Format, making it easier for network analysts to understand and interpret captured traffic related to this protocol.

Key Features:

It is based on ETSI TS 103 221-2 V1.4.1 - April 2021


 0                   1                   2                   3 
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           Version             |           PDU Type  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           Header Length
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           Payload Length
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        Payload Format         |     Payload Direction
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       
|
|                               XID
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           Correlation ID
|    
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|        Conditional Attribute Fields (Variable Length)            
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     Payload (Variable Length)                
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Version, PDU Type, Header Length, Payload Length, Payload Format, Payload Direction, XID (UUID), Correlation ID, and Conditional Attributes are all decoded for thorough analysis.
Payload interpretation for RTP and SIP messages is included.
Conditional attributes are dissected as Type-Length-Value (TLV) structures.
The script enhances the analysis of LI packets by providing a clear representation of the protocol’s fields and their semantic meanings.

Usage:

To use x2x3PduDissector, follow these steps:

Clone or download the repository from GitHub: https://github.com/hyavari/x2x3PduDissector
Place the Lua script in Wireshark’s Plugins directory or load it manually through the “Tools > Lua > Evaluate” menu.
Once loaded, the script will automatically dissect packets using the “X2X3” protocol.

Example Output:

Once the script is applied in Wireshark, you’ll see a detailed breakdown of each packet’s fields and values. This breakdown includes information such as the PDU type, payload format, XID (UUID), and more. Conditional attributes are dissected to provide a deeper insight into the packet’s content.

Contributions and Enhancements:

x2x3PduDissector is an open-source project, which means you can contribute to its development and enhancement. If you encounter any issues or want to improve its functionality, consider submitting pull requests or reporting issues on the project’s GitHub repository.

Notes:

LI (X1/X2/X3) is standard, but most of the vendors do not follow the standards when they are using their proprietary software or implementation. So do Not be upset if this dissector can’t help you :)
I also implement a simple X2/X3 PDU decode in node.js. You can find it here: https://github.com/hyavari/x2x3PduDecoder
These repos are for half-day work and I will update them if I find any issues.
Always there is place for improvement, so please do not hesitate to contribute them.
Do not forget to sarring these projects :))

SIP PANI Header

hyavari26@gmail.com (Hossein Yavari) — Tue, 22 Aug 2023 12:36:39 -0700

If you are testing an NG911 IMS call, one of the most important pieces of info that the NG911 network (including LRF, NGCS, PSAP, …) expects from your SoS SIP INVITE is the P-Access-Network-Info or PANI header. This header carries the information about the UE/Subscriber device location.

If you are using testing platforms like SITE (Mobileum/SIGOS), you have some configs in the test case that need to manipulate based on the requirements.

But let’s focus on TAC (Tracking Area Code) and ECI (E-UTRAN Cell Identity) values. In the screenshot, the values are in decimal, and if you do the simple calculations you get these:

TAC: 55519 => D8DF HEX

ECI: 139116310 => 84ABF16 HEX

Now if you check a sample SIP PANI header that is like below, It would be a little more meaningful:

P-Access-Network-Info: 3GPP-E-UTRAN-FDD;utran-cell-id-3gpp=303230D8DF84ABF16

let’s split the header value:

3GPP-E-UTRAN-FDD, utran-cell-id-3gpp, 303, 230, D8DF, 84ABF16

The first part is radio access technology or access-type => 3GPP-E-UTRAN-FDD. access-type field is followed by the access-info: utran-cell-id-3gpp which identifies the cell that provides network access. And after equal, you have the Cell ID / CGI which its pattern is MCC + MNC + TAC HEX Value + ECI HEX Value.

This info would be useful if you need to fake the header value for testing purposes.

Have fun :)

RTP Decoder/Encoder

hyavari26@gmail.com (Hossein Yavari) — Sat, 19 Aug 2023 00:31:48 -0700

Ever faced the challenge of deciphering RTP packets nested within another protocol PDU? 🤔 I found myself in this exact situation today. Amidst the lack of a specialized protocol decoder in Wireshark, I decided to roll up my sleeves and craft a nifty RTP packet decoder and encoder using Python.

Guess what? It worked like a charm! 🚀 I’ve shared this handy piece of code on Gist, with the aim of becoming a time-saving beacon for fellow developers. Imagine skipping hours of head-scratching and diving straight into efficient RTP packet handling.

Curious to dive into the magic? 🧙‍♂️ Check out the code on my Gist and unlock the power to decode and encode RTP packets effortlessly. Let’s save someone, somewhere, precious time – one protocol at a time!

Feel free to tweak this version to match your writing style and your audience’s preferences.

IMS Call Conferencing

hyavari26@gmail.com (Hossein Yavari) — Fri, 04 Aug 2023 14:40:09 -0700

Really call conferencing is not a feature that I am going to spend time on it to introduce to you or count its pros. We live in the web and cloud era and call/video conferencing, especially over WebRTC is mature enough. But again when you want to touch any feature like this in the 3GPP territory, might it be a little confusing. The idea is simple but it is a habit for some people that make the story complicated when they relate it.

So in this blog post, I wanna share some technical points that I learned from implementing this feature for our IMS core over AWS. Aha, I should mention also that IMS (VoLTE/VoNR) call conferencing is actually a 3-way call, which means you would be on a call with two other subscribers, NOT more. They call it Ad-Hoc Multi Party Conference, means subscriber creates conference room/call on the fly.

Some of useful resources around IMS call conf if you want to get more details:

Let me make it very simple for you. UE-A is in call with UE-B, then puts call on hold and calls UE-C; then seems they need to talk to UE-B together, so UE-A puts UE-C on hold and sends a request for creating a conference room. X-CSCF gets this new INVITE which its To header includes conference URI and forwards it to correct application server (TAS) to handle it. This is how call conferencing begins in IMS; like below call flow diagram:

Do not confuse by naming. TAS or Conference AS, focus or conference factory all are the same and it is actually the element that manages the conferencing creation, moving parties to it, and then removing it. Whatever you want, just name it :)

The INVITE RURI could be like this:

When AS accepts this request, it provides a media path via SIP 200 OK SDP. Note that AS might not handle media itself and works with another element as MRF (Media Resource Function). Actually, MRF mixes the media for the conference. When UE-A receives this OK and gets into the conference, it starts to ask AS to move UE-B and UE-C to the conference. For this purpose, UE-A uses SIP REFER message to address AS about call parties.

Ok, so you need to handle REFER correctly. the Refer-To header is the place including info that AS needs to know.

Refer-To: <sip:1XXX4292400;phone-context=ims.mncXXX.mccZZZ.3gppnetwork.org@ims.mncXXX.mccZZZ.3gppnetwork.org;user=phone;method=INVITE?Replaces=wooY93GbgDHgH7zV7xXS6g..%402001:56f:f800:117d:0:2a:002d:1c01%3Bto-tag%3D20132573814420238217146%3Bfrom-tag%3D43f1826a>

This header includes Call-ID, from-tag, and to-tag of the original calls. So UE-A sends separate REFER for each UE-B and UE-C.

But how UE-A understands that other parties have been moved to the conference? with SIP NOTIFY. UE-A subscribes itself to notifications for conference dialog and then TAS keeps it updated about the steps that it does.

First TAS says I am trying to move party to conference. So SIP NOTIFY has Sipfrag SDP with Trying state:

When UE moved to conf, TAS sends NOTIFY with OK 200 in the Sipfrag SDP:

At this point, all parties are in the conference, and from a signaling point of view, flow is normal to finish the call for all legs, except if the conference initiator (UE-A) wants to drop other parties from the conference, it sends another SIP REFER message that in Refer-To header it says that party’s URI with method BYE. If UE-A terminates the call, the whole conference call will be terminated.

Refer-To: <sip:1XXXY292400;phone-context=ims.mncXXX.mccZZZ.3gppnetwork.org@ims.mncXXX.mccZZZ.3gppnetwork.org;user=phone;method=BYE>

This was a simple overview of the IMS call conferencing feature. But if you want to implement it I have some hints here:

Handle REFER message correctly. This is the only message that carries information about other parties.
NOTIFY messages that you send to the conference initiator are implementation keys. When UE-A gets the correct Sipfrag info for UE-B, it sends a second REFER for UE-C, otherwise, UE-A sends a BYE and terminates the calls.
TAS or conference AS is a simple B2BUA telephony server with media mixing capability. So a FreeSwitch or maybe Asterisk server can do the job for you. As we developed most of our services on AWS, using the FreeSwitch on ECS or EC2 instances was the first option. But later we decided to put the maintenance responsibility on AWS’s shoulder and moved our conferencing factory to AWS Chime. Although AWS Chime also is using the FreeSwitch, at least we don’t care about the maintenance and other server stuff.

More idea: https://docs.aws.amazon.com/chime-sdk/latest/dg/mtgs-sdk-cvc.html

Wirehark is your only friend on this path, hug it :)

I will be really happy to help whoever wants to implement this feature or who is interested in getting more info about our IMS core. Just reach out to me with my contact info !

SIPREC Server on AWS

hyavari26@gmail.com (Hossein Yavari) — Fri, 30 Jun 2023 09:57:42 -0700

Any telco company and MNO must provide tools or interfaces to third-party companies or government agencies for monitoring or lawful interception (LI) purposes. There are various 3GPP standards for LI systems and their interfaces like X1/2/3 that you can refer to. In addition, sometimes when customers complain about calls and voice qualities, and customer services report it, it is required to have a tool to monitor a specific subscriber or bunch of subscribers to verify what the issue is or what causes that.

No matter the 4G(VoLTE) or 5G network, the IMS core is the place where you need those tools or interfaces. Capturing SIP signaling is not complicated, and various options like P-CSCF or PGW exist to capture it. However, PGW is not your option for non-troubleshooting captures! the P-CSCF interfaces after IPSec decryption are one of the best places to fork SIP signaling for any purpose.

Recently, using the eBPF for this purpose is growing and it seems there are many valuable applications for that.

For media capturing, methods are different. Real-Time Transport Protocol (RTP) plays a crucial role in enabling the transmission of multimedia data, such as audio and video, across the network. RTP protocol capturing in IMS involves the monitoring and analysis of RTP packets exchanged between IMS endpoints during communication sessions. By capturing RTP packets, network administrators and engineers gain valuable insights into the quality, performance, and troubleshooting of real-time media streams. This captured data can be used to analyze and diagnose issues related to voice and video call quality, jitter, latency, packet loss, and other metrics. The captured RTP packets are typically analyzed using specialized tools and protocols to extract information about the media streams, codecs used, timing, synchronization, and other relevant parameters, which in LI maps to the X3 interface.

Using the networking interfaces of media servers, sniffing RTP packets, barging into the media session and call recording are some of the methods for media capturing and each has pros and cons. Although, eBPF as I mentioned, could be really interesting for this purpose!

Recently I worked on a SIPREC server to fork media sessions and later it lets us record the call or intercept the voice. If you are not familiar with SIPREC, please check my earlier post here: SIPREC with RTPEngine.

This is the overall architecture:

There are some facts that I didn’t mention in the diagram:

All communications are secured by HTTPS or WSS.
Authentication is managed but Cognito.
VPCLink and VPCE are used for more security in transport.
Obviously, the solution is on multi-region and cross-zone!

Ok good, but what exactly we are doing?

Calls are established normally which is out of the scope of this post ( I copied this from 3GPP docs :| )
Provider logic detects that they need to do recording or forking for a specific call or specific subscriber.
Request is sent to SIPREC service API with information of call or subscriber.
As a result of those information, lambda function logic extracts the hosting media server and sends all requisite information to the SIPREC server.
SIPREC prepares the forking or subscribe request for the media server (e.g. RTPEngine for us) and then after some negotiation for codec and ports, SIPREC starts to receive call legs media streams or RTP packets.
Server can save media locally or sends a bunch of packets over WebSocket to lambda and then lambda process it or save it into S3.

Please note that we have call SIP info, so decoding the media is not complicated.

Good to know that the SIPREC server does not have a very complicated implementation. It understands SDP, codec negotiation, and UDP connection management.

Challenge: WebSocket API gateway on AWS has a 32KB frame size limitation: https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#apigateway-execution-service-websocket-limits-table

So for batching the RTP packets you need to take care of it and do fragmentation and etc.!

Enjoy your day :))

VoLTE on IPhone

hyavari26@gmail.com (Hossein Yavari) — Sun, 04 Jun 2023 10:49:45 -0700

If you are following my blog and have read my previous post about testing VoLTE in Samsung and Pixel handsets, I appreciate that, and this post is a quick update. If not, please review the first part VoLTE in Pixel vs Samsung Phones and then return here.

When we were developing the IMS core, we used a Samsung handset as our engineering phone or test phone and everything seemed good. When we were ready to start testing phones from other brands, we got a report from the marketing and sales department about how different brands are popular in Canada. I think you can guess that iPhone is at the top of the list! and then Samsung and Google Pixel and other brands like LG, Xiaomi, etc. following.

But testing VoLTE on iPhone is not straightforward because it needs Entitlement Service. I will go to the details of the entitlement service in another post but in simple definition, an entitlement server is a set of specifications to allow mobile operators to enable and disable services and features on end-user devices and iPhone queries this service to know that VoLTE is enabled in the network or not and then attach to IMS APN.

So, we started implementing entitlement service as well, and a few weeks ago we could start testing iPhones. Fortunately, iPhone doesn’t do any weird behavior and everything looks normal and standard. It can be registered in our IMS and make voice calls and SMS.

The only point that I noticed is that iPhone sends SMS over IPSec/UDP, while all voice call SIP messages are over TCP.

Also in SIP headers, iPhone sends Session-ID that might be useful in B2BUA cases. I should investigate more about how we can use it! but you can check this RFC 7329.

If I find anything new or interesting about iPhone VoLTE, will keep you posted.

Take care :)

VoLTE in Pixel vs Samsung Phones

hyavari26@gmail.com (Hossein Yavari) — Wed, 03 May 2023 21:39:20 -0700

When you are implementing mobile core services, and struggling with ridiculous 3GPP standards, (don’t panic, if you work with those docs, you can understand what I am saying! ), you should be grateful if you have a production reference capture. In this step, you probably can test some of your functions and at least try to achieve the same output or data structure.

But in the next step, you need to test it in the field with a real network, sim card, and handset. This step is not very straight forward and because of that, some companies sell testers for these types of mobile services, like SIGOS. But after all, production is something else :) so don’t be so happy if everything seems OK in the lab environment.

From my experience, testing services with real phones from different brands, lets you see various behaviors and then add correct handlers into your implementation. Do not forget that you can not find these things in the 3GPP standards, especially in IMS and VoLTE, because it’s very device dependent.

I guess you know the context now, then let’s deep dive into the main subject of this post: VoLTE behaviors in Google Pixel vs Samsung. I did various tests with Pixel 3a and 6Pro, Samsung S8+, and Samsung S22. Different models of each brand don’t have much difference as they are using the same IMS client app, but the two brands are quite different. I found a few of them and have listed them below:

1- IPSec

The first thing you need to have for VoLTE is IPSec. IPSec in Samsung phones is very straightforward. The device sends a REGISTER over TCP to P-CSCF and offered its IPSec SAs, then P-CSCF send 401 with Server IPSec SAs info and then the next REGISTER from UE would be sent over the IPSec tunnel to P-CSCF.

These tunnels would be used while registration is valid. The point is that Samsung phones send all the SIP signaling requests and responses over ESP with TCP transport, something that they agreed with REGISTER, whereas Pixel phones do not follow this rule and they send some SIP messages over ESP packets but to UDP transport!

In other words, if you want to support Pixel phones, you need to establish IPSec tunnels and listeners for both UDP and TCP.

**as you can see in the image, VIA transport is UDP, but RURI transport is TCP. It is really weird!

2- SMS over IP

As you see, Pixel needs the UDP listeners as well in P-CSCF IPSec; it seems if the size of the message is small it uses UDP otherwise it uses TCP transport messages that are encapsulated in ESP packets. Another case that is different in Pixel is SMS over IP or SMS in VoLTE. Samsung always sends SMS over TCP, while Pixel sends over UDP!

3- Dedicated Bearer

One of the VoLTE requirements is the dedicated bearer; which you create a bearer for media (/signaling) when the call is established. It helps that PGW and other elements in the network respect the media traffic and provide enough bandwidth and QoS.

4- SIP Headers

Another minor difference between Pixel and Samsung is using TEL URI and SIP URI. Pixel uses TEL URI for the To header but Samsung always sends the SIP URI in headers including To. It is important because if you developed an in-house SIP stack, you need to add TEL URI parsing capability into it.

to be continued…

Still might be other differences in IMS clients in these popular brands, so whenever I face with, I will update this post. Although, I will add iPhone and other tested brands soon.

AWS NLB Is Transparent

hyavari26@gmail.com (Hossein Yavari) — Thu, 06 Apr 2023 09:13:14 -0700

Hi readers, I just came to share a short story (actually a fact) that I learned recently with you that might not be new to you! The Elastic Load balancer is one of the typical services of AWS that you usually have a case that needs it. It is the same here and we are using ALB (Application Load Balancer) and NLB (Network Load Balancer) in our services. But I noticed a fact about AWS NLB that changed my assumptions. NLB does pass through the TCP connections to the targets, so it is transparent!

If a target goes down, health checks for the target will fail and the target will be marked as unhealthy. NLB will stop routing traffic to that target and reroute traffic to remaining healthy targets.

The TCP sessions are terminated on the target(s) in a target group. The NLB selects a target (whichever is healthy) based on the load balancing algorithm and the selected target return responses to the client. Basically, the NLB is transparent to client requests! My assumption was that NLB terminates the session into itself and then creates a session to target! Something like offloading feature in TLS mode was expected.

Other relevant facts:

NLB is not a gateway! you can not convert attach UDP target to a TCP load balancer.
NLB doesn’t do failover at the session level. If you lose the target, the client will lose the established connection, so need to establish a new connection with the new target.

Hope this short blog, corrects the wrong assumption of someone :))

SDP Bandwidth Attributes

hyavari26@gmail.com (Hossein Yavari) — Wed, 22 Mar 2023 00:31:53 -0700

Session Description Protocol (SDP) is a communication protocol used for describing multimedia sessions between two or more endpoints. SDP provides a standardized way of describing the parameters of a multimedia session, including IPs, ports, codecs and the bandwidth requirements for each media stream. In this blog post, we’ll discuss the SDP bandwidth attributes and how they are used to specify the bandwidth requirements for media streams.

SDP Bandwidth Attributes

The SDP includes an optional bandwidth attribute with the following syntax:

b=<modifier>:<bandwidth-value>

is a single alphanumeric word giving the meaning of the bandwidth figure, and where the default units for are kilobits per second.

SDP defines three bandwidth attributes that can be used to specify the bandwidth requirements for a media stream. These attributes are:

“b=AS”

The “b=AS” attribute for Application Specific (AS) which may be used to specify the total bandwidth for a single media stream from one site (source) in kilobits per second (kbps). This attribute indicates the maximum amount of bandwidth that the endpoint is willing to allocate for the stream. For example, if the “b=AS” attribute is set to 256 kbps, the endpoint will allocate at least 256 kbps for the stream.

“b=RS”

The “b=RS” attribute indicates the RTCP bandwidth allocated to active data senders. For example, if the “b=RS” attribute is set to 800 bps, the endpoint will reserve at least 800 bps for the RTCP stream to ensure a certain level of QoS, which is used for reporting and monitoring purposes in a multimedia session.

“b=RR”

The “b=RR” attribute or Receiver Report (RR) indicates the RTCP bandwidth allocated to other participants in the RTP session (i.e.receivers). For example, if the “b=RR” attribute is set to 2000 bps, the endpoint will allocate up to 2000 bps for the RTCP stream for receivers.

RS and RR attributes are used to ensure that RTCP traffic does not consume more bandwidth than necessary.

For example:

m=audio 50000 RTP/AVP 0
a=rtpmap:0 PCMU/8000
b=AS:80
b=RS:1000
b=RR:2400

In this example, “b=AS:80” would indicate a maximum bitrate of 80 kbps for the media stream. The “b=RS:1000” attribute is specified at the media level for an audio stream. It indicates that the application expects to receive no more than an average of 1000 bytes per second of RTCP traffic for the duration of the session. “b=RR:2400” would indicate that the endpoint is willing to receive a maximum of 2400 bytes per second for the RTCP traffic.

Using SDP Bandwidth Attributes

SDP bandwidth attributes are typically used in session negotiation between two endpoints to specify the bandwidth requirements for media streams. For example, when two endpoints negotiate a session for a video conference, they can use SDP bandwidth attributes to negotiate the bandwidth requirements for the video stream.

The endpoint that initiates the session typically includes SDP bandwidth attributes in the session description to specify its bandwidth requirements for the media streams. The other endpoint can then use these attributes to determine whether it can meet the bandwidth requirements and negotiate the session accordingly. In IMS networks, as QoS is important, usually mobile handsets or UEs put the “b” params in the SIP messages like INVIITE, 183, and 200 whhich includes SDP.

If the “b=RS:” field or “b=RR:” field or both these fields are not included in a received SDP (offer or answer), then the UE must use the recommended default value for the missing field(s) as defined in RFC 3556[1].

But if the UE receives an SDP offer that contains “b=RS” attribute set to zero, then the UE must set the “b=RS” attribute to zero in an SDP answer to that SDP offer. If the UE receives an SDP offer that contains “b=RR” attribute set to zero, then the UE must set the “b=RR” attribute to zero in an SDP answer to that SDP offer. If the UE receives an SDP offer that contains both “b=RR” and “b=RS” attributes set to zero, then the UE must not send RTCP packets and must consider RTCP to be disabled for the session.

It is important to know that we need this bandwidth info for Dedicated Bearer setup that I will cover in another post soon.

SDP Bandwidth Attributes and RTPEngine

If you are using the RTPEngine as RTP proxy server, I should tell you that it doesn’t care about “b=” atributes and you have to add or remove them manually. You should know that if you transcode, for example, from AMR to PCMU, and do not handle the bandwidth parameters manually, as RTPEngine does not handle “b=” in SDP answers, the receiver receives these parameters, and if applied, voice quality would suffer.

For example, UE sends SDP “b=AS:50” and with AMR codec, but the receiver supports only PCMU, so RTPEngine does transcoding for us, then receiver gets PCMU codec with “b=AS:50”, and if it applies the bandwidth, it assigns 50 Kbps to PCMU stream that is not enough!

Conclusion

SDP bandwidth attributes provide a standardized way of specifying the bandwidth requirements for multimedia sessions. These attributes can be used to ensure a certain level of QoS for media streams and to negotiate sessions between two endpoints. By understanding how SDP bandwidth attributes work, you can ensure that your multimedia sessions meet your bandwidth requirements and provide a high level of quality of service.

Usefule Links:

[1] RFC 4556 - https://datatracker.ietf.org/doc/html/rfc3556

[2] GSMA IR92 - https://www.gsma.com/newsroom/resources/ir-92-ims-profile-for-voice-and-sms-17-0/

SIP and UDP Fragmentation

hyavari26@gmail.com (Hossein Yavari) — Fri, 17 Mar 2023 12:10:35 -0700

MTU (Maximum Transmission Unit) is a critical networking parameter that defines the maximum size of a network packet that can be transmitted over a network interface. In general, larger MTUs can result in higher network performance by reducing the number of packets required to transmit a given amount of data. However, when network packets exceed the MTU of the underlying network infrastructure, they must be fragmented into smaller packets before transmission.

This is especially important for UDP (User Datagram Protocol) packets, which do not have built-in fragmentation capabilities. In contrast to TCP (Transmission Control Protocol), which can automatically segment large packets into smaller ones, UDP relies on the network infrastructure to fragment packets if necessary. As a result, it is crucial to carefully manage MTUs when using UDP to avoid packet fragmentation and ensure reliable network performance.

In any VoIP/IMS network, there are some cases for interconnecion to other servers or providers that just accept SIP over UDP! Yeah, I know, It is ridiculous for me as well, but they are more powerful and you should accept when they declare “we can’t enable TCP” ! OK, so I am talking about UDP fragmentation in SIP signaling and some interesting hints that I found.

In VoIP networks, usually SIP signaling messages don’t have much headers and supported codecs by UAC or UAS are limited, that means the packet size usually is not big enough to exceed MTU! but in IMS network situation is different, there are different X-CSCF or other elements in the path, and each of them like to say we are doing very amazing task, so they inject a few headers. Furthermore, AMR, EVS, and other codecs have various attributes that make the SIP INVITE message larger.

I said larger, but larger than what? In the RFC 3261 section 18.1.1 it says SIP over UDP message size should be lower than 1300 bytes and if you are carrying larger than this, you have to use TCP.

In short, in IMS, this usually occurs when SIP messages have to be sent over UDP; these messages are bigger than 1300 bytes and would need to be fragmented if the MTU is 1500 or smaller than message size.

In Wireshark, how can you see fragmentation?

For example, when you are sending a SIP INVITE to other server over UDP, when it exceeds the MTU, you will receive an ICMP message Destination unreachable (Fragmentation needed) that inside it you can find the expected MTU size. Following that, the server fragments the messages and forwards them.

In SIP signaling, re-transmission mechanism helps to sending fragmented message to destination. This was the mechanism that we didn’t have in our Lambda functions but fragmentation forced us to implement it quickly.

So in next try you can see fragmentation in Wireshark like this:

Okay, so we do fragmentation, but other ways can be used to prevent fragmentation and consequent retransmission. First we can adjust correct MTU for the network interface that is linked to the destination UDP servers! If your MTU is higher than your adjacent link MTU, and you are doing fragmentation for all requests, so why you need that?? decrease it or set it to value that fit for your next hop.

You can check the MTU and the set it easily:

ifconfig <Interface_name> mtu <mtu_size> up

[ec2-user@i-0698wer63bd635 ~]$ ifconfig | grep mtu
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1436
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

[ec2-user@i-0698wer63bd635 ~]$ ifconfig eth1 mtu 1500 up

[ec2-user@i-0698wer63bd635 ~]$ ifconfig | grep mtu
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1436
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

To persist your network MTU setting for Amazon Linux 2, add the following line to the /etc/sysconfig/network-scripts/ifcfg-eth0 file:

MTU=1500

change the value to whatever you want. For other OS, better to google.

Bad news: if you are using the ECS Fargate, you can NOT change the MTU size. Default value is 9001 (Jumbo frames)!

In our case, as we are using Fargate, so I tried to explore other solutions and found that Linux does keep a history of the learned MTU sizes for each network path, which is referred to as the “Path MTU Discovery (PMTUD)” mechanism. This is done to avoid repeated fragmentation of packets and improve network performance.

When fragmentation is needed, the sender reduces the packet size and retransmits it until it reaches the destination without fragmentation. During this process, the sender also updates its PMTUD cache to remember the MTU of the path. However, this PMTUD cache is not permanent and can expire after a certain period of time, usually around 10 minutes. If you send another packet with a size larger than the current path MTU after the cache has expired, the process will start over and the router will send an ICMP message again, causing the packet to be dropped.

To check the current PMTUD cache status on Linux, you can use the ip route show command and look for entries with the “mtu” field.

1
2
3

bash-5.2# ip route get to 75.100.127.113
75.100.127.113 via 100.120.111.97 dev eth1 src 100.120.211.105 uid 0
    cache expires 582sec mtu 1500

Ok, good. I tried to create a route to destination with specific MTU. Oh no you need privileged mode in container to do that and this is not possible in Fargate.

The cache expiration time can be configured using the net.ipv4.mtu_expires. Default value is 600 seconds.

bash# sudo sysctl -w net.ipv4.mtu_expires=1200

bash# cat /proc/sys/net/ipv4/mtu_expires
1200

When you run a container, it shares the host kernel, which manages the networking stack and related parameters such as MTU caching. So, if you need to change the net.ipv4.mtu_expires parameter for a specific application running in a container, you may be able to modify the application’s configuration or code to adjust its behavior. However, if you need to change the parameter globally for all applications running in a container, you will need to modify the host operating system’s configuration instead. In short, forget this option for ECS too!

In conclusion, try to use TCP in SIP network and if you can’t afford it, try to adjust your MTU, otherwise you should pay the fragmentation cost!

Usefule Links:

[1] How to change MTU size in Linux - https://linuxhint.com/how-to-change-mtu-size-in-linux/

[2] Fargate task networking - https://docs.aws.amazon.com/AmazonECS/latest/userguide/fargate-task-networking.html#fargate-task-networking-considerations

[3] Network maximum transmission unit (MTU) for your EC2 instance - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html

[4] Docker run –sysctl - https://docs.docker.com/engine/reference/commandline/run/#sysctl

IMS SMS and Wireshark

hyavari26@gmail.com (Hossein Yavari) — Sat, 04 Mar 2023 01:10:14 -0800

Objects in (the) Wireshark are not same as they appear

It is usual in the software teams, without regular code review sessions, that someone leaves the group and, over time that other services are ready to test, something doesn’t work after deployment.

This scenario happened to me last week, when I deployed IMS SMS services in a lab environment and everything was ready for the test with a real mobile handset, I noticed something is not correct.

But what was the issue? When UE A after registration to VoLTE, sent an SMS to UE B, just the first message has delivered without issue, and later messages were not sent. Sometimes two messages were sent, and others were not.

I check the service Lambdas’ logs and found some SIP 400 response messages from sender UE. In these cases, the best option for troubleshooting is network capturing and checking in Wireshark to see what exactly going on. If you have reference captures (from other production networks), you would be lucky because you can check with reference capture.

In the capture everything was good, I checked all the MESSAGE requests and 202 Accepted responses and all headers were correct.

And the second SMS, made this error, when we sent delivery report (RP-ACK) to the sender:

Method mismatch btw Request Line and CSeq didn’t have any meaning to me. Method and CSeq were the same. I checked again other headers in the delivery report SIP Message and they looked good.

Ok, so I started to review the function codes, especially the function that made the delivery report. Ah! I found it. Someone was concatenating the RP-ACK payload buffer as a body to the SIP Message string!! Something like this:

const deliveryReportBuff = Buffer.from(deliveryReport, "hex");

...
  response += `Content-Length: ${deliveryReportBuff.length}\r\n`;
  response += "\r\n";
  response += deliveryReportBuff;

Obviously, it won’t work! or it won’t properly for all messages. But what caused which I to spend more time finding the issue? Wireshark! Wireshark could show the RP-ACK payload and without checking the hex string of the SIP body and checking the body length, it wasn’t easy to see the problem. So I create a buffer from message string and concatinating with payload. So now the length and content of message is correct and SMS over IP works well!

Conclusion: If you see that Wireshark can parse the packets, it doesn’t mean that your packets have been created correctly. You need to check the Hex string and be sure about the exact content.

What Is IaSQL

hyavari26@gmail.com (Hossein Yavari) — Thu, 23 Feb 2023 00:19:01 -0800

I discovered an interesting open-source project today that has an entirely different approach from mine. Honestly, I couldn’t get along with it yet, but it might be in the near future I use it.

Do you have any ideas about creating your own AWS infrastructure, such as EC2s, ECS or etc. via PostgreSQL queries? This is the main idea of this open-source project: IaSQL.

IaSQL is a powerful tool that allows users to deploy infrastructure in AWS using PostgreSQL SQL queries. With IaSQL, users can define their infrastructure as code and automate the deployment process, making it easier to manage and scale their AWS infrastructure. In contrast to AWS consoles and infrastructure as code (IaC) tools, IaSQL lets developers manage cloud infrastructure as data in PostgreSQL.

To deploy infrastructure using IaSQL, users define their infrastructure as code using SQL queries. These queries are executed by IaSQL, which then deploys the defined infrastructure in AWS.

Here is an example of how to use IaSQL to deploy an EC2 instance in AWS:

-- Define EC2 instance parameters
SET ec2_instance_type = 't2.micro';
SET ec2_key_name = 'my-key';
SET ec2_security_group_id = 'sg-0123456789abcdde';

-- Create EC2 instance
CREATE RESOURCE ec2_instance (
  ec2_instance_type text NOT NULL DEFAULT ec2_instance_type,
  ec2_key_name text NOT NULL DEFAULT ec2_key_name,
  ec2_security_group_ids text[] NOT NULL DEFAULT ARRAY[ec2_security_group_id]
) AS
  SELECT aws_ec2.create_instance(
    ec2_instance_type,
    ec2_key_name,
    ec2_security_group_ids
  ) AS instance_id;

This SQL query defines the parameters for an EC2 instance and then creates the instance using the aws_ec2.create_instance() function. The resulting EC2 instance ID is then stored in the ec2_instance resource.

Using IaSQL, users can define and deploy a wide range of AWS resources using SQL queries. IaSQL supports many AWS services, including EC2, RDS, S3, and many others. One of the key benefits of using IaSQL is that it allows users to define their infrastructure as code, which can be version controlled and easily shared with other team members. This approach also allows for easier collaboration, as changes to the infrastructure can be reviewed and tested before being deployed.

I will try to do more exploration of the project and write about it again. You can take a look at project blog posts for more hints.

IP Public on AWS ECS Fargate

hyavari26@gmail.com (Hossein Yavari) — Fri, 17 Feb 2023 09:50:00 -0800

There are many cases that might you need to have direct IP public address on your ECS task. When your tasks are on EC2, situation is a little easier, but if you are on Fargate, you need to do some tricks to get a reachable IP public.

Before checking the script, don’t forget to consider these facts:

For getting a reachable public IP on task, you need to run the cluster on public subnet.
If you are in a subnet that is connected to NAT gateway, your task would not be reachable.
If you need to just get your NAT IP, you can take it easy and just run commands like this in your entry point scripts: NAT_PUBLIC_IP=$(curl checkip.amazonaws.com)
If your app inside container, doesn’t need to know its public IP, so you are good, you don’t need to any special thing. Only enable Auto-assign public IP in your cluster service settings or in CDK.

In my case, that container is running a SIP application server, it needs to know its public interface to use in outbound SIP messages. So, I added below scripts as my container entry point, and then use the public IP value in app settings and the starting up the process.

#!/bin/bash

# Install the ca-certificates package
apt-get update
apt-get install -y ca-certificates

# Update the root certificates
update-ca-certificates --fresh

# Install the AWS CLI version 2 using apt-get
apt-get install -y unzip curl less jq
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install

task_metadata=$(curl -s $ECS_CONTAINER_METADATA_URI_V4/task)
cluster_name=$(echo $task_metadata | jq -r '.TaskARN' | cut -d/ -f2)
echo $cluster_name

task_id=$(echo $task_metadata | jq -r '.TaskARN' | cut -d/ -f3)
echo $task_id

eni_id=$(aws ecs describe-tasks --cluster $cluster_name --tasks $task_id --query 'tasks[0].attachments[0].details[?name==`networkInterfaceId`].value' --output text)
public_ip=$(aws ec2 describe-network-interfaces --network-interface-ids "$eni_id" --query 'NetworkInterfaces[0].Association.PublicIp' --output text)

echo "PUBLIC_IP: $public_ip"

if you installed some of the packages in building image, so you don’t need install again. Just install aws-cli. Then we get task eni-id and with ec2 describe-network-interfaces we get IP public address that is assigned by AWS to task ENI.

Don’t forget that you need the specific role for running ec2 describe-network-interfaces. Then add below in your task IAM role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeNetworkInterfaces"
            ],
            "Resource": "*"
        }
    ]
}

AWS EC2 gai.conf

hyavari26@gmail.com (Hossein Yavari) — Sat, 04 Feb 2023 00:57:05 -0800

gai.conf is a configuration file in Linux systems that controls the behavior of the getaddrinfo library function. This function is used to translate hostnames or IP addresses into socket addresses that are usable by various network protocols. By configuring gai.conf, you can specify the priority and order in which different address families (such as IPv4 and IPv6) are returned by the getaddrinfo function.

The gai.conf file is located in the /etc directory and has a simple syntax. Each line in the file specifies a preference value for a specific address family or an address prefix. The higher the preference value, the more preferred the address family is.

For example, to prefer IPv4 over IPv6, you can add the following line to gai.conf:

`1`	`precedence ::ffff:0:0/96 100`

Yesterday I had an issue in deploying my codes on AWS EC2 instances, so when I investigate the CodeDeploy agent logs, I saw that it can not connect !! After checking different things, I noticed that my ECS subnet is private and just its IPv4 has route to NAT gateway. But my EC2 was trying to connect to CodeDeploy via IPv6 that was not routable.

So I added gai.conf as below in /etc and rebooted the EC2 instances and deployed the code without issue.

abel  ::/0          1
label  2002::/16     2
label ::/96          3
label ::ffff:0:0/96  4
precedence  ::1/128       50
precedence  ::/0          40
precedence  2002::/16     30
precedence ::/96          20
precedence ::ffff:0:0/96  100

The label lines are defining labels for various address prefixes. For example, label ::1/128 1 is defining a label for the loopback address (::1) with a label value of 1.

The precedence lines are setting the precedence values for the various address prefixes. The higher the precedence value, the more preferred the address family is. For example, precedence ::ffff:0:0/96 100 is giving IPv4 addresses a higher priority than any other address family, as the precedence value of 100 is the highest.

So, the priority and order of different address families for the getaddrinfo library function would be as follows:

IPv4 (::ffff:0:0/96) with a precedence value of 100

IPv6 with a prefix of 2002::/16 with a precedence value of 30

IPv6 with a prefix of ::/96 with a precedence value of 20

IPv6 with a prefix of ::/0 with a precedence value of 40

IPv6 loopback address (::1/128) with a precedence value of 50

This configuration is giving IPv4 addresses the highest priority, followed by the other address families in the order specified.

How Run SIPp on AWS EC2

hyavari26@gmail.com (Hossein Yavari) — Tue, 31 Jan 2023 18:20:58 -0800

SIPp is a free, open-source tool for testing Session Initiation Protocol (SIP) communication systems. It was created to simulate thousands of simultaneous incoming and outgoing SIP calls, allowing users to stress test and measure the performance of their SIP infrastructure.

SIPp is a highly flexible tool, and its scripting language supports various SIP message types and parameters, allowing you to test the behavior of your SIP infrastructure in detail. It also supports SIP over Transport Layer Security (TLS) and Secure Real-time Transport Protocol (SRTP) for secure communication.

I have been using SIPp for a while and last week when I was testing a SIP trunk to operator NG911 SBC, I needed to install it on an AWS EC2 instance. Installation is easy, just need to compile and install cmake manually because cmake version on the online repo is old and SIPp doesn’t like that.

So you need to follow these steps:

yum groupinstall "Development Tools" -y
yum install ncurses-compat-libs ncurses-devel wget openssl-devel pcap libpcap-devel libnet gsl -y
wget https://github.com/Kitware/CMake/releases/download/v3.25.2/cmake-3.25.2.tar.gz
tar -xvzf cmake-3.25.2.tar.gz
cd cmake-3.25.2
./bootstrap
make
make install
wget https://github.com/SIPp/sipp/releases/download/v3.6.1/sipp-3.6.1.tar.gz
tar -xvzf sipp-3.6.1.tar.gz
cd sipp-3.6.1
/usr/local/bin/cmake . -DUSE_PCAP=1 -DUSE_GSL=1
make

Then you can run the ./sipp or link it to /usr/sbin

** You can compile sipp with ssl and sctp too:

`1`	`/usr/local/bin/cmake . -DUSE_SSL=1 -DUSE_SCTP=1 -DUSE_PCAP=1 -DUSE_GSL=1`

The link that you need to check based on time:

Cmake download page: https://cmake.org/download/ Latest stable SIPp version: https://github.com/SIPp/sipp/releases/tag/v3.6.1

Now, you need to create your scenario. So, I created a ng911option.xml file with this xml:

<?xml version="1.0" encoding="UTF-8" ?>
<scenario name="Option scenario">
 <send>
  <![CDATA[  
	OPTIONS sip:op.ng911.xxx.ca:5060 SIP/2.0
	Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=z9hG4bKd3eee3f9487fc430b[call_number]
	From: <sip:ims-ng911@10.16.10.40>;tag=cb89092c-c504-45bd-b256-eb49e2bdfd1g[call_number]
	To: <sip:op.ng911.xxx.ca>
	Contact: <sip:ims-lrf@[local_ip]:[local_port]>
	Call-ID: [call_id]
	CSeq: 11667[call_number] OPTIONS
	Max-Forwards: 70
	User-Agent: IMS Core Test
	Allow: INVITE, ACK, OPTIONS, CANCEL, BYE, UPDATE, INFO, REFER, NOTIFY, MESSAGE, PRACK
	Content-Length: [len]
  ]]>
 </send>

And run this scenario with this command:

`1`	`./sipp -sf ng911option.xml -i 10.16.10.40 -p 5060 100.125.250.232:5060 -l 1 -m 1000 -r 1 -rp 30000`

Then SIPp starts to send SIP OPTIONS every 30 seconds to the target IP (100.125.250.232) from the EC2 interface (10.16.10.40).

Fortunately, I got the 200(OK) which means the connection and IP/port/protocol are set correctly and we have time enough to start the actual server later :)

Book Review: Distributed Systems With Node.js

hyavari26@gmail.com (Hossein Yavari) — Tue, 24 Jan 2023 13:33:50 -0800

Titel: “Distributed Systems With Node.js”

Edition: 2020

Author: Thomas Hunter II

Link: OREILLY Website

The book in my list that I was interested in reviewing was this one, which I thought might have some useful tips for my work.

This book, “Distributed Systems with Node.js” by Thomas Hunter, focuses on teaching readers how to use Node.js for building backend services for distributed systems, rather than frontend web development. The book covers topics such as deploying and scaling Node.js applications, making them redundant and resilient to failure, and observing the health of the application. By the end of the book, readers will have a solid understanding of the technologies needed to run Node.js services in a production environment. However, it’s important to note that the book is not intended to make readers experts on these systems and it doesn’t cover all the technology used in different employer’s solutions. But it will help readers to understand how to run a local instances.

Chapters one and two were interesting to me since they referred to some of my old knowledge. I can recommand this book to developers that might not have enough skills in distributed systems deployment and DevOps areas.

Remote SIM Provisioning (RSP)

hyavari26@gmail.com (Hossein Yavari) — Sat, 21 Jan 2023 10:27:24 -0800

Remote SIM Provisioning (RSP) is a method of managing and provisioning SIM cards remotely without the need for physical interaction. This enables mobile operators to remotely activate and configure SIM cards, and manage subscriber data such as phone numbers, services, and preferences.

OTA updates, also known as over-the-air updates, are software updates that are delivered to devices wirelessly, rather than through a wired connection. OTA updates can be used to update the firmware or software of a device, add new features, and fix bugs.

When it comes to RSP, it allows for the remote management of the SIM card’s credentials, the SIM card can be provisioned and activated remotely and it also allows for the management of subscriber data such as phone numbers, services, and preferences. This is accomplished by sending an SMS message or a USSD message over the network to the SIM card, which then triggers a response from the SIM card to update or provision the subscriber data.

When it comes to OTA updates, they play an important role in the management of RSP-enabled devices, as they enable mobile operators to remotely update and maintain the software on the devices. This is accomplished by sending an update package over the air to the device, which then installs the update. OTA updates can be used to add new features, fix bugs, and improve the overall performance of the device.

Remote SIM Provisioning (RSP) offers several benefits for mobile operators, including:

Cost savings: RSP eliminates the need for physical SIM cards to be distributed and activated, reducing the costs associated with SIM card production, logistics, and distribution.

Increased efficiency: RSP enables mobile operators to quickly and easily activate and configure SIM cards, and manage subscriber data, reducing the time and resources required for manual SIM card provisioning.

Improved customer experience: RSP allows for faster activation and provisioning of SIM cards, resulting in a better customer experience and reduced customer churn.

Flexibility: RSP enables mobile operators to remotely manage and provision SIM cards, regardless of the location of the subscriber or the device.

Scalability: RSP enables mobile operators to easily manage and provision large numbers of SIM cards, making it easier to scale their operations as their subscriber base grows.

Security: RSP enables mobile operators to secure the SIM card credentials and subscriber data and also to update the SIM card’s security features remotely.

Remote management: RSP enables mobile operators to remotely manage and provision SIM cards and subscriber data, regardless of the location of the subscriber or the device. This is particularly useful for IoT and M2M devices, which can be located in remote areas or hard-to-reach locations.

Remote troubleshooting: RSP enables mobile operators to remotely troubleshoot and diagnose issues with SIM cards and devices, reducing the need for on-site visits and improving the overall efficiency of customer support.

In summary, Remote SIM Provisioning (RSP) enables mobile operators to remotely activate and configure SIM cards, and manage subscriber data such as phone numbers, services, and preferences. OTA updates, also known as over-the-air updates, are software updates that are delivered to devices wirelessly, rather than through a wired connection and are important for the management of RSP-enabled devices as they enable mobile operators to remotely update and maintain the software on the devices.

More resources:

[1] GSMA eSIM

[2] eSIM Deep Dive

OpenRAN on AWS

hyavari26@gmail.com (Hossein Yavari) — Sat, 21 Jan 2023 00:23:43 -0800

Earlier this week, I read an interesting whitepaper from AWS about how public cloud providers are moving to 5G and optimizing their services to gain more market share. AWS in this whitepaper covers OpenRAN as one of these technologies, providing useful hints about new services around 5G SA networks. This whitepaper is a technical guide that describes the architecture and best practices for deploying OpenRAN solutions on the AWS cloud. The paper begins by providing an overview of OpenRAN and its key components, including the vRAN (virtual RAN) and CU (central unit), as well as the O-RAN (OpenRAN) Alliance and its goals.

This diagram shows O-RAN architecture:

The whitepaper then goes on to discuss the benefits of running OpenRAN on AWS, including the ability to take advantage of the elastic and flexible nature of the cloud to provide cellular services with low latency and high reliability, as well as the ability to reduce the total cost of ownership and accelerate time-to-market.

The paper also provides a detailed architectural overview of a sample OpenRAN deployment on AWS, including the various AWS services that can be used to deploy and operate OpenRAN solutions, such as Amazon Elastic Compute Cloud (EC2), Amazon Simple Storage Service (S3), and Amazon Virtual Private Cloud (VPC). It also covers the different network topologies and deployment options available on AWS.

Finally, it includes best practices for designing and operating OpenRAN solutions on AWS, covering topics such as security, scaling, and monitoring.

Overall, this document provides a comprehensive overview of how to deploy OpenRAN solutions on the AWS cloud and provides valuable information for network operators who are looking to take advantage of the benefits of OpenRAN and the AWS cloud.

You can find this whitepaper here: Open Radio Access Network Architecture on AWS

Also you can get more info on O-RAN alliance

Docker Compose With Podman

hyavari26@gmail.com (Hossein Yavari) — Sat, 14 Jan 2023 20:49:05 -0800

Podman is a container engine that enables users to manage OCI-compliant containers on Linux systems without the need for a daemon. It is popular among users as an alternative to Docker due to its ease of adoption. However, the lack of support for Docker Compose, a command-line tool that coordinates multiple Docker containers for local development, has been a major concern for many users and the container community. Podman starts to support docker-compose from version 3.0 but changes in docker-compose from version 2.0 made some incompatibilities, so a community project called Podman Compose has been born. Podman Compose processes the Compose spec and translates it into Podman CLI commands [1].

There are many articles that you can read about Podman vs Docker, but here I am going to share an experience to run Docker-compose with Podman. In the telecom field, many operators are using Red Had Linux (rhel 8 or 9) and Red Hat uses Podman because it is designed to be a more secure and efficient alternative to Docker.

Podman runs without the need for root privileges, which reduces the risk of security vulnerabilities caused by running containers with elevated permissions. Also, Podman is natively integrated with Red Hat’s container orchestration platform OpenShift, which makes it easier to deploy and manage containerized applications at scale.

Overall, Podman’s security, performance, and integration features make it a good fit for Red Hat’s container orchestration platform and their enterprise customers who are looking for a more secure and efficient solution to run containers. So, what if you need to use Docker-compose and re-use your YAML??

To use Podman with Docker-compose, I did some tricks. So far, I have not encountered any issues, but I will update if something goes wrong.

You need to follow below steps:

These are the version of OS and Podman which I used:

root@rhel:~# uname -a
Linux rhel 5.14.0-162.6.1.el9_1.x86_64

root@rhel:~# podman -v
podman version 4.2.0

Then I run:

root@rhel:~# yum update -y

root@rhel:~# yum install podman-docker -y

root@rhel:~# curl -L https://github.com/docker/compose/releases/download/v2.3.3/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose

root@rhel:~# chmod 755 /usr/local/bin/docker-compose

root@rhel:~# docker-compose -v
Docker Compose version v2.3.3

root@rhel:~# touch /var/run/docker.sock
root@rhel:~# mount --bind /var/run/podman/podman.sock /var/run/docker.sock

I tried with different version of Docker-compose, but among versions > 2, v2.3.3 seems ok with Podman.

[1] https://www.redhat.com/sysadmin/podman-compose-docker-compose

Book Review: The Kubernetes Book

hyavari26@gmail.com (Hossein Yavari) — Fri, 30 Dec 2022 01:43:29 -0800

Titel: “The Kubernetes Book”

Edition: 2022

Author: Nigel Polton

Link: Nigel Polton Website

Many people who tried to learn more about Kubernetes or Docker have heard the name Nigel Polton. He is one of the old guys or captains in this domain and prepared many Kubernetes and docker training videos, and tutorials besides the books.

During the Christmas holidays, I was interested in refreshing my knowledge of Kubernetes, so I thought reviewing Nigel’s book, could be helpful. I have his docker book and I try to find time and review that too in these days.

The book has 16 chapters and starts with good basic information about concepts like microservices and cloud-native apps. Then it talks about Kubernetes’s history. Something that was interesting or at least I didn’t remember whether I read about it before or not, was Borg and Omega at Google and how Kubernetes came out. In addition, it describes K8s as a cloud operating system (OS) and abstraction which is provided.

In the second chapter, we learn more about major Kubernetes cluster components such as control nodes, worker nodes, pods, deployment, and services. As it takes you through the control components, it describes them so that you can easily understand them.

Chapter three covers ways to get Kubernetes cluster up and running on your lab machine and review hosted services like GKE. Then, you will have the pilot K8s cluster for rest of the chapters.

Chapter 4 goes into Pod details. We learn why K8s works with Pods and not containers directly. Then you will learn about Pod manifest and its specs. Nigel suggest you watching “Inception” movie to understand relation of Pods and containers better. I liked the multi-container descriptions in this chapter.

In next chapter, we learn how we can partition single cluster to multi virtual cluster using Namespaces. Among the things I like about this book is that it doesn’t waste your time and delivers what you need to know.

Deployments and Services are main topics for chapter six and seven. How ReplicaSet is different to Deployment is important that you should know.

Ingress and web application access through single LoadBalancer is topic of chapter 8. We learn what issue Insgress is resolving. In the next chapter, the book dives deeper into Service Discovery and how it’s implemented in K8s. You will not be left in the dark about service discovery and DNS stuff in Kubernetes after reading this chapter.

In this state you have learned about the basic things around Kubernetes, so in the following chapters, book focuses on more advance topics. K8s storage is covered in chapter 10 and you will learn about Kubernetes persistent volume subsystem and CSI.

ConfigMaps and Kubernetes Secrets have been covered in chapter 11 and I am sure that we can find lots of use cases in real projects. Next four chapters are dedicated to StatefulSets, RBAC, API and K8s Security and you will get good info but I think this topics need more resources fo reading and practicing.

To wrap up, this book was a great option for refreshing my knowledge and reviewing key Kubernetes concepts and I recommend it.

Hep Protocol and Homer

hyavari26@gmail.com (Hossein Yavari) — Wed, 21 Dec 2022 07:57:11 -0800

Capturing telecom protocols or traffic for various purposes is a topic that has been discussed for a long time and always has been done with different tools or through some vendors.

One of the tools that help to capture protocols, especially SIP, is Homer. This project provides valuable tools to capture and monitor your VoIP network and use it for troubleshooting or analytics purposes.

Homer’s idea is simple, you have some agents to capture the required data, then encapsulate it in the standard format and send it to the server. You can find the below image in project git:

If you are looking for a ready solution, Homer has it and there are different agents for open-source platforms like Kamailio, FreeSWITCH and etc. But if you are looking to use this platform for your application, first of all, you need to understand HEP protocol (actually it is not RFC protocol but there is an RFC draft for that). You can read HEP protocol specification here: HEP V3

Recently for one of my project, needed to send SIP protocol message through HEP to another consumer that uses them for analytics. So, I tried to use HEP-js lib. It is a simple module that encapsulates your protocol message and other info in HEP PDU and then decapsulates HEP PDU to JSON. Somethin like this:

{
  "rcinfo": {
    "protocolFamily": 2,
    "protocol": 6,
    "srcIp": "172.26.26.72",
    "dstIp": "172.26.21.185",
    "srcPort": 64831,
    "dstPort": 8060,
    "timeSeconds": 1592975786,
    "timeUseconds": 669278,
    "payloadType": 1,
    "captureId": 2001,
    "hepNodeName": "2001",
    "conversationId": "456852465",
    "organizationId": "3bac7742-243f-4af7-ba39-f4098b941eda",
  },
  "payload": "INVITE sip:BellUser2@172.26.21.185:8060;transport=tls SIP/2.0\r\nTo:  <sip:BellUser2@172.26.21.185:8060>\r\nFrom:  <sip:BellStation1@172.26.21.185:8060>;tag=974329\r\ncall-id: 3935064a-294e-44d8-930d-1a87b90515bb\r\nCSeq: 1 INVITE\r\nallow-events: conference, talk, hold\r\nContact:  <sip:BellStation1@172.26.26.72:5061;transport=tls>\r\nx-phonesim-proxy-type: primary\r\ncontent-type: application/sdp\r\nx-edge-id: 268c720e-b939-4484-966d-80a1123e3810\r\nx-edge-name: qf-bell\r\nx-test-id: Station2Station\r\nx-test-name: Station to Station Keyword Test\r\nUser-Agent: PolycomSoundPointIP-SPIP_450-UA/4.0.10.0689_000025CC0001\r\nx-phonesim: 1.0.0-534\r\ncontent-length: 567\r\nVia: SIP/2.0/TLS qf-lempel:5060;branch=z9hG4bK416647af6e43448b8fc9c8b804713a0e\r\n\r\nv=0\r\no=- 4056025290 3801964586 IN IP4 172.26.26.72\r\ns= \r\nt=0 0\r\na=group:ANAT 1 2\r\nm=audio 20522 RTP/SAVP 0 8 9 101\r\nc=IN IP4 172.26.26.72\r\na=mid:1\r\na=rtpmap:101 telephone-event/8000\r\na=fmtp:101 0-15\r\na=crypto:1 AES_CM_256_HMAC_SHA1_80 inline:ba6DaKfQLSQQbYNMtL1ng2xCVbJuihEgzeajdEWIHT4qGpfrPwuTMDIasyhSOA\r\na=sendrecv\r\nm=audio 23824 RTP/SAVP 0 8 9 101\r\nc=IN IP6 2620:102:c000:f10:d::6050\r\na=mid:2\r\na=rtpmap:101 telephone-event/8000\r\na=fmtp:101 0-15\r\na=crypto:1 AES_CM_256_HMAC_SHA1_80 inline:ba6DaKfQLSQQbYNMtL1ng2xCVbJuihEgzeajdEWIHT4qGpfrPwuTMDIasyhSOA\r\na=sendrecv\r\n"
}

I have added support for IPv6 to this lib that you can find it here.

In this case, you don’t need any Homer server, as you are encapsulating through HEP and other side when receives the PDU, only extract payload and meta data information and digest it for other purposes. HEP gives you a standard structure that will be helpful for future requirements too. Attention please, you can send HEP PDU over any transport you like, in my case I am sending through HTTPS to AWS API gateway and behind that a Lambda decapsulate it and do other processes.

SIPREC with RTPEngine

hyavari26@gmail.com (Hossein Yavari) — Fri, 25 Nov 2022 23:02:39 -0800

RTPEngine is one of the best options among open-source solutions for RTP or media handling and it besides other platforms like Kamailio will give you many features or ready-to-go solutions.

When you are handling media sessions in IMS or any VoIP network, you should have a solution for recording, media barging or let’s say Lowful Interception (LI). LI in IMS has a long story and it is out of scope; for this post, we focus on call recording.

First of all, we need to what is SIPREC? I shared a ppt here that gives you overal info you need:

SIPREC RTPEngine Media Forking from Hossein Yavari

The ppt gives a good overview of what SIPREC is and how Kamailio and RTPEngine fit into this picture. But, what if you don’t use the Kamailio? How you can use RTPEngine features to implement media recording?

RTPEngine media forking feature let you fork active sessions to another destination that in our case we call SRS. The call recording server or SRS can send a command to RTPEngine through NG protocol. So, might it not be a SIP-aware server, but it should be able to handle RTP protocol and offer an SDP.

Media forking happens through the subscribe command, e.g. SRS sends a subscribe request command to RTPEngine and subscribes itself to the target call. You can see a sample payload below:

{
   "command": "subscribe request",
   "call-id": "call xyz",
   "from-tags": [
      "test"
   ],
   "transport protocol": "RTP/AVP"
}

When RTPEngine receives the command, it responds with SDP that includes a=sendonly attribute:

{
   "result": "ok",
   "sdp": "v=0
         o=- ...
         s=rtpengine-11-2-0-0-0
         t=0 0
         m=audio 30160 RTP/AVP 96
         c=IN IP4 192.168.1.120
         a=rtpmap:96 opus/48000
         a=sendonly
         a=rtcp:30160
         m=audio 30172 RTP/AVP 96
         c=IN IP4 192.168.1.120
         a=rtpmap:96 opus/48000
         a=sendonly
         a=rtcp:30170",
   "from-tags": [
   "test"
   ],
   "to-tag": "9e2a2a7fc4d6..."
}

In the next step, SRS should send its SDP to RTPEngine through subscribe answer with a=recvonly:

{
   "command": "subscribe answer",
   "call-id": "call xyz",
   "to-tag": "9e2a2a7fc4d6...",
   "sdp": "v=0
            o=- 2345 2345 IN IP4 192.168.1.95
            s=-
            t=0 0
            m=audio 33345 RTP/AVP 96
            c=IN IP4 192.168.1.95
            a=rtpmap:96 opus/48000
            a=recvonly
            m=audio 33366 RTP/AVP 96
            c=IN IP4 192.168.1.95
            a=rtpmap:96 opus/48000
            a=recvonly"
}

After this SRS will receive the forked media RTP packets and then it can save it to file or prepare it for LI interfaces. Don’t forget that transcoding also is possible for forked media, when you ask it in subscribe answer SDP, very useful!

How Decrypt ESP Packets in Wireshark

hyavari26@gmail.com (Hossein Yavari) — Fri, 25 Nov 2022 11:53:22 -0800

If you are working in the telecom field, I think it is rare that you don’t work with Wireshark to capture the network or open the tcpdump outputs. Although, as a developer, you need to use the reference packet captures, to see the correct packet flow and structure for a specific protocol while you are implementing (if you couldn’t find a reference pcap, you will be walking in the darkness!)

In the VoLTE network or IMS, subscribers use Internet Protocol Security (IPSec) tunnels to connect to the IMS core. In another word, the Gm interface between subscribers (UEs) and the P-CSCF server is secured with IPSec. I will go through IPSec details in another post, but for now, if you have an IPSec capture, you can see a bunch of Encapsulating Security Payload (ESP) packets that their content is not visible.

IPSec can be used for integrity or integrity and ciphering; in the first mode, there isn’t any encryption on ESP and you can decode it easily. For this purpose just open preferences settings in Wireshark and for ESP protocol, check the first setting:

Edit -> Preferences -> Protocol -> ESP

If subscribers or operator forces for ciphering, ESP packets are encrypted and you need more info to decrypt them. In this case, you need to follow below steps:

Go to same settings: Edit -> Preferences -> Protocol -> ESP
Select 3 checkboxes (not the first one!) and select ‘Edit’ next to ESP SAs:

Create an entry for the each IPSec SAs and add the information obtained from the P-CSCF for IPSec tunnel:

Protocol: IPv6 or IPv4
Src IP and Dst IP: UE IP and P-CSCF address
SPI: eg. 0x7bfa58a1
Encryption: AES-CBC [RFC3602] (or whatever is selected for IPSec tunnel)
Encryption Key: 0xCK
Authentication: HMAC-SHA-1-96 [RFC2404] (or whatever is selected for IPSec tunnel)
Authentication Key: 0xIK

CK and IK are ciphering and integration keys, that you can find from tunnel information on P-CSCF or before removing them from SIP 401 response to subscriber REGISTER.

Now, you can see the ESP content:

RTPEngine Internal Insight

hyavari26@gmail.com (Hossein Yavari) — Tue, 22 Nov 2022 10:45:22 -0800

RTPEngine as you can find in the project GitHub too, is a proxy server for RTP and other UDP-based media traffic, and it is used with projects like Kamailio or OpenSIPS as a SIP proxy server. So, the SIP proxy handles SIP signaling, and RTPEngine takes care of the RTP/Media plane.

In this post, I am not going to show you how we can install or configure RTPEngine. I would like to share some ideas that I found from its source code about RTPEngine’s internal design and packet handling flow in it.

But why I tried to understand the flow? First of all, it helps me to contribute new features to it, because all of us as open-source users, should be committed to the projects that we use and its community! Also, sometimes I spend time getting some ideas about project code to debug issues better. About RTPEngine, there was a third reason; I needed to change the RTP packets’ flow based on some requirements.

Note: my findings are just experimental and might make project owners a little shocked :)

How is RTPEngine internal?

RTPEngine project includes various folders but the main sources are under daemon folder. In this folder, you can find the implementation of different project elements or features. In this folder, there are two important files that I found core functions of RTPEngine in them: call.c and media_socket.c. Obviously, other files are important, but if you are following the main RTP flow in RTPEngine, we need to focus on these files. Let’s see how we reach these files.

RTPEngine gets action commands like offer, and answer through ng protocol; it is a control method that let you control the RTPEngine daemon from a SIP proxy or any other entity that controls RTPEngine functionality. You can configure ng port and protocol in RTPEngine configs. So, there is control_ng.c that in this file you can find a function named control_ng_process(...). This function will be called whenever RTPEngine receives a command through ng port. In this function, RTPEngines detects the command and does proper action. For example, if the command is ‘offer’, this function calls call_offer_ng(...) that is located in call_interfaces.c. Later call_offer_ng calls call_offer_answer_ng(...) which I think is an important function that lets RTEngine create required handlers.

So here, we have this flow:

call_offer_answer_ng(...) is the place where we can actually see the RTPEngine packets handling logic creation. This function, create a call entity that is the main parent structure of all call-related objects. Also, it creates call_monologue that exposes a call participant. call_monologue contains a list of subscribers and subscriptions, which are other call_monologues. These lists are reciprocal and a regular A/B call would have two call_monologues with each subscribed to the other. call_monologues includes an object list of call_media that keeps the media attributes and related things. I hope you are not lost!! call_media also keeps an object list of packet_stream and finally packet_steam is linked to stream_fd. It means object hierarchy under the call structure is:

If we look at this flows from bottom to top, when RTPEngine receives incoming RTP packets, they initially received by a stream_fd which directly links to a packet_stream. Each packet_stream then links to an call_media. There are two packet_stream objects per call_media, one for RTP and one for RTCP. The call_media then links to a call_monologue which corresponds to a participant to a call.

During signalling events, the list of subscriptions for each call_monologue is used to create the list of rtp_sink and rtcp_sink given in each packet_stream. Each entry in these lists is a sink_handler object, which again contains flags and attributes. Flags from a call_subscription are copied into the sink_handler. So, during actual packet handling and forwarding, only the sink_handler objects and the packet_stream objects they related to are used.

We have this flow:

In another way, packet handling is done in media_socket.c and starts with stream_packet(). This operates on the originating stream_fd and its linked packet_stream and eventually proceeds to go through the list of sinks, either rtp_sinks or rtcp_sinks, and uses the contained sink_handler objects which point to the destination packet_stream.

So, if we need to change RTP packets or trigger other procedures in parallel before sending out the traffic, we can use sink_handler and its attributes sink_attrs that would affect rtp_sinks list.

I need to spend more time to understand how RTPEngine handles SRTP and what is appropriate changes in xt_RTPENGINE for kernel module.

Why do you need an OTA for eSIM or IoT?

hyavari26@gmail.com (Hossein Yavari) — Fri, 18 Nov 2022 09:23:17 -0800

Over-The-Air (OTA) is a common concept that enables you to distribute various things, like a piece of code, to a device without physical access. In the context of mobile networks and SIM cards, OTA technology enables mobile network operators (MNOs) or MVNOs to change or update data in SIM cards deployed and get service without reissuing them! So, it means through remote provisioning, OTA reduces the cost of manual configuration not only for carriers but also for subscribers.

In the fast-growing eSIM and IoT field, which the content of the SIM or eUICC on a consumer or M2M device needs to be changed according to the subscriber or network demands, the OTA, also known as OTA provisioning, is a mandatory requirement. Operators must communicate with SIM, introduce new services, upload or update their profiles or update firmware rapidly and cost-effectively. So all of these actions are possible through OTA. OTA also is very important from a security perspective; it helps to keep devices updated and apply security patches, bug fixes, or the latest optimized configurations quickly. Although over-the-air provisioning will be established through a secure channel.

Let’s take a look at how we can implement the OTA now that we know what it is and why it’s needed.

What is the OTA Gateway?

The OTA gateway or sometimes referred to OTA platform is an element in the operator network or service providers which enables the firm to use OTA technology and manage SIMs or eUICCs. OTA gateways usually receive SIM management requests through an API like RESTful API and then create an appropriate data format that SIM cards can understand. In the next step, the OTA gateway sends the formatted service request as a message (SMS) to the SMSC. SMSC delivers the SMS through the cellular network to the recipient’s SIM card. OTA usually uses SMS to communicate with SIM, but there are other methods like HTTP(s) which let the SIM card retrieve data from the operator or central servers that are specified.

In the IoT field, it is usual that the service provider does not have the whole operators’ core infrastructure, and provisioning remote devices is impossible. So in this case, some other companies or operators provide OTA gateway as a service.

The OTA gateway services provide RESTful APIs to receive requests from IoT providers or MVNOs, then transform to send them to SMSCs via SMPP.

SMSC owner can be the carrier, or it might be another service from the OTA service provider. This solution will be cost-effective with zero CAPEX and allows SPs to deploy value-added SIM applications and new services for eSIM M2M and IoT.

Telecom and Software Insights

Testing NTN Apps Without Waiting for a Satellite Pass

What it does

Why this matters

TLE support

API-first testing

Android and mobile workflows

CI and replay

Where it fits

Anonymize SIP Traffic in PCAP Files

Why Anonymize SIP Captures?

Features

Consistent Anonymization

Phone Number Normalization

Comprehensive Header Support

Call-ID Correlation

Optional IP Anonymization

Memory Efficient

Usage

What Gets Anonymized

Always Anonymized

With --anonymize-ips

How It Works

Example Output

Extending the Tool

Important Notes

Installation

Useful Resources for LLMs

Testing RTT Feature with iPhone

Reading Logs

Real-Time Text (RTT) in IMS

IMS Training: Part Two

IMS Training: Part One

AWS ECS and Chime Voice Connector

Trust: A Hidden Treasure Among Teammates

IMS Network Announcements

CloudWatch Log Metric Filters

Usefule Links:

ECXCapture: AWS ECS/EC2 Easy Tcpdump Tool

iPhone Live Voicemail

Reminder: You can change CF* settings on iPhone with these codes:

X2/X3 Lawful Interception Dissector

What is x2x3PduDissector?

Key Features:

Usage:

Contributions and Enhancements:

Notes:

SIP PANI Header

RTP Decoder/Encoder

IMS Call Conferencing

SIPREC Server on AWS

VoLTE on IPhone

VoLTE in Pixel vs Samsung Phones

1- IPSec

2- SMS over IP

3- Dedicated Bearer

4- SIP Headers

AWS NLB Is Transparent

SDP Bandwidth Attributes

SDP Bandwidth Attributes

Using SDP Bandwidth Attributes

SDP Bandwidth Attributes and RTPEngine

Conclusion

Usefule Links:

SIP and UDP Fragmentation

In Wireshark, how can you see fragmentation?

Usefule Links:

IMS SMS and Wireshark

What Is IaSQL

IP Public on AWS ECS Fargate

AWS EC2 gai.conf

How Run SIPp on AWS EC2

Book Review: Distributed Systems With Node.js

Remote SIM Provisioning (RSP)

OpenRAN on AWS

Docker Compose With Podman

Book Review: The Kubernetes Book

Hep Protocol and Homer

SIPREC with RTPEngine

How Decrypt ESP Packets in Wireshark

With `--anonymize-ips`