Azure Hybrid
Network Lab
A site-to-site IPsec VPN with eBGP between a GNS3 on-prem FortiGate and a real Azure FortiGate (BYOL marketplace), fronting an nginx workload in a private subnet. Built over the public internet, validated end-to-end with HTTP 200, then torn down to $0/month. The story is the hybrid plumbing: tunnel crypto, dynamic routing across the tunnel, NSG and UDR design on the Azure side, and the seven things that went sideways before the lab actually worked.
// Section 01
Architecture
Two FortiGates, one on each side, with eBGP running over a single IPsec tunnel. On the on-prem side, a Cisco IOSv router stands in as the "ISP" during Stages 1 and 2; in Stage 3 it gets swapped for the real internet through a GNS3 NAT cloud and the on-prem FortiGate peers with the real Azure FortiGate's public IP. On the Azure side, the FortiGate has two NICs (untrust and trust) and sits in front of an nginx workload VM with no public IP.
Same vendor on both ends was a deliberate trade. The Multi-Vendor Firewall Lab already covers cross-vendor interop pain; this lab is about hybrid design. Both FortiGates use IKEv2 with pre-shared key auth, MD5-authenticated BGP, and the same three-policy budget on each side.
// IP Scheme
| Segment | CIDR |
|---|---|
| On-prem LAN | 192.168.10.0/24 |
| Tunnel interfaces | 10.255.255.0/30 |
| Azure VNet | 10.100.0.0/16 |
| Untrust subnet | 10.100.1.0/24 |
| Trust subnet | 10.100.2.0/24 |
| Workload subnet | 10.100.10.0/24 |
// BGP Plan
| Side | AS | Advertises |
|---|---|---|
| On-prem | 65001 | 192.168.10.0/24 |
| Azure | 65002 | 10.100.10.0/24 |
Peers on tunnel-interface IPs (10.255.255.1 / .2), MD5 auth, no default route propagation. Each side egresses internet via its own WAN.
// Section 02
IPsec & BGP
IKEv2 tunnel, pre-shared key, MD5-authenticated eBGP riding inside. The wrinkle is the trial license: FortiGate VM evaluation licenses restrict phase 1 and phase 2 proposals to DES-based options only (des-md5, des-sha1, des-sha256). AES and GCM are not available. I picked des-sha256 (weak encryption, strong integrity) as the best available, and called the constraint out in the runbook so a production deployment with a full license would reach for AES-256-GCM with DH19/20.
The Azure FortiGate runs the tunnel as a dial-up responder (set type dynamic, no remote-gw needed) with net-device enable andtransport autoper Fortinet's Azure guidance. The on-prem FortiGate initiates the tunnel to the Azure public IP. BGP came up immediately once the tunnel was established and the firewall policies that reference the tunnel interface were in place. The static routes pointing to the peer's subnet had to be removed once BGP came up, otherwise the AD 10 static suppressed the AD 20 BGP route.
| Parameter | Phase 1 (IKE) | Phase 2 (ESP) |
|---|---|---|
| Protocol | IKEv2 | ESP |
| Encryption | DES (trial license cap) | DES (trial license cap) |
| Integrity | SHA-256 | SHA-256 |
| DH Group | 14 | 14 (PFS enabled) |
| Auth | Pre-shared key | N/A |
| Lifetime | 28,800 sec | 3,600 sec |
// Validation Captures
// Section 03
Azure Design
The Azure side is a single VNet (10.100.0.0/16) with three subnets: untrust for the FortiGate's WAN NIC, trust for its LAN NIC, and workload for the nginx VM. UDRs on the workload subnet force all egress (default route and the on-prem CIDR) through the FortiGate's trust NIC. NSGs scope inbound flows to the minimum each subnet needs.
Untrust subnet NSG
- Allow UDP 500 + 4500 from internet (IPsec)
- Allow ICMP from internet during build (path testing)
- Allow HTTPS from a trusted IP only (FortiGate mgmt)
- Deny all else
Trust subnet NSG
- Allow all from VNet (FortiGate enforces policy)
- Outbound: explicit allow for tunnel CIDR 10.255.255.0/24
- Outbound: explicit allow for on-prem 192.168.10.0/24
- VirtualNetwork tag does NOT cover these sources
Workload subnet NSG
- Allow HTTP from tunnel + on-prem CIDRs
- Allow SSH from FortiGate trust NIC (jump access)
- Deny all else
- UDR forces return traffic back through the NVA
The single biggest Azure-side lesson sits in those NSG bullets: the VirtualNetwork service tag covers VNet-resident addresses only. Tunnel interface IPs and the on-prem LAN are not in the VNet CIDR, so any traffic sourced from those (return traffic from the workload, for instance) gets dropped by the default outbound NSG unless explicit allow rules name those CIDRs. The default rule sounds like "allow within the VNet," but the tag is narrower than the phrase.
// Azure Portal Captures
// Section 04
Traffic Flow
End-to-end, an HTTP request from the on-prem Alpine client to the nginx VM at 10.100.10.4 makes five hops, with the BGP-learned route deciding the tunnel direction at hops 2 and 4 and the UDR forcing the return path back through the NVA at hop 5. The traffic flow diagram (also in the hero carousel) walks the packet through with header detail at each step.
The validating capture is anticlimactic on purpose. After tunnel, BGP, NSG, and UDR all aligned, the on-prem Alpine client ran wget -O- http://10.100.10.4 and got back HTTP 200 with the test page. Every step before that produced a different failure (cloud-init, NSG drop, UDR loop). HTTP 200 means everything in the stack landed.
// Section 05
Real-World Hurdles
Every meaningful one of these is documented in the repo's runbook.md. They are the difference between a lab that worked on paper and a lab that worked on the public internet with a real Azure FortiGate doing the work.
// 7 Documented Hurdles
Nearly every B-, D-, and F-series SKU returned SkuNotAvailable with reason NotAvailableForSubscription. The FortiGate VM and the workload VM both failed to deploy on the trial tier in any region tested.
Pay-As-You-Go unlocked the SKU list, but F2s_v2 was genuinely unavailable in East US (real capacity, not entitlement). Available SKUs were ARM64 and DC-series only, neither of which the FortiGate marketplace image supports.
Default FortiGate marketplace image is Gen1 BIOS. The newer v6/v7 Azure VM families are Gen2 (UEFI) only. Deploy failed with: The selected VM size cannot boot Hypervisor Generation '1'.
Standard_F2als_v6 (2 vCPU / 4 GB) failed health checks against the trial license. The FortiGate boots but the license rejects the resource profile.
The nginx HTML test page contained an em dash. Cloud-init failed with: 'latin-1' codec can't encode character. nginx never installed, the workload VM came up with no service, and the eventual HTTP test failed for a reason that had nothing to do with the tunnel.
Trial license restricts crypto to DES-based proposals only. Modern browsers refuse the TLS handshake (PR_CONNECT_RESET_ERROR), so the FortiGate web GUI is functionally unavailable. The license that lets the tunnel come up locks the operator out of the dashboard.
Initial ping from on-prem to the nginx VM failed silently. Packets reached the trust subnet's egress NSG and were dropped because the source IP was the tunnel interface (10.255.255.1), which is not in the VNet CIDR. The VirtualNetwork service tag covers VNet-resident addresses only.
// Section 06
Validation
The success criteria came from the design doc, not from what looked good after the fact. The lab was "working" when these were all true, captured on both sides, and reproducible from configs in the repo.
- [OK] IPsec phase 1 and phase 2 established on both FortiGates over the real internet.
- [OK] eBGP neighbor reached Established in both directions with non-zero uptime.
- [OK] On-prem routing table contains the Azure workload subnet learned via BGP; Azure routing table contains the on-prem LAN.
- [OK] Alpine on-prem client retrieves the Azure nginx test page by private IP, HTTP 200.
- [OK] All FortiGate, IOSv, and runbook artifacts committed to the public repo (sanitized).
- [OK] Azure resources deallocated and resource group deleted; ongoing cost projected at $0/month.
The repo's configs/phase3/ folder contains the sanitized FortiGate configs from both sides, plus the IOSv transit router config from Stages 1 and 2 (kept as documentation of the GNS3-only path before Stage 3 swapped to the real internet).
// Section 07
Cost
The lab fit inside the $200 / 30-day free credit with margin. Two design decisions did the heavy lifting: FortiGate on both ends (no Azure VPN Gateway) and aggressive deallocation between sessions. The lab is "done" at $0/month, which matters because a portfolio piece that costs $40/month to keep running is a portfolio piece I would eventually shut off.
Skipping Azure VPN Gateway saved $27 to $140 per month depending on SKU. The trade is that the Azure FortiGate has to handle the role Gateway would own (IKE responder, public IP termination, NAT awareness) which is exactly the work this lab is built to demonstrate.
The tag-based deallocation script lived in the repo before any VM was deployed. End of every working session: deallocate-lab.sh stops the FortiGate and the workload VM. Compute stops billing immediately; only the public IP keeps a small standing charge.
Configured before the first deploy. Never tripped, but the alert is the backstop. Total spend across the build came in under $20, well inside the credit envelope.
Resource group rg-hybrid-lab was deleted entirely once all artifacts were captured. The portfolio piece is the screenshots, configs, and case study; the live Azure infrastructure was never the deliverable.
// Section 08
Key Takeaways
- 01 // FortiGate-on-Azure is real production tooling, not a tutorial path. The marketplace listing has Gen1/Gen2 SKUs, the license has a size cap, and the trial license shapes crypto choices. None of that surfaces until the first deploy fails.
- 02 // Hybrid connectivity is more than the tunnel. The IPsec + BGP layer is the easy part once both sides are FortiGate. The harder work is the Azure-side fabric: UDRs that force traffic through the NVA, NSGs that allow non-VNet sources, and platform NAT awareness on the FortiGate's public-facing interface.
- 03 // Azure service tags are narrower than they sound. VirtualNetwork covers VNet-resident addresses only. Any tunnel-based design with non-VNet sources (on-prem LAN, tunnel interface IPs) needs explicit NSG rules, not a tag.
- 04 // Cost discipline is a design constraint, not an afterthought. Using FortiGate at both ends instead of FortiGate-to-VPN-Gateway skipped the $27 to $140 per month Gateway charge, and aggressive deallocation kept total spend inside the $200 credit. The lab demonstrates the design and ships at $0/month after teardown.
- 05 // Same-vendor on both ends is a deliberate trade. It removed an entire class of cross-vendor interop pain (DH groups, proxy IDs, license overrides) so the rest of the design could move. The Multi-Vendor Firewall Lab already covers the cross-vendor story; this one is about the hybrid model.
- 06 // Document while context is fresh, not after teardown. Phase 3's runbook had seven Azure-specific troubleshooting sections by the end of deployment day. Without that, half of these takeaways would be hand-waved a week later.
// Cert Alignment
AZ-900 // Cloud Fundamentals
- VNet, subnet, NSG, UDR mental model in practice
- Marketplace VM deployment with BYOL licensing
- Cost Management, budget alerts, free credit discipline
- Service tags (VirtualNetwork) and where they fall short
NSE 4 // FortiGate Prep
- IKEv2 IPsec on FortiGate, route-based tunnels
- Dial-up vs dial-out responder modes
- FortiGate-on-Azure platform NAT and public IP awareness
- BGP authentication, peering on tunnel interfaces
CCNP ENCOR // Direction
- eBGP between private ASNs across a tunnel
- BGP route advertisement and authentication
- Static vs dynamic route administrative distance
- Hybrid cloud transit and dynamic-routing design
// Lab Environment