Open SourcePython

Proxy.py – A lightweight, single file HTTP proxy server in Python

Back in 2014, while working at Appurify Inc. (now Firebase Test Lab) I came across a very interesting problem. To build some context, Appurify enabled mobile test automation on real devices for mobile teams to build better apps. Companies would upload their .apk (Android app) and .ipa (iOS app) files on Appurify servers and configure various test profiles (Network, Memory, GPS etc conditions) to perform wide-scale testing on Appurify mobile devices. This allowed developers to catch bugs and other performance issues on devices they can’t test themselves.

In several cases, developers wanted their uploaded (unreleased) mobile apps to communicate with under-development server-side modules running within company intranet. In absence of a communication channel between Appurify devices and company intranet, developers were forced to either disable such features in their mobile apps or simply ignore testing such features. To facilitate end-to-end testing for such scenarios, I architected a proxy infrastructure; A stripped down version of which was a lightweight HTTP proxy server in Python.

 

How it works

A simplified view of proxy infrastructure

A quick run through of the process:

  1. Developer builds and uploads mobile app on Appurify servers.
  2. The developer also submits one or more test profile, enabling proxy infrastructure if necessary.
  3. Prior to requesting proxy infrastructure, developers were expected to start Appurify provided proxy.py module within the company network (more on it below).
  4. If proxy infrastructure was enabled for a test, mobile app network traffic for that test would be routed through Appurify’s tunnel gateway (using iptables).
  5. Mobile traffic reaches the company internal network via the established tunnel.
  6. Mobile traffic resolves within the company internal network using the HTTP proxy (the open source part) started as part of step 3.

Above workflow allowed mobile developers to enable a test environment on Appurify devices, similar to how it would be if they tested their mobile apps on a locally handheld mobile device. Example:

  1. Mobile apps can now access unreleased server-side module, available only from within the company internal network
  2. Mobile apps would experience external websites tailored for the region where the developer lived. Example, for a developer sitting in China, visiting google.com on Appurify devices will redirect to google.cn.

Within the Proxy Module

Shipped proxy module consisted of 2 parts:

  1. SSH Tunnel — It allowed bidirectional communication channel between Appurify mobile devices and company internal network. SSH tunnels were secured using public-private key infrastructure.
  2. HTTP Proxy — It allowed incoming HTTP traffic from Appurify mobile devices to resolved within the context of a company internal network.

Only the HTTP proxy piece was open sourced, a lightweight, single file module implementation of HTTP specification supporting HTTP, HTTPS, and even WebSocket requests proxy.

I had some real fun architecting, developing and solving this problem and it was a real pleasure listening from customers “Hey, this just magically works. Thank you!!!”.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.