How to use Overplay and other VPNs as a cURL proxy

UPDATE May 2015

When I posted this tutorial two years ago, I gradually got comments that the OpenVPN interface was setup successfully but curl traffic through the interface timed out. I tried to help but responded with the cliched ‘works on my machine’, with my machine being Debian Squeeze and kernel version 2.6. Recently I upgraded my server to Debian Jessie, kernel 3.14 and OpenVPN 2.3.4 and got the timeout problem that everyone was talking about. If I’d to guess, I’d say it’s some change in the kernel and routing tables but I really have no idea. But fear not, a random commentator from May 2014 called William Asssaad saved us all. We need to add a routing table for the VPN interface after it’s all setup. You can do it manually with the ip route add and ip rule add commands but I prefer to do it in a shell script, which I got from this blog post: http://snikt.net/blog/2013/10/10/how-to-force-program-to-use-vpn-tunnel/

Proxies are like hard drive space, you can never have enough. Or enough IPs to be more accurate as Facebook, Google and other services are getting better at flagging the IPs of popular HTTP/SOCKS5 proxies. So we need to find fresh proxies to use in our PHP/Python scripts. A great source is the proliferation of VPN services that are popping up as consumers worry more about their Internet privacy. Problem is these are intended for use by endusers on their desktops and not in serverside PHP scripts. So it’s a bit tricky to get these working with cURL, but fear not, I explain all in this post.

The magic of being able to use a VPN in cURL is the CURLOPT_INTERFACE option. This lets you set the network interface that cURL uses. You can’t use a VPN directly in cURL as cURL/PHP operates on a higher network level than the VPN protocol.

1)
So we need to setup the VPN on a new interface. Note that you absolutely need root access to your server to create interfaces so this guide is only useful for people with their own dedicated servers. People on shared $2/month servers are shit out of luck. You might get it working on a VPS, I’ve no idea. So to create an interface, you need to download and install OpenVPN if you haven’t it installed already. There’s loads of info online to help you do this, so figure it out and come back.

2)
Next we need to get the configuration files we need from our VPN provider of choice. I use Overplay.net. You need an account with them and it cost something like $5 a month. Download the ZIP file of connection files and unzip on your server in a new directory. Also download the Overplay public key certificate and make sure it’s in the same directory:
curl --insecure -o connections.zip https://patch.oppatch.com/connections.zip
unzip -u connections.zip
curl --insecure -o OverplayCert.crt https://patch.oppatch.com/OverplayCert.crt

Now we have the connection files which work fine if you run Linux as your desktop OS and just want to browse the web as described in this Overplay guide. But we DO NOT want to just start the VPNs as is as it will take over the main Internet connection and make your server inaccesible. I did this a few times and had to get my host to reboot my server.

3)
So we need to edit the configuration files and add one command, route-nopull. This prevents Overplay from taking over the routing information. If you take one thing away from this blogpost, it’s the addition of the route-nopull option as it’s what lets you use these config files on your server.

May 2015: we need to change route-nopull to these 3 commands I got from this blogpost: http://snikt.net/blog/2013/10/10/how-to-force-program-to-use-vpn-tunnel/
script-security 2
route-noexec
route-up /root/route_up.sh

route-up is the imporant bit, it runs the shell script at /root/route_up.sh when the interface is created. Here’s the contents of route_up.sh:

#!/bin/sh

echo "$dev : $ifconfig_local -> $ifconfig_remote gw: $route_vpn_gateway"

ip route add default via $route_vpn_gateway dev $dev table 20
ip rule add from $ifconfig_local table 20
ip rule add to $route_vpn_gateway table 20
ip route flush cache

exit 0

I also want to add my login as a file so I don’t have type it everytime. So create a new file in the same directory and name it ‘auth_overplay’ or whatever you want. Enter your username and password, seperated by a newline. So if we’re taking the ‘Overplay – Ireland-1.conf’ file as our example, our config would now look like this. Our additions are in bold at the bottom:
client
dev tun
proto udp
remote 176.67.85.4 1443

resolv-retry infinite
nobind
persist-key
persist-tun
ca OverplayCert.crt
verb 5
route-method exe
route-delay 2

tun-mtu 1500
tun-mtu-extra 32
mssfix 1450
comp-lzo

script-security 2
route-noexec
route-up /root/route_up.sh
auth-user-pass auth_overplay
#daemon

Note we also added daemon at the end, but commented it out. You can uncomment this when you’ve got everything working and want to start the VPN as a daemon so you can use it without having to have the SSH window open.

4)
Now start up the VPN with OpenVPN:
openvpn "Overplay - Ireland-1.conf"

If everything works, you should see output like this:
Fri Jul 26 21:05:13 2013 us=31236 OpenVPN 2.1.3 x86_64-pc-linux-gnu [SSL] [LZO2] [EPOLL] [PKCS11] [MH] [PF_INET6] [eurephia] built on Feb 21 2012
Fri Jul 26 21:05:13 2013 us=31325 WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.
Fri Jul 26 21:05:13 2013 us=31331 NOTE: OpenVPN 2.1 requires '--script-security 2' or higher to call user-defined scripts or executables
Fri Jul 26 21:05:13 2013 us=31678 LZO compression initialized
Fri Jul 26 21:05:13 2013 us=31725 Control Channel MTU parms [ L:1574 D:138 EF:38 EB:0 ET:0 EL:0 ]
Fri Jul 26 21:05:13 2013 us=31748 Socket Buffers: R=[124928->131072] S=[124928->131072]
Fri Jul 26 21:05:13 2013 us=31766 Data Channel MTU parms [ L:1574 D:1450 EF:42 EB:135 ET:32 EL:0 AF:3/1 ]
Fri Jul 26 21:05:13 2013 us=31777 Local Options String: 'V4,dev-type tun,link-mtu 1574,tun-mtu 1532,proto UDPv4,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-client'
Fri Jul 26 21:05:13 2013 us=31781 Expected Remote Options String: 'V4,dev-type tun,link-mtu 1574,tun-mtu 1532,proto UDPv4,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-server'
Fri Jul 26 21:05:13 2013 us=31794 Local Options hash (VER=V4): 'd3a7571a'
Fri Jul 26 21:05:13 2013 us=31802 Expected Remote Options hash (VER=V4): '5b1533a2'
Fri Jul 26 21:05:13 2013 us=31811 UDPv4 link local: [undef]
Fri Jul 26 21:05:13 2013 us=31816 UDPv4 link remote: [AF_INET]176.67.85.4:1443
WRFri Jul 26 21:05:13 2013 us=37221 TLS: Initial packet from [AF_INET]176.67.85.4:1443, sid=a552afa0 928c908a
WFri Jul 26 21:05:13 2013 us=37266 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
WRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRFri Jul 26 21:05:13 2013 us=72079 VERIFY OK: depth=1, /C=UK/ST=LANCS/L=MANCHESTER/O=OVERPLAY.NET_LLP/OU=CA/CN=OVERPLAY_CA/emailAddress=ca@overplay.net
Fri Jul 26 21:05:13 2013 us=72206 VERIFY OK: depth=0, /C=US/ST=IL/L=Chicago/O=OVERPLAY.NET_LLP/OU=SERVERS/CN=vpn1-us/emailAddress=ca@overplay.net
WRWRWRWRWWRRWWWWRRRRWRWRFri Jul 26 21:05:13 2013 us=472472 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
Fri Jul 26 21:05:13 2013 us=472487 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Fri Jul 26 21:05:13 2013 us=472528 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Fri Jul 26 21:05:13 2013 us=472534 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
WFri Jul 26 21:05:13 2013 us=472560 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 1024 bit RSA
Fri Jul 26 21:05:13 2013 us=472576 [vpn1-us] Peer Connection Initiated with [AF_INET]176.67.85.4:1443
Fri Jul 26 21:05:15 2013 us=598837 SENT CONTROL [vpn1-us]: 'PUSH_REQUEST' (status=1)
WRRWRWRFri Jul 26 21:05:15 2013 us=604235 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 176.67.84.100,route 10.204.0.1,topology net30,ping 10,ping-restart 120,ifconfig 10.204.0.54 10.204.0.53'
Fri Jul 26 21:05:15 2013 us=604257 Options error: option 'redirect-gateway' cannot be used in this context
Fri Jul 26 21:05:15 2013 us=604275 Options error: option 'route' cannot be used in this context
Fri Jul 26 21:05:15 2013 us=604293 OPTIONS IMPORT: timers and/or timeouts modified
Fri Jul 26 21:05:15 2013 us=604297 OPTIONS IMPORT: --ifconfig/up options modified
Fri Jul 26 21:05:15 2013 us=604301 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
Fri Jul 26 21:05:15 2013 us=604472 TUN/TAP device tun0 opened
Fri Jul 26 21:05:15 2013 us=604485 TUN/TAP TX queue length set to 100
Fri Jul 26 21:05:15 2013 us=604507 /sbin/ifconfig tun0 10.204.0.54 pointopoint 10.204.0.53 mtu 1500
WFri Jul 26 21:05:17 2013 us=671302 Initialization Sequence Completed

Now open a new SSH session as root and enter ifconfig to see the list of network interfaces. You should see your proxy listed with the interface name tun0 or something like that.

Some OverPlay servers don’t complete the TLS handshake for me but I’m thinking that was because they were old IPs. Overplay seems to change actual proxy servers a lot.

5)
If you see the tun0 interface in ifconfig, then it worked, probably. Test it with cURL on the command line:
curl --interface tun0 http://icanhazip.com

Replace tun0 with whatever your interface is called. icanhazip.com is a simple site that outputs your IP and nothing else and has been online for 3 years so hopefully it stays online. If everything works, the site should output the IP of Overplays proxy and not your server IP.

To use the interface in your PHP scripts, you’d set it with something like this:
curl_setopt($curlh, CURLOPT_INTERFACE, "tun0");

I found from experience that PHP/cURL defaults to using the standard network interface if something doesn’t work, so it grabs pages using your real IP! This can be a disaster if you’re doing dodgy stuff and don’t want to get banned from FB or Google. For this reason, I always grab icanhazip.com in my scripts and check to make sure it’s not my server IP.

Well that’s about it. I also wrote a few PHP scripts that automatically download the Overplay config files, edit them, check what country the IP is and add them to a MySQL table. Then I’ve other PHP code that just queries the table grabbing a random proxy from the country I need. I’ll do a blogpost and share that code if anyone actually reads this one and comments. And also comment if you run into any trouble, I’m usually quick to answer.

Published by Georgie Casey

student. Google+

Join the Conversation

31 Comments

  1. I’ve working with Hide my Ass and i test your work, but i can’t get a connection if i call
    “curl –interface tun0 http://icanhazip.com” i think data i send but no response, and i’ve to cancel with ctrl+c

    1. What’s the output of ifconfig? And what’s the output of Openvpn when you’re connecting to HideMyAss?

    2. I am finding that ‘route-nopull’ is preventing the VPN working at all on the default or tun0 interface.

      Great post though.

      I am still looking for a way to establish these vpn connections without the main interface being interfered with.

      1. Hi , maybe i’m a little late but i’ve found a solution for your problem you need to add ip routes like this then it’ll work :
        [ ip route add default via {P-t-P-IP} dev tun0 table 10 ]
        [ ip rule add from {tun0-inet addres} table 10 ]

        I had the same problem a while ago, hop it works for you !

  2. Very useful, just what i was looking for. Any chance of getting the scripts that download the config files and edit them?

  3. Hi there,
    great tutorial!!! One of a kind!
    I’m trying to make this work with HideMyAss VPN Pro service.

    I successfully tweaked the configuration file in order to create the parallel interface! In the ouput of ‘ifconfig’ I can see the tun0 interface.. It looks like this:

    tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
    inet addr:10.200.2.24 P-t-P:10.200.2.24 Mask:255.255.252.0
    UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
    RX packets:1 errors:0 dropped:0 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:100
    RX bytes:44 (44.0 b) TX bytes:0 (0.0 b)

    Unfortunately when I run curl –interface tun0 http://icanhazip.com nothing seems to happen and I have to break the command in order to continue.

    When I launch the openvpn I’m getting the following message in the logs
    kernel: tun0: Disabled Privacy Extensions
    I don’t know if this is important…

    Another thing is that every time I start a new openvpn session I get a new interface layer… tun0, tun1, tun2….
    How can I terminated the existing tun interface before starting a new one?

    Thanks,
    Mike

    1. These VPN services seem to be hit and miss with OpenVPN and I often have problems myself with different services and even different proxies of each service. Fortunately, the correct OpenVPN config change nearly always fixes things as long as the proxy is actually working! Check first to see if it works on Windows with the default client.

      What country of HideMyAss are you trying to get working? And that ‘disabed privacy extenstions’ is nothing major, can you paste the rest of the OpenVPN log?

      To stop all OpenVPN instances, I just use ‘killall openvpn’

      1. Hi Georgie,
        I tried to use a simpler approach in order to simplify the debugging process.. 😉
        I got rid of HideMyAss network and tried to do exactly what you are describing using OverPlay.

        I get the “Initialization Sequence Completed” message and the new interface (tun0) but when I try to run curl via tun0 interface (curl –interface tun0 http://icanhazip.com) I get nothing. The command stalls until I manually stop it.
        I tried running openvpn both in console and daemon mode with the same result.

        Here (https://dl.dropboxusercontent.com/u/6759341/OpenVPN_mike.zip) you can find both the OverPlay config I used and the full OpenVPN log.

        Any help would be appreciated..!

        Kind regards,
        Mike
        PS: I had to turn off my CSF firewall in order to get the “Initialization Sequence Completed”. Do you have any suggestions on the rules I should use in order to run OpenVPN with the firewall?

        1. I tried that overplay config and it worked fine for me. And your OpenVPN log seems fine. I’m pretty sure I know what your problem is. Are you trying to run (curl –interface tun0 http://icanhazip.com) without root privileges? This is something I forgot to mention in my post, you need root to be able to use a different interface with curl. I understand this is a pretty big omission, I only realised it after I’d published my post. I’ve another post in draft about the safest way to let your PHP scripts get root privilieges needed to use a VPN interface. First verify if that is indeed your problem.

          1. Hi Georgie,
            Unfortunately this doesn’t seem to be the problem as I’m logged in as root when I run curl –interface tun0 http://icanhazip.com

            If I let the above command run for about 90 sec I get the following error:
            curl: (7) Couldn’t bind to ‘tun0’

            By the way here’s the ifconfig printout concerning tun0… does this look OK?

            tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
            inet addr:10.204.0.14 P-t-P:10.204.0.13 Mask:255.255.255.255
            UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:100
            RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

            Mike

          2. Jesus I’m kinda lost now. I was sure it was root causing it. That ifconfig output is the exact same as mine apart from IP address. What kind of server is it, a VPS or proper dedicated server? Run this command and paste the output to see what cURL is outputting.
            curl --verbose --interface tun0 http://ipv4.icanhazip.com

            EDIT: Yes, try that line that only returns the IPv4 response of icanhazip. Doing a diff on our Overplay OpenVPN logs, you’ve a lot of IPv6 stuff that I don’t have. Might be caused by this: http://stackoverflow.com/questions/18340021/curl-interface-not-working-couldnt-bind-to-error

          3. Hi Georgie,
            It a dedicated server… actually I have 2 dedicated servers with different linux distributions and I have the exact same problem on both of them..

            Here’s the curl output:
            # curl –verbose –interface tun0 http://ipv4.icanhazip.com
            * About to connect() to ipv4.icanhazip.com port 80 (#0)
            * Trying 216.69.252.101… Local Interface tun0 is ip 10.204.0.42 using address family 2
            * Local port: 0
            * Connection timed out
            * Trying 216.69.252.100… Local Interface tun0 is ip 10.204.0.42 using address family 2
            * Local port: 0
            * Connection timed out
            * couldn’t connect to host
            * Closing connection #0
            curl: (7) couldn’t connect to host

            Thanks for the stackoverflow link.. I’ll check it out asap!!

  4. Is there any way to prevent curl from falling back to your actual IP if the VPN times out or disconnects. Dodgy stuff.

    1. Yes, very dodgy stuff Google IP commentator! I wonder what you’re doing that you need to hide your IP?! I couldn’t find a rocksolid way to do this that I could trust in cURL itself so I simply had an IP check in my application code before every request. I placed code in my curlRequest method that GET a PHP script URL on my server that just outputs $_SERVER[‘REMOTE_ADDR’] and aborts the script if it’s one of my server IPs. Of course, there’s the possibility the IP check HTTP request can go through the proxy but fail and use your real IP on the real request. But this solution is good enough for me.

      1. Well, I don’t believe what he’s going to do is the actually your concern at all. AT ALL. One could say that you using Overplay, circumventing license restrictions to watch what you’re not supposed to watch/listen to (say, Netflix, iBBC, Pandora, etc) you’re breaking the law and stealing someone’s money [from licenses].
        Does anyone here judges you? I don’t think so.

        If he wants to use VPN he can use it. Whether he’s hiding from snooping government, abusive employer or his own ISP (think of Verizon’s supercookies you can’t opt out from).

        Really, I’m disappointed. Please quickly give me your facebook and email account passwords, I would like to make sure that you don’t have anything to hide. And your social insurance number, and medical history, of course – we’re all human, can’t we learn about your weak spots and hard times?

        Please, STOP judging people you do not know. It’s disguisting.

  5. Hello, first thank you for your post.

    Everything work fine except the last step

    vpn status
    Sat Mar 15 08:51:55 2014 TUN/TAP device tun0 opened
    Sat Mar 15 08:51:55 2014 TUN/TAP TX queue length set to 100
    Sat Mar 15 08:51:55 2014 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
    Sat Mar 15 08:51:55 2014 /sbin/ifconfig tun0 x.xxx.x.xx netmask 255.255.255.22 4 mtu 1500 broadcast x.xxx.xx.xxx
    Sat Mar 15 08:51:57 2014 Initialization Sequence Completed

    ifconfig
    tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
    inet adr:x.xxx.xxx.xx P-t-P:x.xx.xxx.xx Masque:255.255.255.224
    UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
    RX packets:6 errors:0 dropped:0 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 lg file transmission:100
    RX bytes:426 (426.0 B) TX bytes:0 (0.0 B)

    curl –interface tun0 http://icanhazip.com
    curl: (45) Failed binding local connection end

    Maybe can you help me on this strange issue ? Thank you.

      1. I have the same issue :s Im running as root, but my vpn setup is pptp based.
        here is the command output:
        root@vps75483:/var/run/pptp# ifconfig
        lo Link encap:Local Loopback
        inet addr:127.0.0.1 Mask:255.0.0.0
        inet6 addr: ::1/128 Scope:Host
        UP LOOPBACK RUNNING MTU:16436 Metric:1
        RX packets:174 errors:0 dropped:0 overruns:0 frame:0
        TX packets:174 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:19137 (18.6 KiB) TX bytes:19137 (18.6 KiB)

        ppp0 Link encap:Point-to-Point Protocol
        inet addr:10.1.1.10 P-t-P:10.1.1.1 Mask:255.255.255.255
        UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1496 Metric:1
        RX packets:6 errors:0 dropped:0 overruns:0 frame:0
        TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:3
        RX bytes:60 (60.0 B) TX bytes:66 (66.0 B)

        venet0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
        inet addr:127.0.0.2 P-t-P:127.0.0.2 Bcast:0.0.0.0 Mask:255.255.255.255
        inet6 addr: 2001:41d0:52:a00::c86/56 Scope:Global
        UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1
        RX packets:1814 errors:0 dropped:0 overruns:0 frame:0
        TX packets:1631 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0
        RX bytes:387463 (378.3 KiB) TX bytes:221268 (216.0 KiB)

        venet0:0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
        inet addr:XX.222.168.207 P-t-P:XX.222.168.207 Bcast:XX.222.168.255 Mask:255.255.255.0
        UP BROADCAST POINTOPOINT RUNNING NOARP MTU:1500 Metric:1

        root@vps75483:/var/run/pptp# curl –interface ppp0 http://icanhazip.com
        curl: (45) Failed binding local connection end

        its by privateinternetaccess.com it was working fine on my other vps which got expired and i forgot to take a backup.. now im trying to set it up again on a new vps, it connects fine but doesnt bind to curl. Please help.

        1. another thing, the vpn is working fine on my laptop and its connecting properly on the server as well. i tested it by looking into this directory:
          root@vps75483:/var/run/pptp# dir
          255.255.255.255:199.167.138.130

  6. I figured out the issue, it was some issue with the network configuration of my vps. It was configured with ipv4 and ipv6 as well.. hence the conflicts. i tried forcing curl to use ipv4 addresses only with the -4 switch, but that didnt work either, I suppose it has something to do with the network configuration of the server my vps is on.
    I tried it on my other dedicated server on a different datacenter, it worked perfectly fine without a problem. I still dont know how to make it work on the vps though.. maybe the router on their datacenter is blocking the connection or something..

  7. Is this post still valid? If so – With the Overplay.net VPN, do you get a bunch of IPs in the configuration files? It doesn’t look like it gives you a bunch of IPs for the $10/mo plan.

  8. When running as deamon , make sure all file paths in config file are absolute , else the deamon wont start.

  9. This is great!

    OpenVpn has been a steep learning curve for me and this proved invaluable.

    I couldn’t find similar instructions anywhere else.

Leave a comment

Your email address will not be published. Required fields are marked *