tcp-tunnel can’t bind a connection

[MCApollo]

A few days ago, I wanted to try to make a buildserver out of this project (fork here), so I updated the source to qemu 5.0.0 and apply some PVE/Proxmox patches so it’ll work with my use case; Sadly there’s no serial output, but I’m trying to see if maybe iOS is continuing to run in the background.

I’ve been having some trouble SSH’ing into iOS with tcp-tunnel and see if someone maybe knows a answer; The distro is debian buster and my thoughts are a header conflict between apple-darwin and gnu-linux, help would be appreciated, Thanks!

[sickcodes]

Only works on MacOSX as the host because the networking system calls are different on Linux. I have also been trying to get around this.

Secondly, the no-frills way to SSH in is using ash shell.

I'll pull out a docker image so people who know how to fix it, can quickly get in started :)

[sickcodes]

I also see 97 which I think was bind error. It works on OSX thru KVM.

You'll also see 255 error if it's starting on boot. @aronsky said it's simply the hard coded system calls in getting a TCP socket in both the QEMU side and the tunnel in the xnu-qemu-arm64 tools folder.

If anyone knows how to change that to Unix ports that would be coolio.

OR

the easiest way to leap over it is possibly emulating the Darwin kernel via qemu which is what I have to do to ssh in, but it's heavy as I'm booting OSX each time. Maybe qiling framework or usercorn can remap those system calls automatically?

[sickcodes]

Here's his write up, https://alephsecurity.com/2020/03/29/xnu-qemu-tcp-tunnel/

Would be awesome to get the tunnel working on GNU/Linux!

[aronsky]

Hi, @MCApollo, the error in tcp-tunnel happens here: https://github.com/alephsecurity/xnu-qemu-arm64-tools/blob/master/tcp-tunnel/src/main.c#L295. This call to connect is handled by the host (QEMU) here: https://github.com/alephsecurity/xnu-qemu-arm64/blob/master/hw/arm/guest-socket.c#L128. Can you debug your execution of QEMU, set a breakpoint there, and verify that the passed addr variable is reasonable?

[MCApollo]

@aronsky https://gist.github.com/MCApollo/9b26e3986634515f6430bb040891b145

I don't really write C code that often, but that looks right; Let me know if you need any other information.

[aronsky]

It seems you set the breakpoint on qc_handle_bind, while the failure in your earlier screenshot happens inside qc_handle_connect. Can you run it again, with the breakpoint at qc_handle_connect? But even here, at qc_handle_bind, the value in addr->sin_family doesn't seem correct - 52 is a weird number, it's not AF_INET as it should be (which is 2 under both Linux and macOS)...

[MCApollo]

Sorry about that, beat me to the edit. I can't hit the breakpoint in qc_handle_connect, only bind hits.

bash-4.4# tunnel 2222:127.0.0.1:22

Thread 3 "qemu-system-aar" hit Breakpoint 4, qc_handle_bind (
    cpu=0x5555567f5310, sckt=0, g_addr=0x16fdffbe0, addrlen=16)
    at /home/mcapollo/iOS-builder/diff/xnu-qemu-arm64/hw/arm/guest-socket.c:88
88      {
(gdb) c
Continuing.
accept failed: 11

[MCApollo]

sys/errno.h:
#define EDEADLK 11 /* Resource deadlock avoided *

and there's no define for 52 in sys/socket.h, spooky.

Edit
I did plug a Ethernet cord into the box, that seemed to changed the behavior; The connection doesn't drop, but no data is being sent.

bash-4.4# /iosbinpack64/usr/local/bin/wget localhost       
--1970-01-01 00:21:36--  http://localhost/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... failed: Connection refused.
Connecting to localhost|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... No data received.
Retrying.

I'm starting to think that the mismatch of headers is causing a issue and is the root of the problem:
#define ENOLINK 97 /* Reserved */
#define ENETRESET 52 /* Network dropped connection on reset */

[aronsky]

Hm... It's interesting that the connect breakpoint is never hit, since in your previous screenshot it's where the problem happens... Are you executing tcp-tunnel with the same arguments? And there's definitely a good chance that the values mismatch between Linux and macOS is the issue, just trying to identify what exactly is happening.

[MCApollo]

Executed the example.com example and tried to connect to localhost, but for the life of me I can’t reproduce the issue in the picture.

Did mess with /etc/interfaces in the last couple days, if that’s some insight.

[aronsky]

What are the arguments to tcp-tunnel when you hit the breakpoint in qc_handle_bind? Do you get any errors from the iOS shell once you continue the execution?

[MCApollo]

There we go, the wget/example.com combo; connect happens on out, my bad.

--1970-01-01 00:12:43--  http://localhost/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... failed: Connection refused.
Connecting to localhost|127.0.0.1|:80... 
Thread 3 "qemu-system-aar" hit Breakpoint 2, qc_handle_connect (cpu=0x5555567f5310, 
    sckt=0, g_addr=0x16fdffb58, addrlen=16)
    at /home/mcapollo/iOS-builder/diff/xnu-qemu-arm64/hw/arm/guest-socket.c:115
115     {
(gdb) s
118         VERIFY_FD(sckt);
(gdb) s
120         int retval = 0;
(gdb) s
122         if (addrlen > sizeof(addr)) {
(gdb) s
125             cpu_memory_rw_debug(cpu, (target_ulong) g_addr, (uint8_t*) &addr,
(gdb) p addr
$2 = {sin_family = 52, sin_port = 0, sin_addr = {s_addr = 0}, 
  sin_zero = "\000\360\337o\001\000\000"}
(gdb) s
cpu_memory_rw_debug (cpu=0x5555567f5310, addr=6171917144, buf=0x7fffafffe2c0 "4", 
    len=16, is_write=0) at /home/mcapollo/iOS-builder/diff/xnu-qemu-arm64/exec.c:3753
3753    {
(gdb) next
3757        cpu_synchronize_state(cpu);

Arguments for qc_handle_bind where the ssh example on xnu-tools page and qc_handle_connect is the example.com example.

No errors for the wget example, the connection now just doesn’t go through.

[sickcodes]

@MCApollo so excited! Can you push to your repo if you got the tunnel working on Linux? I pulled from your repo last night, I'll push my changes including Dockerfile.

Were you able to resize the disk on Linux?

I can mount and edit, just can't resize

[aronsky]

@MCApollo okay, looking good. However, the value of addr you're printing is actually before cpu_memory_rw_debug is called (and, therefore, is not initialized yet). Use n instead of s to step over function calls, and print (p addr) the value of addr after the call to cpu_memory_rw_debug, when the debugger is stopped at line 128.

[sickcodes]

@aronsky @MCApollo shouldn't internet be disabled until the device is unlocked or has a SIM card or fake WiFi network since we are booting into regular iOS?