Skip to content

[VPP-1150] SIGABRT in icmp_match_out2in_slow #2682

Closed
@vvalderrv

Description

@vvalderrv

Description

I'm not yet able to define the exact steps that lead to this abort, but for now I'm at least attaching the backtrace:

 

#0 0x00007f439edcc428 in __GI_raise (sig=sig@entry=6)

at ../sysdeps/unix/sysv/linux/raise.c:54

#1 0x00007f439edce02a in __GI_abort () at abort.c:89

#2 0x0000000000406ab6 in os_panic ()

at /opt/vpp-agent/dev/vpp/build-data/../src/vpp/vnet/main.c:268

#3 0x00007f439f5b292b in debugger ()

at /opt/vpp-agent/dev/vpp/build-data/../src/vppinfra/error.c:84

#4 0x00007f439f5b2d33 in _clib_error (how_to_die=2, function_name=0x0,

line_number=0, fmt=0x7f435cbef188 "%s:%d (%s) assertion `%s' fails")

at /opt/vpp-agent/dev/vpp/build-data/../src/vppinfra/error.c:143

#5 0x00007f435cb85a37 in icmp_match_out2in_slow (

sm=0x7f435ce0a520 <snat_main>, node=0x7f435eeeb7c0, thread_index=0,

b0=0x7f435a530a00, ip0=0x7f435a530b0e,

p_proto=0x7f435ee4285c "z\264\343\237\220(\344^C\177",

p_value=0x7f435ee428a0,

p_dont_translate=0x7f435ee4285d "\264\343\237\220(\344^C\177",

d=0x7f435ee42bb0, e=0x0)

at /opt/vpp-agent/dev/vpp/build-data/../src/plugins/nat/out2in.c:372

#6 0x00007f435cb85db4 in icmp_out2in (sm=0x7f435ce0a520 <snat_main>,

b0=0x7f435a530a00, ip0=0x7f435a530b0e, icmp0=0x7f435a530b22,

sw_if_index0=5, rx_fib_index0=0, node=0x7f435eeeb7c0, next0=1,

thread_index=0, d=0x7f435ee42bb0, e=0x0)

at /opt/vpp-agent/dev/vpp/build-data/../src/plugins/nat/out2in.c:485

#7 0x00007f435cb8622f in icmp_out2in_slow_path (

sm=0x7f435ce0a520 <snat_main>, b0=0x7f435a530a00, ip0=0x7f435a530b0e,

icmp0=0x7f435a530b22, sw_if_index0=5, rx_fib_index0=0,

node=0x7f435eeeb7c0, next0=1, now=177.69664372052404, thread_index=0,

p_s0=0x7f435ee42bb0)

at /opt/vpp-agent/dev/vpp/build-data/../src/plugins/nat/out2in.c:592

#8 0x00007f435cb8919e in snat_out2in_node_fn (

vm=0x7f43a0b55480 <vlib_global_main>, node=0x7f435eeeb7c0,

frame=0x7f435e52b4c0)

at /opt/vpp-agent/dev/vpp/build-data/../src/plugins/nat/out2in.c:1364

#9 0x00007f43a08b2d6b in dispatch_node (vm=0x7f43a0b55480 <vlib_global_main>,

node=0x7f435eeeb7c0, type=VLIB_NODE_TYPE_INTERNAL,

dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7f435e52b4c0,

last_time_stamp=192703367656078)

at /opt/vpp-agent/dev/vpp/build-data/../src/vlib/main.c:988

#10 0x00007f43a08b3324 in dispatch_pending_node (

vm=0x7f43a0b55480 <vlib_global_main>, pending_frame_index=2,

last_time_stamp=192703367656078)

at /opt/vpp-agent/dev/vpp/build-data/../src/vlib/main.c:1138

#11 0x00007f43a08b5526 in vlib_main_or_worker_loop (

vm=0x7f43a0b55480 <vlib_global_main>, is_main=1)

at /opt/vpp-agent/dev/vpp/build-data/../src/vlib/main.c:1615

#12 0x00007f43a08b55d4 in vlib_main_loop (vm=0x7f43a0b55480 <vlib_global_main>)

#13 0x00007f43a08b5e4a in vlib_main (vm=0x7f43a0b55480 <vlib_global_main>,

input=0x7f435ee42fb0)

at /opt/vpp-agent/dev/vpp/build-data/../src/vlib/main.c:1789

#14 0x00007f43a091d1eb in thread0 (arg=139928435774592)

at /opt/vpp-agent/dev/vpp/build-data/../src/vlib/unix/main.c:567

#15 0x00007f439f5d6e3c in clib_calljmp ()

at /opt/vpp-agent/dev/vpp/build-data/../src/vppinfra/longjmp.S:110

#16 0x00007ffe213fcdd0 in ?? ()

#17 0x00007f43a091d64c in vlib_unix_main (argc=20, argv=0x25e5740)

at /opt/vpp-agent/dev/vpp/build-data/../src/vlib/unix/main.c:631

#18 0x000000000040679e in main (argc=20, argv=0x25e5740)

at /opt/vpp-agent/dev/vpp/build-data/../src/vpp/vnet/main.c:207

Assignee

Matus Fabian

Reporter

Milan Lenco

Comments

  • matfabia (Fri, 26 Jan 2018 09:14:25 +0000): There is issue with ICMP error messages translation for inner IP packet matching out2in-only static mappings or load balancing static mappings at all.
  • milanlenco (Fri, 26 Jan 2018 08:18:48 +0000): Here is the config:

DBGvpp# sh nat44 detail

NAT plugin mode: dynamic translations enabled

loop1 in out

tapcli-0 in out

tapcli-1 in out

tapcli-2 out

tapcli-3 in out

tapcli-4 in out

tapcli-5 in out

NAT44 pool addresses:

NAT44 twice-nat pool addresses:

1 users, 0 outside addresses, 1 active sessions, 6 static mappings, 0 twice-nat addresses

Hash table in2out-ed

1 active elements

1 free lists

0 linear search buckets

151 cache hits, 1 cache misses

Hash table out2in-ed

1 active elements

1 free lists

0 linear search buckets

159 cache hits, 1 cache misses

Thread 0 (vpp_main at lcore 0):

Hash table in2out

7 active elements

1 free lists

0 linear search buckets

0 cache hits, 0 cache misses

Hash table out2in

6 active elements

1 free lists

0 linear search buckets

0 cache hits, 0 cache misses

2 list pool elements

10.0.2.15: 0 dynamic translations, 1 static translations

static mappings:

tcp local 10.0.2.15:12379 external 192.168.16.1:32379 vrf 0 out2in-only

tcp local 10.0.2.15:12379 external 10.102.7.54:12379 vrf 0 out2in-only

tcp local 10.0.2.15:6443 external 10.96.0.1:443 vrf 0 out2in-only

udp local 10.1.1.2:53 external 10.96.0.10:53 vrf 0 out2in-only

tcp local 10.1.1.2:53 external 10.96.0.10:53 vrf 0 out2in-only

tcp vrf 0 external 10.109.107.254:80 out2in-only

local 10.1.1.4:8080 probability 2

local 10.1.1.5:8080 probability 2

local 10.1.1.6:8080 probability 2

 

Unfortunatelly I have hit this only once. I will provide more info once I know under which conditions this happens.

  • matfabia (Fri, 26 Jan 2018 07:35:27 +0000): Could you please provide at least VPP NAT plugin config?

Original issue: https://jira.fd.io/browse/VPP-1150

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions