BeforeEach method runs before every state

[Katka92]

Software versions

• OS: Fedora Linux 37 (Workstation Edition)
• Consumer Pact library: pact-foundation/pact 11.0.2, jest-pact 0.10.3
• Provider Pact library: pact-foundation/pact-go/v2 v2.0.0-beta.23
• Golang Version: go1.19.12 linux/amd64
• Golang environment:
  GO111MODULE=""
  GOARCH="amd64"
  GOBIN=""
  GOCACHE="/home/katka/.cache/go-build"
  GOENV="/home/katka/.config/go/env"
  GOEXE=""
  GOEXPERIMENT=""
  GOFLAGS=""
  GOHOSTARCH="amd64"
  GOHOSTOS="linux"
  GOINSECURE=""
  GOMODCACHE="/home/katka/go/pkg/mod"
  GONOPROXY=""
  GONOSUMDB=""
  GOOS="linux"
  GOPATH="/home/katka/go"
  GOPRIVATE=""
  GOPROXY="https://proxy.golang.org,direct"
  GOROOT="/usr/local/go"
  GOSUMDB="sum.golang.org"
  GOTMPDIR=""
  GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
  GOVCS=""
  GOVERSION="go1.19.12"
  GCCGO="gccgo"
  GOAMD64="v1"
  AR="ar"
  CC="gcc"
  CXX="g++"
  CGO_ENABLED="1"
  GOMOD="/home/katka/hac/application-service/go.mod"
  GOWORK=""
  CGO_CFLAGS="-g -O2"
  CGO_CPPFLAGS=""
  CGO_CXXFLAGS="-g -O2"
  CGO_FFLAGS="-g -O2"
  CGO_LDFLAGS="-g -O2"
  PKG_CONFIG="pkg-config"
  GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2914814499=/tmp/go-build -gno-record-gcc-switches"

Expected behaviour

BeforeEach runs before every test/interaction.

beforeEach
firstState
secondState
test

Actual behaviour

BeforeEach runs before every state. AFAIK the multiple states during one interaction are supported as stated there.
In my case, beforeEach method is doing a cleanup of test environment. So I run into the situation when the first state creates some resource, but it is immediately cleaned before the second state is executed.

beforeEach
firstState
beforeEach
secnodState
test

Steps to reproduce

Provide a repository, gist or reproducable code snippet so that we can test the problem. You may also want to adapt one of the examples in the repository to demonstrate the problem.
https://github.com/Katka92/application-service/tree/non_working_before_each

Relevent log files

pact.json states:

      "providerStates": [
        {
          "name": "Application exists",
          "params": {
            "appName": "myapp",
            "namespace": "default"
          }
        },
        {
          "name": "Application has components",
          "params": {
            "components": [
              {
                "app": {
                  "appName": "myapp",
                  "namespace": "default"
                },
                "compName": "gh-component",
                "repo": "https://github.com/devfile-samples/devfile-sample-java-springboot-basic"
              }
            ]
          }
        }
      ],

2023-10-25T11:17:38.901569Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: pact_verifier: Executing provider states
2023-10-25T11:17:38.901587Z  INFO ThreadId(02) verify_interaction{interaction="Get app with its components."}: pact_verifier: Running setup provider state change handler 'Application exists' for 'Get app with its components.'
2023-10-25T11:17:38.902326Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: pact_verifier::provider_client: Sending HTTP Request ( method: POST, path: /, query: None, headers: Some({"Content-Type": ["application/json"]}), body: Present(98 bytes, application/json) ) to state change handler
2023-10-25T11:17:38.902387Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: reqwest::connect: starting new connection: http://localhost:33343/    
2023-10-25T11:17:38.902417Z DEBUG tokio-runtime-worker hyper::client::connect::dns: resolving host="localhost"
2023-10-25T11:17:38.902621Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: hyper::client::connect::http: connecting to [::1]:33343
2023-10-25T11:17:38.902743Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: hyper::client::connect::http: connected to [::1]:33343
2023-10-25T11:17:38.902883Z DEBUG tokio-runtime-worker hyper::proto::h1::io: flushed 244 bytes
beforeeach
clean up namespaces
creating application
2023-10-25T11:17:39.166134Z DEBUG tokio-runtime-worker hyper::proto::h1::io: parsed 2 headers
2023-10-25T11:17:39.166161Z DEBUG tokio-runtime-worker hyper::proto::h1::conn: incoming body is empty
2023-10-25T11:17:39.166236Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: hyper::client::pool: pooling idle connection for ("http", localhost:33343)
2023-10-25T11:17:39.166281Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: pact_verifier::provider_client: State change request: Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(33343), path: "/__setup", query: None, fragment: None }, status: 200, headers: {"date": "Wed, 25 Oct 2023 11:17:39 GMT", "content-length": "0"} }
2023-10-25T11:17:39.166426Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: pact_verifier: State Change: "ProviderState { name: "Application exists", params: {"appName": String("myapp"), "namespace": String("default")} }" -> Ok({})
2023-10-25T11:17:39.166444Z  INFO ThreadId(02) verify_interaction{interaction="Get app with its components."}: pact_verifier: Running setup provider state change handler 'Application has components' for 'Get app with its components.'
2023-10-25T11:17:39.167527Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: pact_verifier::provider_client: Sending HTTP Request ( method: POST, path: /, query: None, headers: Some({"Content-Type": ["application/json"]}), body: Present(238 bytes, application/json) ) to state change handler
2023-10-25T11:17:39.167611Z DEBUG ThreadId(02) verify_interaction{interaction="Get app with its components."}: hyper::client::pool: reuse idle connection for ("http", localhost:33343)
2023-10-25T11:17:39.167725Z DEBUG tokio-runtime-worker hyper::proto::h1::io: flushed 385 bytes
beforeeach
clean up namespaces
creating component error "unable to get the Application"

[mefellows]

This is actually mentioned here: https://github.com/pact-foundation/pact-go/blob/master/docs/provider.md#lifecycle-of-a-provider-verification

I'd be open to a PR that can find another way to address this.

Also, thank you very much for taking the type to file a report and produce a repro 🙏

[Katka92]

Ah, I'm sorry I missed that. I probably checked the javascript docs and not the golang.

I wanted to create an issue for AfterEach which is not running when a test fails - is it also known or should I create it?

[Katka92]

And just for the record, I was able to workaround the beforeEach "issue" by using env vars like this:
verifier setup:

	verifyRequest.BeforeEach = func() error {
		if os.Getenv("SETUP") == "true" {
			return nil
		}
		cleanUpNamespaces()
		return nil
	}

and in stateHandler:

func myStateHandlerFunction(setup bool, state models.ProviderState) (models.ProviderStateResponse, error) {
	if !setup {
		os.Setenv("SETUP", "false")
		return nil, nil
	}
	os.Setenv("SETUP", "true")
       ...
}

I know it is not nice but at least solved my issue.

[mefellows]

Thanks for confirming. Yes, this is definitely a way around it, albeit it's not nice.

I'll leave this issue open, perhaps there is a way we can improve this.

@JP-Ellis @uglyog the previous way that hooks were set up in Go (and JS) were attached to the lifecycle of the state handlers. Now that there may be many state handlers per test, it means that these hooks fire more frequently than previously. Is it a bad idea to create some "hook" mechanism in the core that clients can register to?

[rholshausen]

The only way to do "hooks" with FFI is storing pointers to callback functions. This is not very safe. If the parameters differ, for instance, between what the caller expects and what the callback function has, it will result in undefined behavior.

[mefellows]

I was actually thinking a simple hook mechanism could be via HTTP, similar to the way provider states are managed. FFI clients could opt-in to the additional communications and the verifier could publish JSON events to it. Possibly it could be done over sockets or another local interface for better performance if needed.