Skip to content

Add ListResource RPC #1157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jun 18, 2025
Merged
14 changes: 7 additions & 7 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ toolchain go1.23.7

require (
github.com/google/go-cmp v0.7.0
github.com/hashicorp/terraform-plugin-go v0.28.0
github.com/hashicorp/terraform-plugin-go v0.28.1-0.20250616135123-a19df43120ea
github.com/hashicorp/terraform-plugin-log v0.9.0
)

Expand All @@ -16,7 +16,7 @@ require (
github.com/hashicorp/go-hclog v1.5.0 // indirect
github.com/hashicorp/go-plugin v1.6.3 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hashicorp/terraform-registry-address v0.2.5 // indirect
github.com/hashicorp/terraform-registry-address v0.3.0 // indirect
github.com/hashicorp/terraform-svchost v0.1.1 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/mattn/go-colorable v0.1.12 // indirect
Expand All @@ -25,10 +25,10 @@ require (
github.com/oklog/run v1.0.0 // indirect
github.com/vmihailenco/msgpack/v5 v5.4.1 // indirect
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
golang.org/x/net v0.39.0 // indirect
golang.org/x/sys v0.32.0 // indirect
golang.org/x/text v0.24.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a // indirect
google.golang.org/grpc v1.72.1 // indirect
golang.org/x/net v0.41.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.26.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250324211829-b45e905df463 // indirect
google.golang.org/grpc v1.73.0 // indirect
google.golang.org/protobuf v1.36.6 // indirect
)
48 changes: 24 additions & 24 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ github.com/hashicorp/go-plugin v1.6.3 h1:xgHB+ZUSYeuJi96WtxEjzi23uh7YQpznjGh0U0U
github.com/hashicorp/go-plugin v1.6.3/go.mod h1:MRobyh+Wc/nYy1V4KAXUiYfzxoYhs7V1mlH1Z7iY2h0=
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/terraform-plugin-go v0.28.0 h1:zJmu2UDwhVN0J+J20RE5huiF3XXlTYVIleaevHZgKPA=
github.com/hashicorp/terraform-plugin-go v0.28.0/go.mod h1:FDa2Bb3uumkTGSkTFpWSOwWJDwA7bf3vdP3ltLDTH6o=
github.com/hashicorp/terraform-plugin-go v0.28.1-0.20250616135123-a19df43120ea h1:U9EAAeQtszGlR7mDS7rY77B/a4/XiMDB8HfAtqLAuAQ=
github.com/hashicorp/terraform-plugin-go v0.28.1-0.20250616135123-a19df43120ea/go.mod h1:hL//wLEfYo0YVt0TC/VLzia/ADQQto3HEm4/jX2gkdY=
github.com/hashicorp/terraform-plugin-log v0.9.0 h1:i7hOA+vdAItN1/7UrfBqBwvYPQ9TFvymaRGZED3FCV0=
github.com/hashicorp/terraform-plugin-log v0.9.0/go.mod h1:rKL8egZQ/eXSyDqzLUuwUYLVdlYeamldAHSxjUFADow=
github.com/hashicorp/terraform-registry-address v0.2.5 h1:2GTftHqmUhVOeuu9CW3kwDkRe4pcBDq0uuK5VJngU1M=
github.com/hashicorp/terraform-registry-address v0.2.5/go.mod h1:PpzXWINwB5kuVS5CA7m1+eO2f1jKb5ZDIxrOPfpnGkg=
github.com/hashicorp/terraform-registry-address v0.3.0 h1:HMpK3nqaGFPS9VmgRXrJL/dzHNdheGVKk5k7VlFxzCo=
github.com/hashicorp/terraform-registry-address v0.3.0/go.mod h1:jRGCMiLaY9zii3GLC7hqpSnwhfnCN5yzvY0hh4iCGbM=
github.com/hashicorp/terraform-svchost v0.1.1 h1:EZZimZ1GxdqFRinZ1tpJwVxxt49xc/S52uzrw4x0jKQ=
github.com/hashicorp/terraform-svchost v0.1.1/go.mod h1:mNsjQfZyf/Jhz35v6/0LWcv26+X7JPS+buii2c9/ctc=
github.com/hashicorp/yamux v0.1.1 h1:yrQxtgseBDrq9Y652vSRDvsKCJKOUD+GzTS4Y0Y8pvE=
Expand Down Expand Up @@ -56,32 +56,32 @@ github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAh
github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/otel v1.34.0 h1:zRLXxLCgL1WyKsPVrgbSdMN4c0FMkDAskSTQP+0hdUY=
go.opentelemetry.io/otel v1.34.0/go.mod h1:OWFPOQ+h4G8xpyjgqo4SxJYdDQ/qmRH+wivy7zzx9oI=
go.opentelemetry.io/otel/metric v1.34.0 h1:+eTR3U0MyfWjRDhmFMxe2SsW64QrZ84AOhvqS7Y+PoQ=
go.opentelemetry.io/otel/metric v1.34.0/go.mod h1:CEDrp0fy2D0MvkXE+dPV7cMi8tWZwX3dmaIhwPOaqHE=
go.opentelemetry.io/otel/sdk v1.34.0 h1:95zS4k/2GOy069d321O8jWgYsW3MzVV+KuSPKp7Wr1A=
go.opentelemetry.io/otel/sdk v1.34.0/go.mod h1:0e/pNiaMAqaykJGKbi+tSjWfNNHMTxoC9qANsCzbyxU=
go.opentelemetry.io/otel/sdk/metric v1.34.0 h1:5CeK9ujjbFVL5c1PhLuStg1wxA7vQv7ce1EK0Gyvahk=
go.opentelemetry.io/otel/sdk/metric v1.34.0/go.mod h1:jQ/r8Ze28zRKoNRdkjCZxfs6YvBTG1+YIqyFVFYec5w=
go.opentelemetry.io/otel/trace v1.34.0 h1:+ouXS2V8Rd4hp4580a8q23bg0azF2nI8cqLYnC8mh/k=
go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE=
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ=
go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y=
go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M=
go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE=
go.opentelemetry.io/otel/sdk v1.35.0 h1:iPctf8iprVySXSKJffSS79eOjl9pvxV9ZqOWT0QejKY=
go.opentelemetry.io/otel/sdk v1.35.0/go.mod h1:+ga1bZliga3DxJ3CQGg3updiaAJoNECOgJREo9KHGQg=
go.opentelemetry.io/otel/sdk/metric v1.35.0 h1:1RriWBmCKgkeHEhM7a2uMjMUfP7MsOF5JpUCaEqEI9o=
go.opentelemetry.io/otel/sdk/metric v1.35.0/go.mod h1:is6XYCUMpcKi+ZsOvfluY5YstFnhW0BidkR+gL+qN+w=
go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs=
go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc=
golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a h1:51aaUVRocpvUOSQKM6Q7VuoaktNIaMCLuhZB6DKksq4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a/go.mod h1:uRxBH1mhmO8PGhU89cMcHaXKZqO+OfakD8QQO0oYwlQ=
google.golang.org/grpc v1.72.1 h1:HR03wO6eyZ7lknl75XlxABNVLLFc2PAb6mHlYh756mA=
google.golang.org/grpc v1.72.1/go.mod h1:wH5Aktxcg25y1I3w7H69nHfXdOG3UiadoBtjh3izSDM=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250324211829-b45e905df463 h1:e0AIkUUhxyBKh6ssZNrAMeqhA7RKUj42346d1y02i2g=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250324211829-b45e905df463/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
google.golang.org/grpc v1.73.0 h1:VIWSmpI2MegBtTuFt5/JWy2oXxtjJ/e89Z70ImfD2ok=
google.golang.org/grpc v1.73.0/go.mod h1:50sbHOUqWoCQGI8V2HQLJM0B+LMlIUjNSZmow7EVBQc=
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
Expand Down
4 changes: 4 additions & 0 deletions internal/fwschemadata/data_description.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ const (
// a plan-based value.
DataDescriptionPlan DataDescription = "plan"

// DataDescriptionResource is used for Data that represents
// a resource value.
DataDescriptionResource DataDescription = "resource"

// DataDescriptionState is used for Data that represents
// a state-based value.
DataDescriptionState DataDescription = "state"
Expand Down
10 changes: 10 additions & 0 deletions internal/fwserver/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,16 @@ type Server struct {
// access from race conditions.
listResourceFuncsMutex sync.Mutex

// listResourceSchemas is the cached ListResource Schemas for RPCs that
// need to convert configuration data from the protocol. If not found, it
// will be fetched from the [list.ListResource.ListResourceConfigSchema]
// method.
listResourceSchemas map[string]fwschema.Schema

// listResourceSchemasMutex is a mutex to protect concurrent
// listResourceSchemas access from race conditions.
listResourceSchemasMutex sync.RWMutex

// providerSchema is the cached Provider Schema for RPCs that need to
// convert configuration data from the protocol. If not found, it will be
// fetched from the Provider.GetSchema() method.
Expand Down
147 changes: 147 additions & 0 deletions internal/fwserver/server_listresource.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0

package fwserver

import (
"context"
"errors"
"iter"

"github.com/hashicorp/terraform-plugin-framework/diag"
"github.com/hashicorp/terraform-plugin-framework/internal/fwschema"
"github.com/hashicorp/terraform-plugin-framework/internal/logging"
"github.com/hashicorp/terraform-plugin-framework/list"
"github.com/hashicorp/terraform-plugin-framework/tfsdk"
)

// ListRequest is the framework server request for the ListResource RPC.
type ListRequest struct {
// ListResource is an instance of the provider's list resource
// implementation for a specific managed resource type.
ListResource list.ListResource

// Config is the configuration the user supplied for listing resource
// instances.
Config *tfsdk.Config

// IncludeResource indicates whether the provider should populate the
// Resource field in the ListResult struct.
IncludeResource bool

ResourceSchema fwschema.Schema
ResourceIdentitySchema fwschema.Schema
}

// ListResultsStream represents a streaming response to a [ListRequest]. An
// instance of this struct is supplied as an argument to the provider's List
// function. The provider should set a Results iterator function that pushes
// zero or more results of type [ListResult].
//
// For convenience, a provider implementation may choose to convert a slice of
// results into an iterator using [slices.Values].
type ListResultsStream struct {
// Results is a function that emits [ListResult] values via its push
// function argument.
Results iter.Seq[ListResult]
}

func ListResultError(summary string, detail string) ListResult {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: add a doc comment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's not a "publicly" exported field, no need to add a doc comment unless you feel this helper needs more explanation for maintainers.

Although I'm wondering if this function even needs to be exported (or even exist, it's only used once ATM, maybe it will be used more later?)

return ListResult{
Diagnostics: diag.Diagnostics{
diag.NewErrorDiagnostic(summary, detail),
},
}
}

// ListResult represents a listed managed resource instance.
type ListResult struct {
// Identity is the identity of the managed resource instance. A nil value
// will raise an error diagnostic.
Identity *tfsdk.ResourceIdentity

// Resource is the provider's representation of the attributes of the
// listed managed resource instance.
//
// If [ListRequest.IncludeResource] is true, a nil value will raise
// a warning diagnostic.
Resource *tfsdk.Resource

// DisplayName is a provider-defined human-readable description of the
// listed managed resource instance, intended for CLI and browser UIs.
DisplayName string

// Diagnostics report errors or warnings related to the listed managed
// resource instance. An empty slice indicates a successful operation with
// no warnings or errors generated.
Diagnostics diag.Diagnostics
}

var NoListResults = func(func(ListResult) bool) {}

// ListResource implements the framework server ListResource RPC.
func (s *Server) ListResource(ctx context.Context, fwReq *ListRequest, fwStream *ListResultsStream) error {
listResource := fwReq.ListResource

if fwReq.Config == nil {
fwStream.Results = NoListResults
return errors.New("Invalid ListResource request: Config cannot be nil")
}

req := list.ListRequest{
Config: *fwReq.Config,
IncludeResource: fwReq.IncludeResource,
ResourceSchema: fwReq.ResourceSchema,
ResourceIdentitySchema: fwReq.ResourceIdentitySchema,
}

stream := &list.ListResultsStream{}

logging.FrameworkTrace(ctx, "Calling provider defined ListResource")
listResource.List(ctx, req, stream)
logging.FrameworkTrace(ctx, "Called provider defined ListResource")

// If the provider returned a nil results stream, we return an empty stream.
if stream.Results == nil {
stream.Results = list.NoListResults
}

fwStream.Results = processListResults(req, stream.Results)
return nil
}

func processListResults(req list.ListRequest, stream iter.Seq[list.ListResult]) iter.Seq[ListResult] {
return func(push func(ListResult) bool) {
for result := range stream {
if !push(processListResult(req, result)) {
return
}
}
}
}

// processListResult validates the content of a list.ListResult and returns a
// ListResult
func processListResult(req list.ListRequest, result list.ListResult) ListResult {
if result.Diagnostics.HasError() {
return ListResult(result)
}

if result.Identity == nil { // TODO: is result.Identity.Raw.IsNull() a practical concern?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the question would: does Terraform core care if the provider returns a null identity? I would say we probably shouldn't return a null identity unless core says that's acceptable.

In which case we should just check both:

  • result.Identity == nil being the provider likely didn't set anything
  • result.Identity.Raw.IsNull() being the provider explicitly set a null value

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does Terraform core care if the provider returns a null identity? I would say we probably shouldn't return a null identity unless core says that's acceptable.

cc: @dbanck @dsa0x for input

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Terraform currently raises an error when a list result has no identity

return ListResultError(
"Incomplete List Result",
"The provider did not populate the Identity field in the ListResourceResult. This may be due to an error in the provider's implementation.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: We can probably use more guided language since it's almost certainly a bug in the provider and should be reported there first:

Something to the tune of -> This is always a problem with the provider. Please report to the provider developer

Copy link
Contributor Author

@bbasata bbasata Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Updating.

)
}

if req.IncludeResource {
if result.Resource == nil { // TODO: is result.Resource.Raw.IsNull() a practical concern?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the above comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub notifications aren't working well today, I hope I don't miss any mentions.

Ideally a provider should always send a resource when include resource is true, but Terraform doesn't enforce that

result.Diagnostics.AddWarning(
"Incomplete List Result",
"The provider did not populate the Resource field in the ListResourceResult. This may be due to the provider not supporting this functionality or an error in the provider's implementation.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps future PR, but if this is a warning that practitioners are likely to see often, maybe we should tailor it specifically to them (i.e. not using the Go types since they have no reference of what these fields mean and won't be reporting to provider developers)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, and +1 to a future PR for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 I'm less certain of the guidance for a nil Resource warning. It's a warning, after all. Is it actionable by the user? Open to suggestions.

)
}
}

return ListResult(result)
}
Loading