01 Oct 06:27

russcam

912a15b

7.3.1

7.3.0...7.3.1

Bug Fixes

#4077 Build DebugInformation uri and method from Request or ApiCallDetails (#3687)
#4078 BulkIndexDescriptor IfPrimaryTerm sets IfSequenceNumber.
#4087 Fix fluent serialization of if_primary_term (#4078) Thanks @andrx 👍
#4105 [PropertyName("name", Ignore=true)] ignore not honoured.
#4106 Serialize first precision value on GeoSuggest. Thanks @khalidabuhakmeh 👍
#4108 Honour PropertyNameAttribute Ignore property in mappings
#4112 Fix initial randomization of nodes

Features & Enhancements

#4086 Add Flags property to PatternReplaceCharFilter. Thanks @awelburn 👍
#4090 Add PatternReplaceTokenFilter Flags property

View the full list of issues and PRs

Assets 2

05 Sep 02:51

russcam

7.3.0

9914ccd

7.3.0

7.2.1...7.3.0

Features & Enhancements

#4033 #4046 #4048 Support for search_as_you_type field type and bool_prefix query type
#4053 Add minimum_interval to AutoDateHistogram aggregation
#4054 Implement RareTerms aggregation
#4058 Add support for "flattened" datatype
#4060 Add wildcard and prefix interval rules to Intervals query
#4062 Add FollowerAliasesVersion to follow index shard stats
#4064 Add voting_only to ClusterNodeStats and XPackInfo
#4065 Add x-pack features to XPackUsage and XPackInfo responses
#4066 Add shards_acknowledged and indices results to CloseIndexResponse
#4068 Allow execution of each Watcher action in array with foreach
#4069 Add timing stats to machine learning job stats
#4073 Add support for SourceOnly snapshot repository
#4072 Add metadata to snapshots.

Deprecations

#4056 Deprecate CommonTerms query and CutoffFrequency of Match query
#4071 Add max_docs to ReindexOnServer API and deprecate size

Bug Fixes

#4059 #4057 Use CultureInfo.InvariantCulture when falling back to parsing double
#3857 #4052 Honour StringTimeSpanAttribute to serialize TimeSpan as strings

Misc

#4047 Ensure filters created are deleted
#4061 Regen APIs with v7.3.1 specs
#4075 Skip max_docs query string params

View the full list of issues and PRs

Assets 2

26 Aug 06:09

russcam

7.2.1

37196e5

7.2.1

7.2.0...7.2.1

Features & Enhancements

#4035 Support StringEnumAttribute on a property

Bug Fixes

#4032 Use enum item.Name instead of Enum.GetName() to get enum name. Thank you @sklirg 👍
#4042 Serialize Distance and Percentiles aggregate using an invariant locale. Thank you @rickardp 👍

Misc

#3811 Invert document generation so it is not run by default.
#4039 fix: Remove redundant word from exception message. Thank you @sklirg 👍

View the full list of issues and PRs

Assets 2

19 Aug 01:53

russcam

6.8.2

50a7e34

6.8.2

6.8.1...6.8.2

Features & Enhancements

#4005 Add a CloudConnectionPool IConnectionPool type
This makes it easier than ever to work with Elasticsearch service on Elastic Cloud 🎉
#4002 make sure types that implement DebuggerDisplay override ToString()
#4015 Add if_seq_no and if_primary_term to bulk updates.
#4024 Add Missing method to Date Range Aggregation. thanks @fontourafernando 👍

Bug Fixes

#3977 MoreLikeThis Query not working with artificial document
#3981 JsonNetSerializer doesn't work for GeoLocation
#3997 Serialize GeoLocation with internal serializer
#3999 Fix 6.x query validity in integration tests
#4014 BulkAll() improvements

Misc

#4008 Fix Composite Aggregation integration test
#4022 Delete existing docs when generating new docs

View the full list of issues and PRs

Assets 2

16 Aug 09:10

codebrain

7.2.0

58f806a

7.2.0

7.1.0...7.2.0

Features & Enhancements

#3979 Update API REST specs to 7.2.0
#3851 Map failures on NodeStatistics
#3943 Support for if_seq_no and if_primary_term in the bulk update request
#3963 GetTaskResponse does not expose Failures for reindex task
#3966 Expose Response on GetTaskResponse
#3986 Add 7.2 to the azure devops matrix
#4002 Ensure types that implement DebuggerDisplay override ToString
#4005 Add a CloudConnectionPool
#4015 Add if_seq_no and if_primary_term to bulk updates.
#4024 Add missing property to DateRangeAggregation
#4014 BulkAll() improvements
#3951 BulkAll() requires DefaultIndex or DefaultMappingFor with index
#3983 Implement distance_feature query
#3984 Introduce the columnar option for SQL REST requests.
#3989 AuditTrail integration assertion fix
#3992 Expose external refreshes through the stats API
#3995 Add packaging to cluster stats response.
#3996 Add created_by info to usage stats.
#3998 Add numeric_type sorting option.
#4019 Map node failures.

Deprecations

#3988 Force selection of calendar or fixed intervals in date histo aggregations. Adding [Obsolete] attributes as required.

Bug Fixes

#3970 Bitnami image does not work without DisablePing
#3971 Indices.Exists fails in Full .NET framework, works in .NET Core
#3972 Fix for #3971, change check for mimeType to include empty string
#3981 BUG JsonNetSerializer doesn't work for GeoLocation
#4026 Ensure GeoLocationFormatter returns GeoLocation type
#4027 Normalize exception from cancelling an observable coordinated operation
#4007 Fixed formatting typos (’ --> ')

Docs

#4022 Delete existing docs when generating new docs
#3968 Improve Index name inference docs
#3976 Doc generation should remove docs directory contents

View the full list of issues and PRs

Assets 2

24 Jul 10:27

russcam

6.8.1

178d354

6.8.1

6.8.0...6.8.1

Features

#3752 Add net461 target framework for NEST.JsonNetSerializer nuget package
#3784 Add User-Agent string to API calls to Elasticsearch
#3823 and #3827 Add index name to ICreateIndexResponse

Docs

#3914 and #3969 Improve Index name inference docs

Bug Fixes

#3778 HandleNestTypesOnSourceJsonConverter does not serialize/deserialize Nest.DateRange
#3819 and #3820 Use CultureInfo.InvariantCulture when writing with GeoWKTWriter

View the full list of issues and PRs

Assets 2

19 Jul 07:57

codebrain

7.1.0

c3f2796

7.1.0

7.0.1...7.1.0

Features & Enhancements

#3843, #3919 Implement Script Score Query
#3911 Include size of snapshot in snapshot metadata
#3912 Implement Api Key authorisation on requests
#3927 Add CCR Follow Info API
#3932 Implement GeoTileGrid aggregation
#3933 date_nanos type implementation
#3934 Implement adjust_offsets on word delimiter graph token filter
#3937 Additions to Invalidate Token API
#3938, #3941 Add rank_feature, rank_features field datatypes and rank_feature query
#3939 Implement Forget Follower API
#3940 Implement Freeze and Unfreeze API

Bug Fixes

#3899 Anchored DateMath no longer serializes time zone information for local DateTime
#3904 Using weight parameter in script_score resulted in an invalid request
#3905 TypeMappingDescriptor<T> still supports setting AllField
#3906 TrackTotalHits is ignored during MultiSearch call
#3907 UseSocketsHttpHandler fix
#3909 Serialize only script in script_score function
#3910 Include time zone information for Local and Utc DateTimeKind
#3917 Add track_total_hits to search request body
#3918 Obsolete AllField and IndexField
#3920 Issue in deserializing WKT Geoshape
#3922 Support numbers with exponent in WKT
#3926 JsonSerializerSettings not honored for Field Expressions
#3936 Exception thrown when using cat APIs with nonexistent index
#3942 Honour IPropertyMappingProvider on custom IElastisearchSerializer
#3944 Handle cat API 404 responses
#3945 NEST requires explicit installation of System.Diagnostics.DiagnosticSource nuget package.
#3948 Add System.Diagnostics.DiagnosticSource reference for NetStandard 2.0
#3952 Transfer-Encoding: chunked is enabled by default
#3958 Fix default chunked transfer encoding and expose as configurable

Deprecations

#3905 TypeMappingDescriptor still supports setting AllField

View the full list of issues and PRs

Assets 2

05 Jul 04:43

russcam

7.0.1

a2ac5c0

7.0.1

7.0.0...7.0.1

Bug Fixes

#3913 Enabling http compression results in empty http request body (PR #3921)

View the full list of issues and PRs

Assets 2

27 Jun 14:17

russcam

7.0.0

00eaaa9

7.0.0

7.0.0-beta1...7.0.0

NEST & Elasticsearch.Net 7.0: Now GA!

After many months of work, two alphas and a beta, we are pleased to announce the GA release of the NEST and Elasticsearch.Net 7.0 clients.

The overall themes of this release have been based around faster serialization, performance improvements, codebase simplification, and ensuring parity with the many new features available in Elasticsearch 7.0.

Types removal

Specifying types within the .NET clients is now deprecated in 7.0, in line with the overall Elasticsearch type removal strategy.

In instances where your index contains type information and you need to preserve this information, one recommendation is to introduce a property to describe the document type (similar to a table per class with discriminator field in the ORM world) and then implement a custom serialization / deserialization implementation for that class.

This Elasticsearch page details some other approaches.

Faster Serialization

After internalizing the serialization routines, and IL-merging the Newtonsoft.Json package in 6.x, we are pleased to announce that the next stage of serialization improvements have been completed in 7.0.

Both SimpleJson and Newtonsoft.Json have been completely removed and replaced with an implementation of Utf8Json, a fast serializer that works directly with UTF-8 binary. This has yielded a significant performance improvement, which we will be sharing in more detail in a later blog post.

With the move to Utf8Json, we have removed some features that were available in the previous JSON libraries that have proven too onerous to carry forward at this stage.

JSON in the request is never indented, even if SerializationFormatting.Indented is specified. The serialization routines generated by Utf8Json never generate an IJsonFormatter<T> that will indent JSON, for performance reasons. We are considering options for exposing indented JSON for development and debugging purposes.
NEST types cannot be extended by inheritance. With NEST 6.x, additional properties can be included for a type by deriving from that type and annotating these new properties. With the current implementation of serialization with Utf8Json, this approach will not work.
Serializer uses Reflection.Emit. Utf8Json uses Reflection.Emit to generate efficient formatters for serializing types that it sees. Reflection.Emit is not supported on all platforms, for example, UWP, Xamarin.iOS, and Xamarin.Android.
Elasticsearch.Net.DynamicResponse deserializes JSON arrays to List<object>. SimpleJson deserialized JSON arrays to object[], but Utf8Json deserializes them to List<object>. This change is preferred for allocation and performance reasons.
Utf8Json is much stricter when deserializing JSON object field names to C# POCO properties. With the internal Json.NET serializer in 6.x, JSON object field names would attempt to be matched with C# POCO property names first by an exact match, falling back to a case insensitive match. With Utf8Json in 7.x however, JSON object field names must match exactly the name configured for the C# POCO property name.

We believe that the trade-off of features vs. GA release has been worthwhile at this stage. We hold a view to address some of these missing features in a later release.

High to Low level client dispatch changes

In 6.x, the process of an API call within NEST looked roughly like this

client.Search()
  => Dispatch()
    => LowLevelDispatch.SearchDispatch()
      => lowlevelClient.Search()
        => lowlevelClient.DoRequest()

With 7.x, this process has been changed to remove dispatching to the low-level client methods. The new process looks like this

client.Search()
  => lowlevelClient.DoRequest()

This means that in the high-level client IRequest now builds its own URLs, with the upside that the call chain is shorter and allocates fewer closures. The downside is that there are now two URL building mechanisms, one in the low-level client and a new one in the high-level client. In practice, this area of the codebase is kept up to date via code generation, so it does not place any additional burden on development.

Given the simplified call chain and debugging experience, we believe this is an improvement worth making.

Namespaced API methods and Upgrade Assistant

As the API surface of Elasticsearch has grown to well over 200 endpoints, so has the number of client methods exposed, leading to an almost overwhelming number to navigate and explore through in an IDE. This is further exacerbated by the fact that the .NET client exposes both synchronous and asynchronous API methods for both the fluent API syntax as well as the object initializer syntax.

To address this, the APIs are now accessible through sub-properties on the client instance.

For example, in 6.x, to create a machine learning job

var putJobResponse = client.PutJob<Metric>("id", c => c
    .Description("Lab 1 - Simple example")
    .ResultsIndexName("server-metrics")
	.AnalysisConfig(a => a
        .BucketSpan("30m")
        .Latency("0s")
    	.Detectors(d => d.Sum(c => c.FieldName(r => r.Total)))
	)
	.DataDescription(d => d.TimeField(r => r.Timestamp))
);

This has changed to

var putJobResponse = client.MachineLearning.PutJob<Metric>("id", c => c
    .Description("Lab 1 - Simple example")
    .ResultsIndexName("server-metrics")
	.AnalysisConfig(a => a
        .BucketSpan("30m")
        .Latency("0s")
    	.Detectors(d => d.Sum(c => c.FieldName(r => r.Total)))
	)
	.DataDescription(d => d.TimeField(r => r.Timestamp))
);

Notice the client.MachineLearning.PutJob method call in 7.0, as opposed to client.PutJob in 6.x.

We believe this grouping of functionality leads to a better discoverability experience when writing your code, and improved readability when reviewing somebody else's.

The Upgrade Assistant

To assist developers in migrating from 6.x, we have published the Nest.7xUpgradeAssistant Nuget package. When included in your project and the using Nest.ElasticClientExtensions; directive is added, calls will be redirected from the old API method names to the new API method names in 7.0. The result is that your project will compile and you won't need to immediately update your code to use the namespaced methods; instead you'll see compiler warnings indicating the location of the new API methods in 7.0.

This package is to assist developers migrating from 6.x to 7.0 and is limited in scope to this purpose. It is recommended that you observe the compiler warnings and adjust your code as indicated.

Observability and DiagnosticSource

7.0 introduces emitting System.Diagnostics.DiagnosticSource information from the client, during a request. The goal is to enable rich information exchange with the Elastic APM .NET agent and other monitoring libraries.

We emit DiagnosticSource information for key parts of a client request via an Activity event, shipping support for Id, ParentId, RootId, as well as request Duration.

To facilitate wiring this up to DiagnosticListener.AllListeners, we ship both with static access to the publisher names and events through Elasticsearch.Net.Diagnostics.DiagnosticSources as well as strongly typed listeners, removing the need to cast the object passed to activity start/stop events.

An example listener implementation that writes events to the console is given below

private class ListenerObserver : IObserver<DiagnosticListener>
{
  public void OnCompleted() => Console.WriteLine("Completed");
  
  public void OnError(Exception error) => Console.Error.WriteLine(error.Message);
	
  public void OnNext(DiagnosticListener value)
  {
		void WriteToConsole<T>(string eventName, T data)
		{
			var a = Activity.Current;
			Console.WriteLine($"{eventName?.PadRight(30)} {a.Id?.PadRight(32)} {a.ParentId?.PadRight(32)} {data?.ToString().PadRight(10)}");
		}
		if (value.Name == DiagnosticSources.AuditTrailEvents.SourceName)
			value.Subscribe(new AuditDiagnosticObserver(v => WriteToConsole(v.Key, v.Value)));

		if (value.Name == DiagnosticSources.RequestPipeline.SourceName)
			value.Subscribe(new RequestPipelineDiagnosticObserver(
				v => WriteToConsole(v.Key, v.Value),
				v => WriteToConsole(v.Key, v.Value))
			);

		if (value.Name == DiagnosticSources.HttpConnection.SourceName)
			value.Subscribe(new HttpConnectionDiagnosticObserver(
				v => WriteToConsole(v.Key, v.Value),
				v => WriteToConsole(v.Key, v.Value)
			));

		if (value.Name == DiagnosticSources.Serializer.SourceName)
			value.Subscribe(new SerializerDiagnosticObserver(v => WriteToConsole(v.Key, v.Value)));
  }
}

Using the following example

var pool = new SniffingConnectionPool(new[] { node.NodesUris().First() });
var settings = new ConnectionSettings(pool).SniffOnStartup();
var client = new ElasticClient(settings);
var x = client.Search<object>(s=>s.AllIndices());
               	
Console.WriteLine(new string('-', Console.WindowWidth - 1));
                                  	
var y = client.Search<object>(s=>s.Index("does-not-exist"));

Emits the following console output:

SniffOnStartup.Start       	|59e275e-4f9c835d189eb14a.    	Event: SniffOnStartup
Sniff.Start                	|59e275e-4f9c835d189eb14a.1. 	|59e275e-4f9c835d189eb14a.   	GET _nodes/http,settings
Sniff.Start                	|59e...

Assets 2

12 Jun 13:11

codebrain

7.0.0-beta1

a9b2fb1

7.0.0-beta1 Pre-release

Pre-release

7.0.0-alpha2...7.0.0-beta1

Following the advice given in 7.0.0-alpha1 the namespacing change has been implemented for this release.

Namespaced API methods

In both Elasticsearch.Net and NEST clients, API methods are made available on the root of the client. As the API surface of Elasticsearch has grown to well over 200 endpoints, so has the number of API methods exposed, leading to an almost overwhelming number of methods to navigate through to choose the method aligning with the desired API call. This is further exacerbated by the fact that the client exposes synchronous and asynchronous API methods for both the fluent API syntax as well as the object initializer syntax.

To address this, the APIs are now accessible through sub-properties on the client instance.

For example, in 6.x, to create a Machine Learning job the code is:

var putJobResponse = client.PutJob<Metric>("id", c => c
    .Description("Lab 1 - Simple example")
    .ResultsIndexName("server-metrics")
    .AnalysisConfig(a => a
        .BucketSpan("30m")
        .Detectors(d => d.Sum(c => c.FieldName(r => r.Total)))
    )
    .DataDescription(d => d.TimeField(r => r.Timestamp));
);

This has changed to:

var putJobResponse = client.MachineLearning.PutJob<Metric>("id", c => c
    .Description("Lab 1 - Simple example")
    .ResultsIndexName("server-metrics")
    .AnalysisConfig(a => a
        .BucketSpan("30m")
        .Detectors(d => d.Sum(c => c.FieldName(r => r.Total)))
    )
    .DataDescription(d => d.TimeField(r => r.Timestamp));
);

Notice the client.MachineLearning.PutJob method call in 7.0.0-beta1, as opposed to client.PutJob in 6.x

Features & Enhancements

#3809 Add overloads of SerializeToString and SerializeToBytes
#3765 Clean up GET /_cluster/stats
#3807 Use indexers to set default mapping values
#3806 User SecureString for Basic and Proxy passwords

Bug Fixes

#3771 Breaking formatter backwards compatibility
#3805 Routing must be specified when doc used
#3706 Calling ConnectionSettings.DefaultMappingFor throws ArgumentException when the Type has been previously added
#3719 Wrong converting datetime fields
#3751 Unable to get raw document JSON without deserializing

View the full list of issues and PRs

Assets 2

Releases: elastic/elasticsearch-net

7.3.1

Bug Fixes

Features & Enhancements

Uh oh!

7.3.0

Features & Enhancements

Deprecations

Bug Fixes

Misc

Uh oh!

7.2.1

Features & Enhancements

Bug Fixes

Misc

Uh oh!

6.8.2

Features & Enhancements

Bug Fixes

Misc

Uh oh!

7.2.0

Features & Enhancements

Deprecations

Bug Fixes

Docs

Uh oh!

6.8.1

Features

Docs

Bug Fixes

Uh oh!

7.1.0

Features & Enhancements

Bug Fixes

Deprecations

Uh oh!

7.0.1

Bug Fixes

Uh oh!

7.0.0

NEST & Elasticsearch.Net 7.0: Now GA!

Types removal

Faster Serialization

High to Low level client dispatch changes

Namespaced API methods and Upgrade Assistant

The Upgrade Assistant

Observability and DiagnosticSource

Uh oh!

7.0.0-beta1

Namespaced API methods

Features & Enhancements

Bug Fixes

Uh oh!