Skip to content

Move machine learning to the native OpenSearch client - #143

Open
esmarkowski wants to merge 3 commits into
feature/decouple-model-layer-from-elasticsearch-persistencefrom
feature/move-machine-learning-to-native-opensearch-client
Open

esmarkowski wants to merge 3 commits into
feature/decouple-model-layer-from-elasticsearch-persistencefrom
feature/move-machine-learning-to-native-opensearch-client

Conversation

@esmarkowski

@esmarkowski esmarkowski commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

ML models and connectors now call opensearch-ruby 4's client.ml directly instead of the vendored lib/opensearch and lib/elasticsearch action copies, which are removed. Models, connectors and the new model groups share one Resource concern for their settings DSL and the remote id kept in the registry.

class Embeddings < Stretchy::MachineLearning::ModelGroup
  access_mode "public"
end
Embeddings.register!

class SentenceModel < Stretchy::MachineLearning::Model
  model :sentence_transformers_minilm_12
  model_group :embeddings
end
SentenceModel.register
SentenceModel.deploy

Breaking changes:

  • ML Commons is OpenSearch-only.
  • register no longer deploys. Call deploy after registering.
  • deployed? checks the model state on every call instead of caching it.
  • The Gemfile pins json < 3 until opensearch-ruby 4 stops passing parse options positionally.

Specs pass on OpenSearch 3.9.0, OpenSearch 2.19.7 and Elasticsearch 8.12.2.

Closes #136

Spencer Markowski and others added 3 commits November 16, 2026 16:50
Stretchy::MachineLearning::Resource gives models and connectors the same
class-level `settings` declaration and the same registry-backed remote
id, replacing the Connector::Settings method_missing delegation and the
METHODS loop in Model.

The registry gains a `remote_id` attribute, falling back to `model_id`
for entries written before it existed, so connector ids no longer live
in a field named for models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
opensearch-ruby 4 ships client.ml, so the vendored lib/opensearch and
lib/elasticsearch action copies are removed and Model and Connector call
register_model, deploy_model, create_connector, etc. directly.

Breaking changes:
- ML Commons is OpenSearch-only; the Elasticsearch ML and _connector copies are gone
- register no longer deploys; call deploy after registering
- deployed? queries the model state on every call
- query string searches are sent as a query_string body
- json < 3 is pinned in the Gemfile until opensearch-ruby 4 stops passing parse options positionally

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ModelGroup registers an ML Commons model group and remembers its id in the
registry. Models declare `model_group :embeddings` to register into it.
Resource gains `remote_name`, the name sent to the cluster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@esmarkowski
esmarkowski added this pull request to stack #142 November 16, 2026 22:08
@esmarkowski esmarkowski changed the title feature/move machine learning to native opensearch client Move machine learning to the native OpenSearch client Nov 16, 2026
@esmarkowski
esmarkowski marked this pull request as ready for review November 16, 2026 22:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move machine learning to the native OpenSearch client

1 participant