Skip to content

Batch import doesn't apply rate limit when using async indexing #1542

@rlmanrique

Description

@rlmanrique

Using the Python client in its version 4.10.4

The function rate_limit is not working if Weaviate is configured in async indexing mode:

    with open(dataset_path, "r") as f:
        collection.batch.rate_limit(100)
        with collection.batch.dynamic() as batch:
            for raw_json_email in f:
                email = json.loads(raw_json_email)
                batch.add_object(
                    properties=email["data"],
                    uuid=email["id"],
                )

The batches have 1000 objects, no mater the rate configured.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions