Unexpected argument when trying to iterate

[paulheijman]

Hi,

I need to iterate over a large dataset, so according to documentation I need to use --iterate-by and a corresponding --query.
So I tried the following command:

solrcopy backup --query 'tdate:[{begin} TO {end}]' --iterate-by day --between '1970-01-01' '2071-01-01'

but this fails:
error: unexpected argument 'TO' found

Tested on Linux and Windows with Solrcopy 0.8.1

Could you please tell me what I'm doing wrong?
Thanks in advance!

[juarezr]

Hi @paulheijman ,

Could you please tell me what I'm doing wrong?

At first glance, I can't tell you if it's a mistake or a bug. However, we can investigate the issue further.

I have a couple of questions to divide the scope of the problem:

• Can you tell me what Solr version you are using?
• What happens if you run a query in the Solr web UI using a one-day range like: tdate:[2026-02-03 TO 2026-02-04]?
• What happens if you pass the argument --query 'tdate:["{begin}" TO "{end}"]' ?
  • In this case, solrcopy is filling the {begin} and {end} placeholders without any quotes.
  • If my guess is correct, it may be an error message that comes from Solr itself, due to an invalid query.
  • And I should update the instructions in the documentation example of solrcopy to the correct quoted form.

--iterate-by day --between '1970-01-01' '2071-01-01'

Also, consider:

• The number of days in a century (36890) is quite big.
  • In this case, solrcopy is slicing your core (no pun intended) by sending multiple queries and generating multiple archives.
  • Although solrcopy should handle it, you may end up with a lot of small archives in the local filesystem.
• You may further tweak it by:
• Increasing the number of days for each step by setting the --step argument.
• Choosing a smaller date range like: --between '2020-01-01' '2026-03-01'.

I'll investigate it further as soon as I get some spare time.

[juarezr]

Hi @paulheijman,

Can you check if the v0.9.0 release solves your problem?
Please let me know if the problem persists, and what Solr version and configuration you are using.