Skip to content

Clouseau: Separate write folders for each instance#6059

Open
poor-a wants to merge 2 commits into
apache:mainfrom
poor-a:clouseau/isolated-index-folders
Open

Clouseau: Separate write folders for each instance#6059
poor-a wants to merge 2 commits into
apache:mainfrom
poor-a:clouseau/isolated-index-folders

Conversation

@poor-a

@poor-a poor-a commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Overview

The default Clouseau configuration in ./dev/run results in all Clouseau instances trying to write index files to the same folder. That creates a race condition. Observable symptom of the race condition is the following curl output:

$ curl -v -X GET --location 'http://admin:secret@localhost:25984/movies/_design/by-director-search/_search/by-director?query=director_3astring:*&include_docs=true'

Output:

{"error":"unknown_error","reason":"Lock obtain timed out: NativeFSLock@/Users/arturpoor/repo/couchdb/target/indexes/shards/00000000-7fffffff/movies.1782896092/c146b6fb754fa06bbbcbf78855ddb020/write.lock"}
* Connection #0 to host localhost left intact

Checklist

  • This is my own work, I did not use AI, LLM's or similar technology
  • Code is written and works correctly
  • Changes are covered by tests
  • Any new configurable parameters are documented in rel/overlay/etc/default.ini
  • Documentation changes were made in the src/docs folder
  • Documentation changes were backported (separated PR) to affected branches

@pgj

pgj commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

I do not think that is needed. The index directories are configured ad hoc via overrides to the configuration by Java properties, i.e. by passing -Dclouseau.dir=<path>. Please note that dev/run script still does that and it is Clouseau 3.x that ignores that. So instead of touching dev/run we should fix this regression in the Clouseau source code itself.

@poor-a

poor-a commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

The index directories are configured ad hoc via overrides to the configuration by Java properties, i.e. by passing -Dclouseau.dir=

That is also a viable option. IMHO it's better to keep all configuration settings in one place (e.g. the clouseau.conf config file) for easier understanding. It's more difficult to follow if configuration is scattered across several files in the repo.

@poor-a

poor-a commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Also the CouchDB database files are located under dev/lib/node{n} - it makes sense to keep the Clouseau index files there as well.

Comment thread dev/run
domain: 127.0.0.1
}
clouseau: {
dir: dev/lib/node1/indexes

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gabor's point aside for a moment, this path needs to use the ctx["devdir"] variable rather than assume 'dev/'

@rnewson

rnewson commented Jul 2, 2026

Copy link
Copy Markdown
Member

I agree with gabor that the -Dclouseau.dir behaviour in clouseau should be restored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants