CondaPkg.toml in test directory

[icweaver]

Would it be possible to just have this file in the test directory for a package? It seems to only be found when it lives in the parent directory when running pkg> test on files that use PythonCall here. I was hoping to just have all Python related files isolated to the test directory if that seems reasonable

[cjdoris]

Do you mean that you want to declare some Conda dependencies which are only installed when running the tests? That would be reasonable.

I'm not sure exactly how Julia sets up the test environment. You can make the test folder be its own project (i.e. put a Project.toml there), maybe if you do that then it will get included into the load_path, and then CondaPkg will find dependencies in that folder. Maybe?

[icweaver]

Thank you for the workaround. After using CondaPkg.jl (and Julia's test harness) a little bit more now, I think I was mistaken when saying that PythonCall.jl could not find the CondaPkg.toml if it's stored in test. I think it's just not even checking there in the first place? LOAD_PATH and Base.load_path() seem to be automatically updated to the new tmp directory when running (MyPkg) pkg> test:

# @show LOAD_PATH, and @show Base.load_path() inside of test/runtests.jl

LOAD_PATH = ["@", "/tmp/jl_K84EE9"]
Base.load_path() = ["/tmp/jl_K84EE9/Project.toml"]

IIUC, if CondaPkg.jl does not see CondaPkg.toml here, then it will just create its own when making its own environment. Could an extra conditional be added to check for a test folder first:

/tmp/jl_K84EE9/test

before falling back to creating its own CondaPkg.toml in the root directory?

[cjdoris]

We'd only want to include test/CondaPkg.toml if we're actually running the tests. I wonder if it's possible to check if the tests are being run?

What's actually in that folder? Is it a copy of your project? Is the Project.toml modified in any way by Julia's testing machinery?

[icweaver]

We'd only want to include test/CondaPkg.toml if we're actually running the tests

Do you mean that you'd like to be able to support multiple CondaPkg.toml files, one for the main project, then potentially one for the test directory if it exists? That'd be really cool!

What's actually in that folder? Is it a copy of your project?

hmm, it looks like it's only showing [".CondaPkg", "Manifest.toml", "Project.toml"] in the tmp directory even though CondaPkg.toml is located in the original test folder as well. This is just from printing it out when runtests.jl runs though, so I guess things will be different during precompilation? I just uploaded a quick skeleton package that I am trying this on here

Just fair warning though that I am firmly outside of my level of familiarity with these sorts of things now 😅

[icweaver]

I thing having genuine per subdirectory Project.tomls in the future could be really cool, especially combined with CondaPkg.jl's take on directory specific envs.

Until #1233 is merged, I think I am happy with just using the pre-Julia v1.2 standard, as your package currently does! By the way, I stumbled on this collapsed thread from #499 that discusses some of the current pitfalls with the more "modern" approach of separate Project.tomls that I had never considered before. Here's hoping for a much cleaner approach in the future. Thanks again!

[cjdoris]

Thanks, those are some really useful links to understand the test mechanism. Even better, I think we concluded I don't need to do anything to CondaPkg. I'll add a note to the readme about this.

[icweaver]

ah, on second thought it doesn't look like Julia is moving the test directory to the tmp directory

[cjdoris]

I'm not sure there will be a reliable way to do this automatically, given the testing machinery is a bit magic. The best solution is probably to copy YourProject/test/CondaPkg.toml into the tmp directory at the top of runtests.jl.