Load_ndcs module as separate runnable module

[yevgenybulochnik]

We should consider making the load_ndcs module separately runnable from the web server itself. This would avoid the current process of commenting out part of the run.py module.

[finish06]

An approach to doing this is separate folder / code structure for the load_ndcs & connect_db modules. Create a folder, e.g. ./build_data and place the files in the such folder. Then enable the load_ndcs module to run via command line and make a change to docker-compose so it is a separate container. I think if we intend to decouple the code, we need to make the folder structure foretelling. Thoughts?

[yevgenybulochnik]

makes sense, if we go the container route for loading data we likely need another docker-compose. maybe docker-compose.build-data.yml. This way we can specify (docker-compose up -f docker-compose.build.yml) when the data should be built. Otherwise on every docker-compose up it will run through the data. Another approach would be to build in some sort of check to see if the data has already been loaded.

[yevgenybulochnik]

Since we would be changing the dir structure of the project what dir and names make most sense?

build_data/
server/
database/

build_data/
server/
connect_db.py

or we could keep connect_db.py and just make a models specific dir

build_data/
server/
models/
connect_db.py

just thoughts, I always find structure and naming to be tricky not sure what is best @hl1 any thoughts on this

[finish06]

connect_db.py is only used by load_ndcs.py therefore I think it can stay in the same directory as load_ndcs.py. That may change in the future, but we can make changes when that happens. I prefer the following:

build_data/
server/
models/

[finish06]

I am struggling to import the helper function from the parent directory. Thoughts?