enable smooth environment variable naming transition by MoralCode · Pull Request #311 · chaoss/CollectOSS

MoralCode · 2026-05-08T20:52:34Z

Description
This PR introduces a new SystemEnv class that lives alongside SystemConfig and aims to centralize/unify all environment variable access.

The primary purpose of this is to slightly decouple the requested environment variable name from the name where the value is actually pulled from. For example, even if all in-code references use COLLECTOSS_DB for the database connection string, this layer will allow a values to be read from AUGUR_DB as a fallback if COLLECTOSS_DB (preferred) is not set.

This class is also set up to print a deprecation warning/notice whenever the AUGUR variant of an environment variable is used as a fallback. The message makes it clear that this fallback behavior is only temporary and may not be around forever (although I don't see this code being removed anytime in the foreseeable future)

This PR includes a unit tests for this new functionality and works as intended, including edge cases such as when the key the code requests doesn't use a known prefix (os.getenv("CONFIG_DATADIR"))

It also includes a special case function get_bool() because there were ~3 different places in the code where a small python one-liner (that i wrote) had been copied around for convenience, and this was a good opportunity to refactor that to prevent those three versions from ever drifting in implementation from each other. There are certainly more places this logic can be reused as well as we unify the way all env vars are processed.

Environment variables handled via Click (the CLI library) are addressed too. the click automatic prefix has been updated and the ~5 lines in 1 file where we were explicitly setting the variables are now updated to include both variants.

This PR resolves one of the large items that needs to be done to complete #279 for the next release.

Notes for Reviewers
This change includes unit tests for "backwards" variable access that keep the code able to refer to AUGUR_ variables so that those can be renamed later

This PR does not actually change any environment variable names, but that can be done either in this PR or a followup depending on the discussion results

TODOs:

Not yet run to full collection or tested much beyond the unit tests - this will need to be done before merge (ideally by multiple maintainers)
env var access in shell scripts (docker startup) also need adjusting to potentially fall back to the old names

Signed commits

Yes, I signed my commits.

github-actions · 2026-05-08T20:53:14Z

@@ -130,7 +131,7 @@ def start(ctx, disable_collection, development, pidfile, port):
    processes = start_celery_worker_processes((core_worker_count, secondary_worker_count, facade_worker_count), disable_collection)


[pylint] _{reported by reviewdog 🐶}
W0621: Redefining name 'processes' from outer scope (line 396) (redefined-outer-name)

github-actions · 2026-05-08T20:53:14Z

@@ -15,6 +15,7 @@
 import requests
 from redis.exceptions import ConnectionError as RedisConnectionError


[pylint] _{reported by reviewdog 🐶}
W0611: Unused ConnectionError imported from redis.exceptions as RedisConnectionError (unused-import)

github-actions · 2026-05-08T20:53:14Z

@@ -31,7 +32,7 @@

 from keyman.KeyClient import KeyClient, KeyPublisher


[pylint] _{reported by reviewdog 🐶}
W0611: Unused KeyClient imported from keyman.KeyClient (unused-import)

shlokgilda

Added some inline comments

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

…ode slowly over time Signed-off-by: Adrian Edwards <adredwar@redhat.com>

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

Co-authored-by: Shlok Gilda <gildashlok@hotmail.com> Signed-off-by: Adrian Edwards <17362949+MoralCode@users.noreply.github.com>

impact: low due to the new env variable interpretation layer Signed-off-by: Adrian Edwards <adredwar@redhat.com>

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

…s where env vars are explicitly needed in the CLI Signed-off-by: Adrian Edwards <adredwar@redhat.com>

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

+
+class TestExtractPrefix:
+    def test_env_extract_prefix(self):
+        assert extract_prefix("OTHER_DB", prefixes) == "OTHER_"


+class TestExtractPrefix:
+    def test_env_extract_prefix(self):
+        assert extract_prefix("OTHER_DB", prefixes) == "OTHER_"
+        assert extract_prefix("COLLECTOSS_DB", prefixes) == "COLLECTOSS_"


+        assert extract_prefix("COLLECTOSS_DB", prefixes) == "COLLECTOSS_"
+
+    def test_env_extract_prefix_default(self):
+        assert extract_prefix("SOME_DB", prefixes) is None


+
+    def test_env_extract_prefix_default(self):
+        assert extract_prefix("SOME_DB", prefixes) is None
+        assert extract_prefix("THINGY_DB", prefixes) is None


+
+
+    def test_env_extract_prefix_unprefixed(self):
+        assert extract_prefix("DB", prefixes) is None


+        assert SystemEnv.get("COLLECTOSS_MISSING", None, prefixes) is None
+
+    def test_fetching_env_default(self):
+        assert SystemEnv.get("COLLECTOSS_DEFAULT", "SOME", prefixes) == "SOME"


+    def test_no_known_prefix(self):
+        # fallback handling
+        os.environ["THING"] = "C"
+        assert SystemEnv.get("THING", None, prefixes) == "C"


+
+        for case in cases:
+            os.environ["OTHER_BOOL"] = case
+            assert SystemEnv.get_bool("OTHER_BOOL", False, prefixes) == True, f"value '{case}' should resolve to True"


+
+        for case in cases:
+            os.environ["OTHER_BOOL"] = case
+            assert SystemEnv.get_bool("OTHER_BOOL", True, prefixes) == False, f"value '{case}' should resolve to False"


+
+        for case in cases:
+            os.environ["OTHER_BOOL"] = case
+            assert SystemEnv.get_bool("OTHER_BOOL", False, prefixes) == False, f"value '{case}' should resolve to Default value"


Signed-off-by: Adrian Edwards <adredwar@redhat.com>

MoralCode requested a review from shlokgilda May 8, 2026 20:52

github-advanced-security AI found potential problems May 8, 2026

View reviewed changes

github-actions Bot reviewed May 8, 2026

View reviewed changes

MoralCode added this to the v1.1 Migration Release milestone May 8, 2026

shlokgilda requested changes May 10, 2026

View reviewed changes

MoralCode added the waiting This change is waiting for some other changes to land first label May 18, 2026

MoralCode added 18 commits May 20, 2026 16:08

add a deprefix helper and some unit tests

60024b1

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

refactor into a better prefix extraction helper

6735697

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

add first pass SystemEnv

a547500

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

allow different prefixes to be passed in for testing purposes

a0b3e20

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

update deprecation message

657cc53

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

basic functionality unit tests

578c4a3

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

handle cases with no known prefix to avoid breaking stuff

ac8fcef

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

factor out bool fetching class to deduplicate logic

6aa89a2

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

Test to make sure we can potentially migrate the AUGUR_ vars in the c…

5d33715

…ode slowly over time Signed-off-by: Adrian Edwards <adredwar@redhat.com>

replace all references to os.getenv with new class

1e07a3a

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

deprecate older Environment class that is buried in the module tree

af281ec

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

get_bool docstring

01c2fad

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

basic setter

f52626b

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

replace references to os.environ[] with new class

35845f1

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

remove some redundant wrapping code

ed673bf

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

replace references to os.environ.get

c72520e

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

Swap out only usage of deprecated Environment class

9ed52b8

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

Remove no-longer-used Environment class in API

a6070fd

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

MoralCode marked this pull request as draft May 20, 2026 20:27

MoralCode self-assigned this May 27, 2026

github-project-automation Bot added this to CollectOSS Feature Roadmap May 27, 2026

github-project-automation Bot moved this to Todo in CollectOSS Feature Roadmap May 27, 2026

MoralCode moved this from Todo to In Progress in CollectOSS Feature Roadmap May 27, 2026

MoralCode changed the title ~~introduce a SystemEnv class to unify environment variable use and enable smooth transition~~ enable smooth environment variable naming transition May 29, 2026

MoralCode and others added 14 commits May 29, 2026 17:43

Refactor extract_prefix

67bf777

Co-authored-by: Shlok Gilda <gildashlok@hotmail.com> Signed-off-by: Adrian Edwards <17362949+MoralCode@users.noreply.github.com>

refactor get_bool

ec58731

Co-authored-by: Shlok Gilda <gildashlok@hotmail.com> Signed-off-by: Adrian Edwards <17362949+MoralCode@users.noreply.github.com>

fix ~ path expansion in default scorecard value

b47a836

Co-authored-by: Shlok Gilda <gildashlok@hotmail.com> Signed-off-by: Adrian Edwards <17362949+MoralCode@users.noreply.github.com>

replace a bunch of env var names the application accesses

31f25c1

impact: low due to the new env variable interpretation layer Signed-off-by: Adrian Edwards <adredwar@redhat.com>

update env var names in RST docs

a9101b5

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

update vars in example env

dc7e090

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

update variable names in docker compose

fb7d113

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

hard change env var prefix for CLI commands

bf8b20c

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

attempt to add transitional variables for the specific existing place…

43c576a

…s where env vars are explicitly needed in the CLI Signed-off-by: Adrian Edwards <adredwar@redhat.com>

factor tests into a class

d382165

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

import SystemEnv into celery_app

a80349d

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

move test next to the actual known good config tests

aeb05b5

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

add more detailed failure reasons to get_bool tests

b340f65

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

split environment tests into two classes

1f89a32

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

MoralCode force-pushed the feat/flexible_env branch from 8041729 to 1f89a32 Compare May 29, 2026 21:44

github-advanced-security AI found potential problems May 29, 2026

View reviewed changes

MoralCode added 4 commits May 29, 2026 17:46

apply homedir resolution fix to SCC path as well

741fba0

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

fix docs underline lengths

de2fc9f

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

remove unused imports

6330627

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

use SystemEnv for fetching database variable

a023827

Signed-off-by: Adrian Edwards <adredwar@redhat.com>

		@@ -130,7 +131,7 @@ def start(ctx, disable_collection, development, pidfile, port):
		processes = start_celery_worker_processes((core_worker_count, secondary_worker_count, facade_worker_count), disable_collection)

		@@ -15,6 +15,7 @@
		import requests
		from redis.exceptions import ConnectionError as RedisConnectionError

		@@ -31,7 +32,7 @@

		from keyman.KeyClient import KeyClient, KeyPublisher



		def test_env_extract_prefix_unprefixed(self):
		assert extract_prefix("DB", prefixes) is None

Conversation

MoralCode commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot May 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot May 8, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shlokgilda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MoralCode commented May 8, 2026 •

edited

Loading